Question: Export in JPG

[wizardjo]

Hi amazing team,

I have a very quick question: There is any way to export the RGB and Segmentation images in JPG format ?

Thank you for your help.

[sleal-unity]

Hi @wizardjo!

At the moment, the Perception Camera does not have any image encoding options besides PNG. However, in our next release, we plan to update our Labelers to use faster encoding algorithms recently added to the Unity capture package to improve performance. During this process, I think it's likely that we could sneak in the ability to encode captured RGB images with JPEG. Segmentation images would likely remain PNG only though considering that image compression would introduce artifacts that would disturb the semantic color to label mapping.

[wizardjo]

Thank you @sleal-unity

[jjwebb]

@wizardjo -- I implemented this feature in a forked branch off 0.8.0.preview.4_staging because I wanted the same thing. You can find it here. A checkbox to save RGB data as JPG will appear when Save Camera RGB Output to Disk is enabled: Segmentation data is still saved as PNG as it's the better suited format for those images for the reason sleal-unity mentioned and because the difference in file size is negligible.

@sleal-unity I can submit a PR for this if you would like, though it sounds like you're planning to implement it differently. As a matter of curiosity -- what's the reason for having the default for RGB be PNG? The images generated seem to be around ten times larger than a comparable JPG, with a really minute difference in image fidelity. Given that we're presumably simulating real-world image data, we wouldn't be expecting pixel-perfect representation from camera captures in the first instance 🤔

[wizardjo]

@wizardjo -- I implemented this feature in a forked branch off 0.8.0.preview.4_staging because I wanted the same thing. You can find it here. A checkbox to save RGB data as JPG will appear when Save Camera RGB Output to Disk is enabled: Segmentation data is still saved as PNG as it's the better suited format for those images for the reason sleal-unity mentioned and because the difference in file size is negligible.

@sleal-unity I can submit a PR for this if you would like, though it sounds like you're planning to implement it differently. As a matter of curiosity -- what's the reason for having the default for RGB be PNG? The images generated seem to be around ten times larger than a comparable JPG, with a really minute difference in image fidelity. Given that we're presumably simulating real-world image data, we wouldn't be expecting pixel-perfect representation from camera captures in the first instance 🤔

Thank you Julian, I will check it

[ChumbaWambalala]

was this added to latest release?

[JonathanHUnity]

No, output format was not added in this release unfortunately.

[CharalamposGeorgiadis]

@wizardjo -- I implemented this feature in a forked branch off 0.8.0.preview.4_staging because I wanted the same thing. You can find it here. A checkbox to save RGB data as JPG will appear when Save Camera RGB Output to Disk is enabled: Segmentation data is still saved as PNG as it's the better suited format for those images for the reason sleal-unity mentioned and because the difference in file size is negligible.

@sleal-unity I can submit a PR for this if you would like, though it sounds like you're planning to implement it differently. As a matter of curiosity -- what's the reason for having the default for RGB be PNG? The images generated seem to be around ten times larger than a comparable JPG, with a really minute difference in image fidelity. Given that we're presumably simulating real-world image data, we wouldn't be expecting pixel-perfect representation from camera captures in the first instance 🤔

@jjwebb Sorry for the intrusion, but how do I add this to my existing perception package project?

[jjwebb]

@XarisGeorgiadis It's been a number of months since I touched this and I'm not sure I added the package in the "right" way, but here is what I did. Clone my version of the repository, then open your Unity project and open the Packages folder. Open manifest.json and change the "com.unity.perception" value to "file:\<filepath to cloned repo>". In my case the line now reads "com.unity.perception": "file:../PerceptionPackage-Git/com.unity.perception/com.unity.perception",. Save and open the packages-lock.json file and change "version" in "com.unity.perception" to the same filepath and "source" from "registry" to "local". If this is not changed, Unity will overwrite the new package with the version from the registry. Again, this may have been a dumb way to do it and you can probably just add the package from the Unity UI, but this was how I did it.

[CharalamposGeorgiadis]

@XarisGeorgiadis It's been a number of months since I touched this and I'm not sure I added the package in the "right" way, but here is what I did. Clone my version of the repository, then open your Unity project and open the Packages folder. Open manifest.json and change the "com.unity.perception" value to "file:". In my case the line now reads "com.unity.perception": "file:../PerceptionPackage-Git/com.unity.perception/com.unity.perception",. Save and open the packages-lock.json file and change "version" in "com.unity.perception" to the same filepath and "source" from "registry" to "local". If this is not changed, Unity will overwrite the new package with the version from the registry. Again, this may have been a dumb way to do it and you can probably just add the package from the Unity UI, but this was how I did it.

@jjwebb Your way worked so I do not really care if it is a "dumb" way or not :p. Thank you so much! As a follow-up question: Is there a file or script that I can modify so that I can reduce the amount of compression from png to jpg, since the drop-off in quality is a bit too much?

[jjwebb]

@XarisGeorgiadis I have just added this for you, just pull the latest changes from my branch into your folder :^)

[CharalamposGeorgiadis]

@jjwebb Wow, you are amazing. Thank you so much!

[Yonni123]

how do I add it from unity UI? I tried changing the manifest.json and the package lock but no results

[CharalamposGeorgiadis]

@Yonni123 I did not add it from Unity's UI. I uninstalled Perception from the Package Manager, downloaded jjwebb's modified one from https://github.com/jjwebb/com.unity.perception/tree/jpg-support and put the com.unity.perception folder inside my Packages folder. I then modified manifest.json and packages-lock.json accordingly.

Maybe after uninstalling the default Perception Package from the Package Manager you can try to Add package from disk and select the modified one that you have downloaded, but I can not confirm that this works since I have not tried it.

[prasetion]

I have already add the package but get error result

com.unity.perception-jpg-support\com.unity.perception\Runtime\GroundTruth\RenderPasses\CrossPipelinePasses\GroundTruthCrossPipelinePass.cs(55,19): error CS0246: The type or namespace name 'RendererListDesc' could not be found (are you missing a using directive or an assembly reference?)