Empty PNG output with PostProcess Material used after Tonemapping

[krzysztoffudala-digica]

Describe the bug

png output are empty for all of types of renders.

To Reproduce

Steps to reproduce the behavior:

1. Check main branch.
2. Create new unreal scene.
3. Add any object (sphere, cube)
4. Add new sequencer with camera
5. Use the plugin as stated in manual
6. The png output is empty -> probably transparency is 100% (not sure) (this is example image)

Configuration (if applicable):

• OS: Windows 10
• GPU: Nvidia 3070
• Unreal: 4.27 Additional context The problem is connected with the tonemapper. When I change the mode of PPMaterial from "After Tonemapper" to "Before Tonemapper", the output exists. The problem is the same for the setting "Replace Tonemapper". Have you encountered the issue by yourself?

[NikolaJov96]

Hi, thanks for reaching out! Here is the explanation based on my understanding of your situation.

You are right about the transparency, here is your image with the alpha channel removed (you may do this using Gimp or a simple Python script):

You are also right that this is a tone mapper issue. Unfortunately, the After Tonemapper setting is needed so that the precise color outputs are not "stylized" by the said tone mapper. The second part of the problem is that, if I remember correctly, post-process material outputs cannot export the alpha channel. Because of this UE does not write anything to the alpha channels of file types that support them (such as PNG), so we are left with transparent images.

Sorry for the inconvenience of having to remove the alpha channel manually, I can see that this can be very confusing. Will mark this as a to-do in the next release.

[krzysztoffudala-digica]

Thanks!

[krzysztoffudala-digica]

@NikolaJov96 I have found the solution for the missing PNG. You need to set Output Alpha to true and add Constant Node with 0 value.

[krzysztoffudala-digica]

@NikolaJov96 I have other issue that you may have encountered in the past - masks have not perfect color for each pixels, they slightly differ. This is huge problem for me as I try to use rendered images to create COCO datasets and use the color to mark object instances. Do you know what the issue could be or what are the workarounds?

[NikolaJov96]

@krzysztoffudala-digica thanks for the PP material fix suggestion, I did try messing with these settings but could not fix the problem (I might have assumed opacity should be 1 instead of 0 :thinking:). I should be able to have these fixes in master next week.

I see, there are some pixels very close to valid colors, but not exactly right. We did not use the semantics rendering internally for training yet, so we did not bump into this issue, although I am not surprised that there are color imperfections coming from UE. Unfortunately, I don't have any advice on this. In some similar cases (unrelated to EasySynth) I had to create a script to go through all of the images and round all pixels to the nearest valid color, I don't see an easier solution here. :disappointed: