exporting alphas with the background

[TontonTremblay]

So right now computing the alpha with the background is slowing my rendering pipeline a lot.

The code looks like something like this:

        visii.sample_pixel_area(
            x_sample_interval = (0,1),
            y_sample_interval = (0,1)
        )
        dict_id = visii.entity.get_name_to_id_map()
        seg_all = np.zeros((cfg.height,cfg.width))
        for i in range(100):
            segmentation_array = visii.render_data(
                width=int(cfg.width),
                height=int(cfg.height),
                start_frame=0,
                frame_count=1,
                bounce=int(0),
                options="entity_id",
                seed = i
            )
            segmentation_array = np.array(segmentation_array).reshape(cfg.height,cfg.width,4)[:,:,0]
            segmentation_array[segmentation_array>3.4028234663852886e+37]=0
            segmentation_array[segmentation_array<3.4028234663852886e-37]=0
            segmentation_array[segmentation_array>0] = 1
            seg_all+=segmentation_array
        seg_all = np.flipud(seg_all)/100
        cv2.imwrite(f"{cfg.outf}/{str(i_frame).zfill(5)}.alpha.png",
            (seg_all*255).astype(np.uint8)
        )

Would it be possible to have a render_data file like "alpha_background" and it takes as input spp as well. Really want I want is pretty much antianliasing output from the render.

An other approach could be like blender does. Add the alpha blend to the rendered png as a flag in nvisii? https://blender.stackexchange.com/questions/1303/can-blender-render-pngs-with-the-background-transparent.

Thoughts?

[natevm]

I’m not sure I follow what you’re doing. Could you explain the problem you’re working on a bit more?

entity ID shouldn’t be averaged over multiple frames, since then you’ll get invalid IDs at the boundary of objects.

[TontonTremblay]

Here is an example, look at the alpha .png on the boundaries, they are not hard boundaries. They are soft, so I could do a nice masking with this and put it over something else. But really this is used to train nerf. Check the link I shared to see how blender does it. You could imagine that this alpha is added to the final png by adding a flag.

Also look how I put the background at 0 and the rest at 1 then take the average to generate the smooth boundary.

[natevm]

Ah, I see. So you’d like the background color to be alpha transparent for primary rays? What should the behavior be for a mirror reflecting a transparent background?

[TontonTremblay]

Not really! What I want is given a pixel how many rays in that pixel hit the background vs. how many hit any geometries.

So for a given pixel you can do a sum(hit_geometry)/num_of_rays. You can also think about it as computing anti-anliaising for that given pixel.

[natevm]

sum(hit_geometry)/num_of_rays

what you are describing is exactly what I just said. 😅 We call this alpha transparency, and it’s done by setting the background color to something transparent.

Sounds like you don’t need anything fancy yet for secondary rays.

[TontonTremblay]

This has been added. Pass the alpha value to the background in the domelight or in the hdri 4th channel.

[esx2ve]

Hi,

I also ran into the issue of wanting to generate object masks and they're not antialiased. I did not understand the proposed solution: would it be perhaps possible to have some example code? Thank you!