Holdout reflections with RectLight not showing in alpha channel

[Corefrecs]

Hi, i can't get Holdout reflections to work when just using RecLights and rendering to IT or using batch render. The Alpha channel just contains the shadow part but not the reflections.

First image is the scene without a holdout, second with holdout and third the alpha channel with holdout.

when using a DomeLight or EnvDayLight everything works as expected and the reflections show up in the alpha channel. Is this intentional or am i missing something?

With an EnvDayLight:

but when rendering directly to blender it works again, i think because the alpha is directly baked into the beauty pass?

Here are two simple szenes example.zip

[prman-pixar]

Hmm...I'm not sure what's going on here. I don't think this is a problem with the addon itself, as the same thing seems to happen in our Maya and Houdini plugins.

It seems to work in our regression tests, so I'm still trying to figure out what our DCC plugins are doing incorrectly.

[prman-pixar]

@Corefrecs

I'm still not sure why you're not seeing the reflection in the alpha, but I think if you want to composite this over a plate the best way to do this would be:

1. Change the Render Holdouts from In Alpha to Separate AOV
2. In your compositor take the alpha of the main image and invert it.
3. Take the separate hold out AOV and invert that as well
4. Multiple the inverted alpha and inverted holdout AOV
5. Bring in your plate and multiply that with the result from step 4
6. Use an Add node and add together the beauty and the result from 5.

Here's a screenshot of the Blender compositor. It might be a bit hard to see, but there are reflections from the render.

[Corefrecs]

Thanks for investigating this! Your suggestion works quiet well and is a good workaround.

I've tried a little bit further and as soon as two RectLights are involved, strange things happen with the alpha channel (first image without holdout, second image the alpha channel with holdout):

Adaptive sampling was deactivated, thought first it had something to do with it.

This pattern is also visible in shadows without holdout and very low samples:

So I'm not sure if it is related to holdouts or they just exaggerate this.

[Corefrecs]

Update on this: the strange Alpha is caused by "Learn Light Results" under Advanced. Sry if this is known, i'm fairly knew to Renderman. Unfortunately this doesn't fix the missing reflections.

[prman-pixar]

re: learn light results

Yes, it's best to turn this off when you're rendering holdouts. Glad you were able to figure this out

re: reflections

I can't reproduce what you're seeing when I add a second rect light to the scene that you sent before. Looking at the image, it looks like the second light might be really hot, so it's overblowing everything to the point that the reflections disappear?

[Corefrecs]

Yes you're right, i boosted the lights and brightened the materials in this scene to better show the noise pattern in the shadows. That test was purely for the strange alpha without paying attention to the reflections, i'm sorry for the confusion!