Psyop / Cryptomatte

Cryptomatte Nuke plugin, Fusion plugin, sample images, and specification
BSD 3-Clause "New" or "Revised" License
633 stars 151 forks source link

Can the Cryptomatte channels be maintained if re-rendered out of Nuke? #42

Closed fredrikaverpil closed 6 years ago

fredrikaverpil commented 6 years ago

I'm rendering multichannel EXRs from V-Ray and I can successfully pick my objects using the Cryptomatte Nuke plugin (v1.1.4).

I would now like to render this out from Nuke into another EXR file, still maintaining the (otherwise untouched) Cryptomatte channels. It seems this is possible only if I set the "datatype" to "32-bit" float and also setting "metadata" to "all metadata" in the Write node. If I instead choose "16-bit half", the re-written EXR cannot be used with the Cryptomatte Nuke plugin.

Questions

  1. Is it not possible to render out the EXR in 16-bit half from Nuke and maintain the Cryptomatte channels?
  2. Would it be considered safe to re-render the EXR in 32-bit, like described above, with reliable results?

EDIT: So, yes, using e.g. Nuke 10.5v6 I can render out 32-bit float EXR files and still maintain the Cryptomatte channels perfectly fine. You also need to set metadata to "all metadata" in the Write node. Nuke can not write out a "mixed bit-depth" EXR, unfortunately, so even if many channels are 16-bit half, I'll have to write out a fully 32-bit float EXR from Nuke.

AndrewHazelden commented 6 years ago

Hi Fredrik.

I would now like to render this out from Nuke into another EXR file and still maintaining the (otherwise untouched) Cryptomatte channels. It seems this is possible only if I set the "datatype" to "32-bit" float and also setting "metadata" to "all metadata" in the Write node. If I instead choose "16-bit half", the re-written EXR cannot be used with the Cryptomatte Nuke plugin.

With V-ray 3.6 based Cryptomatte renderings coming out of your 3D package, it is possible to have an EXR image that stores mixed bit depths in the same file.

The RGB "beauty" channel can be set to either 16 bit Half-float or 32-bit float per channel image data.

V-Ray will automatically set the Cryptomatte matte channels in the EXR image to be rendered at full 32-bit float depth regardless of any options you set in your 3D package.

To the best of my knowledge, Cryptomatte always needs its matte channels to be stored as full floating point 32-bit channel image data to function correctly. This means you should never intentionally degrade the bit depth of those "cryptomatte##.R,G,B" image channels from 32-bit down to 16-bit during your Cryptomatte centric compositing tasks in your compositing package.

For the benefit of anyone else reading this GitHub issues post, here is a sample V-ray for Maya Cryptomatte EXR rendered image I created that shows this concept of the mixed bit depths in the same file:

crypto_vray_balls_test.exr

If you open up this V-ray rendered EXR image in a tool like DJV_View and browse through the image channels you will notice this mixed bit depth thing happening.

This is a screenshot of the "crypto_vray_balls_test.exr" image loaded in DJV View with the RGB channels visible. On the bottom right of the window is the text "RGBA F16" which means you are viewing a half-float 16 bit per channel image.

vray-rgb-16-bit-channels

If you go to DJV Viewer's File > Layer menu and select the "Cryptomatte.R,G,B,A" channel in the image you will see the text on the bottom right of the window is showing "RGBA F32" which means you are viewing the Cryptomatte based matte channels as full 32 bit per channel floating point images:

vray-cryptomatte-32bit-channels


I'm not a Nuke user so I can't comment on best practices but I'd have to guess that you'd likely need to render out to a full 32-bit per channel output for all image channels in the Nuke Write node if you want your RGB beauty pass images to be stored at the same bit depth as the Cryptomatte 32-bit float channels are natively.

The Cryptomatte for Nuke developers can chime in here and clarify that point.

Regards, Andrew Hazelden

fredrikaverpil commented 6 years ago

Thanks for clearing this up. 👍 I actually suspected this, but wasn't sure. Let's see what the plugin devs have to say about it.

jonahfriedman commented 6 years ago

Thanks Andrew, for giving that detailed answer with pictures even! All of it is correct, including the part at the end about having to write out all channels at the same bit depth because that's what the Nuke write node provides.

We should keep a list of answered questions, especially with thorough answers like the above.

AndrewHazelden commented 6 years ago

Hi Jonah.

It’s great to hear your feedback on the EXR bit depth issue.

I was wondering if I had understood the concept correctly on how Cryptomatte handles the matte channel data or not and you helped clarify things. Thanks!

I don’t know if you guys are willing to turn ON the GitHub Wiki section for the Cryptomatte project. But I’d be happy to start writing up some content in the Wiki module to build out a knowledge base out of the existing tips and solutions that were mentioned and discussed by the Cryptomatte users and developers in the GitHub issues section here.

Cheers, Andrew

jonahfriedman commented 6 years ago

@AndrewHazelden

Sounds like a good idea. I've enabled the wiki, if you'd like to start messing with it.

I'm not sure how to split the documentation between the repo and the wiki. I guess that's an issue for future conversations - there's value in having it version controlled with the plugin, but it's also not good to have too much redudancy. Perhaps basic docs in the repo, everything else in the wiki? I'm not sure.

Anyway, the Wiki is editable now, have at it! I'm going to close this ticket.

fredrikaverpil commented 6 years ago

Regarding a wiki; my personal experience is that it is much easier to maintain one large README.md file instead of the Wiki. Or split it up into a couple of .md files. This makes it much easier to cooperate on and maintain (and backtrack) as changes to it can be included in PRs and reviews.

Andrew, it's so nice of you to take your time to start working on documentation!

AndrewHazelden commented 6 years ago

Hi Fredrik.

I'm planning to write the content initially as a folder full of .md files with lots of small cropped images attached for screenshots + examples. Then that information would be pushed into the GitHub wiki module as a 2nd stage for easier viewing on the web.

fredrikaverpil commented 6 years ago

Then that information would be pushed into the GitHub wiki module as a 2nd stage for easier viewing on the web.

It sounds like you mean you can actually push git-managed .md files into the github wiki in some clever fashion. Which does sound great. Is that what you mean (and if so, I'd like to know more)?

fredrikaverpil commented 6 years ago

@jonahfriedman I'm trying to figure out if Nuke can somehow detect the bit-depth of all channels in an EXR (using Python). Do you know if this is at all possible?

AndrewHazelden commented 6 years ago

@fredrikaverpil Right now it would be less then 20 different Wiki pages I'd be creating so I'd just go for a copy/paste approach from the 100% pre-composed markdown files on disk into the webpage editor view. I'd double check there have been no edits on the online version first with the revision history.

I'm sure the GitHub API provides better controls but I haven't gone there yet. :-)

tokejepsen commented 6 years ago

Looks like you should be able to pull/push to the wiki; https://help.github.com/articles/adding-and-editing-wiki-pages-locally/

I'm with @fredrikaverpil on having the documentation in the repository instead and use the README.md as a landing page for linking to different md files.

AndrewHazelden commented 6 years ago

@tokejepsen The content is all markdown files either way so that is a decision the Cryptomatte developers can make on how they'd prefer it to be published in the end.

AndrewHazelden commented 6 years ago

@tokejepsen Thanks for the GitHub help article link! That will make things a lot more streamlined and efficient. It's great to know that approach is possible.

tokejepsen commented 6 years ago

No problem:grinning: Think I tried doing this once, but it became too troublesome. If the README.md is enough, then maybe readthedocs is worth looking into.

AndrewHazelden commented 6 years ago

I hadn't heard of readthedocs before. Nice.

It would be overkill for my humble efforts but for far more complex user guide/book publishing projects there are even tools like GitBook, that have a lot of controls and optionally PDF formatted output.

TouchDesigner had some really nice user generated guides done using GitBook.