Support DaVinci Resolve MediaIn Node

[cedricduriau]

Currently the Fusion Cryptomatte Node cannot be used in DaVinci Resolve due to the input requirements of the fuse, which is the EXR file path inside the input image metadata. Inside Fusion this comes from a Loader, but in Resolve it comes from a MediaIn which does not hold the same metadata keys.

ERROR:

...ci Resolve\\Fusion\Modules\Lua\cryptomatte_utilities.lua:1092: ERROR: could not read EXR data (EXRIO ERROR: Cannot read image file "". Invalid argument.)
stack traceback:
    [C]: in function 'error'
    ...ci Resolve\\Fusion\Modules\Lua\cryptomatte_utilities.lua:1092: in function 'get_exr_layer_images'
    ...sign/DaVinci Resolve/Fusion/Fuses/Matte/cryptomatte.fuse:498: in function <...sign/DaVinci Resolve/Fusion/Fuses/Matte/cryptomatte.fuse:474>
MediaOut1 cannot get Parameter for Input at time 0

[cedricduriau]

@AndrewHazelden: I tried quickly and I cannot access the MediaIn node information because the code is contained within the Fuse. I can't just go about parse the node inside the comp, goes against the sheer nature of nodal based systems as well. I made a post on the form to address this issue. It's still in review but once I have the URL I'll edit it in here.

Let's see if the devs can help us out on this one.

Cheers Cedric

[AndrewHazelden]

Hi Cedric.

Let's see if the devs can help us out on this one.

If you would like to see this metadata Filename issue fixed before SIGGRAPH 2018 happens I'd personally like to suggest you look up the Peter Chamberlain (Resolve Product Manager) email address in Kristof's BMD contact Google Sheet. Also do take the time to CC Anish on the message (who is also listed in the BMD contact Google Sheet).

As far as I can tell, Peter Chamberlain has the sole ability to set the priority for what changes and bug fixes are implemented in Resolve's Fusion page in the remaining ~2 week lead up to SIGGRAPH.

If you mention in your email that you want to see the same Cryptomatte.fuse file work seamlessly across Fusion Standalone 9 and Resolve 15 in time for the SIGGRAPH official presentation talk on Cryptomatte you have a much better chance of seeing this issue fast tracked. 😀

I think this metadata fix can realistically happen if you send an email tonight or on Sunday at the latest. Peter Chamberlain and Anish are in Singapore, so timezone wise an email you send this weekend would arrive on their desk the moment they start work on Monday.

---

If you are still at GRID, you or GRID's person who is on the BMD Fusion beta program should try to meet Steve Roberts, or Peter Chamberlain at SIGGRAPH to see what can be done to improve / solve the remaining Cryptomatte issues like the fu:PurgeCache() memory leak bug before too much more time passes. If you can't get someone from GRID to chat with Steve or Peter to express why this is important for VFX artists, I really have no idea when things would get better.

Failing to reach out to BMD directly before SIGGRAPH, I'd likely guess the only way to get the Cryptomatte memory leak fixed and the Resolve 15 support to work is:

• To wait several more months (on top of the time you've waited since the Fu 9.0.2 release in Dec 2017) for things to get better on their own.

or

• Do something more drastic and change your code to manage the EXR image sequence loading process via EXRIO yourself.

IMHO the available workarounds would be if you made a new EXRIO CryptoLoader node, or if you added an (optional) Filename field in a nest control to the Fuse GUI in addition to the standard image input connection on the Cryptomatte.fuse.

A Filename slot could allow the Cryptomatte node to (optionally) load the EXR multi-channel/multi-multipart data itself. Doing your own Filename field is likely something you wouldn't like but in theory that node could feed the rest of the Crypto nodes downstream with the Filename metadata tag info formatted just how you like it so you could remain happily nodal.

From my view of things, pitching the Loader node input requirement in Cryptomatte would (in theory) instantly fix the memory leak and make Resolve support work too. If you have better suggestions I'd be very interested to hear them. I've been pondering these Cryptomatte issues and the possible solutions from the first day I got my hands on a Resolve beta.

Good Luck.

Regards, Andrew

[AndrewHazelden]

I made a post on the form to address this issue. It's still in review but once I have the URL I'll edit it in here.

If you are taking about posting the issue on the BMD forums, then you are possibly going to the wrong place for answers.

It is primarily there for user-to-user support.

SdR and Bryan might be able to provide some advice on the BMD Forums for you but BMD devs rarely seem to attend. I give you a near zero percent odds chance to get a comment on the BMD forums from a knowledgeable Fusion page dev. 🤪

[cedricduriau]

HI Andrew

If you would like to see this metadata Filename issue fixed before SIGGRAPH 2018 happens I'd personally like to suggest you look up the Peter Chamberlain (Resolve Product Manager) email address in Kristof's BMD contact Google Sheet. Also do take the time to CC Anish on the message (who is also listed in the BMD contact Google Sheet).

Yeah you're right, especially considering the forum board has to approve my posts... Just checked the sheet, got their details. I will mail them and keep you posted.

As far as I can tell, Peter Chamberlain has the sole ability to set the priority for what changes and bug fixes are implemented in Resolve's Fusion page in the remaining ~2 week lead up to SIGGRAPH. If you mention in your email that you want to see the same Cryptomatte.fuse file work seamlessly across Fusion Standalone 9 and Resolve 15 in time for the SIGGRAPH official presentation talk on Cryptomatte you have a much better chance of seeing this issue fast tracked. 😀

Well, I hope you're right, but you mostly are so let's hope for the best haha!

I think this metadata fix can realistically happen if you send an email tonight or on Sunday at the latest. Peter Chamberlain and Anish are in Singapore, so timezone wise an email you send this weekend would arrive on their desk the moment they start work on Monday.

It sounds like such an easy fix! Hope we can get the word across fast enough and that they are able to make that happen.

If you are still at GRID, you or GRID's person who is on the BMD Fusion beta program should try to meet Steve Roberts, or Peter Chamberlain at SIGGRAPH to see what can be done to improve / solve the remaining Cryptomatte issues like the fu:PurgeCache() memory leak bug before too much more time passes. If you can't get someone from GRID to chat with Steve or Peter to express why this is important for VFX artists, I really have no idea when things would get better.

Well, let me put it this way, I have already met a few BMD officials in person which I prefer not to name publicly. That was MONTHS ago, after that we had full radio silence until recently. Again, can't/won't say much more about this, but let's put it this way, I will know some more real soon.

Failing to reach out to BMD directly before SIGGRAPH, I'd likely guess the only way to get the Cryptomatte memory leak fixed and the Resolve 15 support to work is:

• To wait several more months (on top of the time you've waited since the Fu 9.0.2 release in Dec 2017) for things to get better on their own.

or

• Do something more drastic and change your code to manage the EXR image sequence loading process via EXRIO yourself.

Well I am aiming the Resolve 15 support, but not particularly the memory leak fix, for SIGGRAPH that is. I have stated my opinions on this before, so I won't do it again.

IMHO the available workarounds would be if you made a new EXRIO CryptoLoader node, or if you added an (optional) Filename field in a nest control to the Fuse GUI in addition to the standard image input connection on the Cryptomatte.fuse.

A Filename slot could allow the Cryptomatte node to (optionally) load the EXR multi-channel/multi-multipart data itself. Doing your own Filename field is likely something you wouldn't like but in theory that node could feed the rest of the Crypto nodes downstream with the Filename metadata tag info formatted just how you like it so you could remain happily nodal.

Well you're right on the fact that I do not like adding a filename field inside the fuse itself. Then the crypto loader would make more sense. But this whole workaround to me makes no sense. We're talking about shipping an extra node to do something the fuse already can in Fusion, thanks to a single value/metadata key. IMHO We're better off just checking what the Resolve devs have to say about this. It would highly surprise me that they tell us there is no way at all to get the path of a MediaIn node source from within the API (not the scripting API).

From my view of things, pitching the Loader node input requirement in Cryptomatte would (in theory) instantly fix the memory leak and make Resolve support work too. If you have better suggestions I'd be very interested to hear them. I've been pondering these Cryptomatte issues and the possible solutions from the first day I got my hands on a Resolve beta.

I don't follow you on this one, it would "fix" the Resolve support, by chaning or the fuse or the workflow, but what does the loader have to do with fixing the memory issues? It would be the exact same scenario to my knowledge, custom fuse with exact same EXRIO code, no?

If you are taking about posting the issue on the BMD forums, then you are possibly going to the wrong place for answers.

It is primarily there for user-to-user support.

SdR and Bryan might be able to provide some advice on the BMD Forums for you but BMD devs rarely seem to attend. I give you a near zero percent odds chance to get a comment on the BMD forums from a knowledgeable Fusion page dev. 🤪

Well I've seen some posts where Peter Ch. was replying, but I am afraid of the same thing. At least now it's public, they cannot blame in the future me for not having done that haha.

Cheers Cedric

[AndrewHazelden]

Hi Cedric.

I really do appreciate you considering doing some Cryptomatte compatibility testing with Resolve. Thanks!!!

---

I thought over what you said about the MediaIn node's Filename issue and I came up with a (very temporary) stop-gap macro that would at least let the Cryptomatte for Resolve testing go further while you wait for feedback from Resolve's product manager (and then by extension the downstream Fusion page devs on the Resolve team that would implement the change.)

MetadataFilename Macro

The "MetadataFilename.setting" macro is kinda simple and dumb but it should help you go further to see what other issues are present for getting Cryptomatte to work in the Resolve Fusion page operating environment.

The MetadataFilename macro internally wraps a "Set Metadata" fuse with the following settings:

Set Metadata:

FieldName = Filename FieldValue = </YourFilepath/YourImage.exr>

The Set Metadata.fuse's "FieldValue" in the MetadataFilename macro is the only exposed control and it has an onscreen GUI name of "Filename".

MetadataFilename Macro Download Link

http://www.andrewhazelden.com/projects/wesuckless/cryptomatte/MetadataFilename.setting

MediaIn Metadata

Would you be able to take a look at this Fusion page screenshot and let me know if you see the extra minimal Metadata tags in the Viewer windows' SubView overlay that is needed for a happy Cryptomatte:

Adding the Set Metadata fuse to Resolve

In Resolve, you can add the required "Set Metadata.fuse" dependency quickly and easily using Reactor's "ResolveEssentials" atom package which is located in the "Resolve" category.

Regards, Andrew

[AndrewHazelden]

Hi again.

To take the Resolve testing process further, I manually installed the Cryptomatte version that can be found in Reactor by hand. (The manual install was required to work around the current Resolve version compatibility tag based installation block that was added to the Cryptomatte atom package so it only was selectable by Fusion Standalone 9.0.2 users).

Here are some results from my Cryptomatte in Resolve test this morning:

Step 1. I loaded this V-Ray for Maya rendered Cryptomatte EXR image into the Resolve 15 Beta 7 Fusion page comp using a MediaIn node.

Step 2. I assigned the MetadataFilename macro that is available on this GitHub issues thread. In the Filename field on the macro, I pasted in the full filepath to the EXR image on disk.

Step 3. I then copy/pasted a Cryptomatte node that I set up in advance in Fusion Standalone into the Resolve Fusion page composite and connected the Cryptomatte node to the MetadataFilename node.

(I had previously enabled theCryptomatte settings [x] Matte only and [x] Keyable Surface options in Fusion Standalone before I pasted this node into Resolve. I had also picked the right sphere in the EXR test image using the Matte Locator cursor and then pressed the Add button / Toggle hotkey to add the Locater selected item to the Matte List text field.)

Step 4. I enabled a viewLUT in the two Resolve viewer windows so I could preview the Linear Gamma 1.0 EXR images as sRGB Gamma 2.2.

Step 5. When viewing the Cryptomatte output in Resolve it looks like there is a DoD window/proxy issue type of thing happening where the image is scaled down to like 1/10th its size and placed into the bottom left corner of the viewer context.

---

I haven't downloaded and install your newest GitHub Cedric Cryptomatte repo Dev branch files yet that have been updated in the last few days so possibly you recent code changes would solve the DoD/proxy issue I am seeing?

I hope these notes and screenshots might be of some help. I'd really love to see the same Cryptomatte fuse working in both Fusion Standalone and Resolve 15 by SIGGRAPH if possible.

Cryptomatte in Resolve 15 Beta 7 Screenshot

Cryptomatte in Fusion Standalone 9.0.2 Fusion Standalone

Closing Comments

The reason I was commenting earlier that you might consider changing how your fuse's Loader node dependency works is that it really might help fix/mitigate the Fusion purge cache memory issue. The logic behind that statment was:

From my view of things, you are (possibly) causing Fusion Standalone to double read in and RAM Cache every single frame of the EXR imagery TWICE each time a EXR image sequence frame is displayed in the Fusion viewer window or rendered and Cryptomatte is connected to the Loader node.

Why am I saying this?

1. The EXR imagery is first loaded in from disk in the Loader node just so you can read the textual Filename string and a few extra metadata tags that your EXRIO fuse could easily read into the metadata Lua table by itself.

2. The Cryptomattefuse then loads the EXR imagery again from disk using EXRIO a 2nd time. Fusion then RAM caches the frame inside the fuse. This 2nd EXR image read operation that is done inside your fuse fully doubles up on the network/disk image reading load and the Fusion viewer/ RAM caching usage.

As best as I can tell, there is nothing the Loader node's initial image reading operation is doing that your EXRIO based fuse couldn't do in 1/2 the disk bandwidth usage if you optimized your workflow. I fully admit in advance that I could totaly be wrong but I kinda think I'm not.... ;)

This closing comments section is possibly me incorrectly reading the situation and assuming that disk/network performance and system resource usage is something you care about as a TD/coder? I do know you like compact and beautiful code though... Haha. ;)

You could easily test this theory and prove me wrong by:

Loading a really really large EXR frame into Fusion that was like 50MB+ in size per frame.

Watch your operating systems disk/network throughput control panel to check and see if your Cryptomatte "input requirement" really is consuming ~2x the disk throughput to load the EXR twice then if a more efficient and less wasteful approach was considered and implemented. Also check the Fusion RAM cache memory usage indicator to see if a noticeably higher amount of memory was consumed too.

I'd be interested and very curious to hear what your own tests on this matter report back. :D

[AndrewHazelden]

Hi Cedric.

I wanted to suggest that if you added this code around the Cryptomatte Filename metadata tag reading code in your Cryptomatte fuse you would make it possible for more complex scripted tools to work down the road that would handle relative PathMaps with ease:

filename = self.Comp:MapPath(filename)

Cryptomatte for Fusion is very rigid on how it treats the Filename metadata tag so an entry with a PathMap in it (coming from an EXRIO based tool other then a classical Loader node) causes a Fusion Console error like this to happen:

ERROR: Cannot read image file "Comp:/Media/crypto_vray_balls_test.exr". No such file or directory.

This small change to add Fusion style PathMap support would allow the EXRIO based Reactor delivered LifeSaver.fuse to add a new "[x] Read Image" type of option down the road that would let this Saver node replacement tool act as a Loader node too when required by an artist.

The LifeSaver node already has a Filename field and outputs a Filename metadata tag. This means it could potentially be put into service for loading Cryptomatte EXR images (in place of a Loader node or Cryptomatte node) inside of Fusion or Resolve if I added a Read/Write switchable option to my fuse that was similar to what can be done in Nuke.

[AndrewHazelden]

For edge case reference, this image shows what you get in Resolve if a MediaIn > LifeSaver fuse > Cryptomatte series of nodes are chained together. The result of the workaround is a working copy of Cryptomatte v1.2.1 (WSL Reactor Edition) running in Resolve 15 beta 7.

The LifeSaver v1.0.4 fuse has the Format > Save Frames > None ComboControl menu setting active, and the same base Filename setting specified as the MediaIn node would be using to load footage from the Media Pool.

Then the timeline sequence frame updating Filename metadata tag is embedded into the Resolve image stream by the LifeSaver fuse. As a bonus of having the LifeSaver fuse in the comp, Cryptomatte has a working selectable Matte Locator control and DoD/Proxy frame size as a side effect of LifeSaver pre-digesting the imagery for it via the LifeSaver fuse's Preview Input Connection control.

It appears that the LifeSaver fuse > Cryptomatte approach correctly digested and passed the Resolve Fusion page based frame size info correctly to the Viewer window output in a way that Cryptomatte node doesn't by iteself. In the screenshot at the top of this post, you can see the view bounds are correctly rendered by Cryptomatte without the ?DoD? issues.

I think this is the code in the LifeSaver that set the correct view bounds but haven't looked at the exact reason why this works better then a stock output does:

    -- Create a new image from the image input connection data
    local img = Image({IMG_Like = imgInput, IMG_NoData = req:IsPreCalc()})

    -- Copy the image using the crop feature without any offsets applied
    imgInput:Crop(img, { })

[cedricduriau]

Hi Andrew

This closing comments section is possibly me incorrectly reading the situation and assuming that disk/network performance and system resource usage is something you care about as a TD/coder? I do know you like compact and beautiful code though... Haha. ;)

I do, as you have been seeing with last changes. Fun discovery, ChannelOpOf seems to be the one creating the glitch. I changed the entire matte creation logic, gained readabilty and also performance. Kristof tested this and it has been deemed glitch free since. So glitch fixed in #11.

I wanted to suggest that if you added this code around the Cryptomatte Filename metadata tag reading code in your Cryptomatte fuse you would make it possible for more complex scripted tools to work down the road that would handle relative PathMaps with ease:

filename = self.Comp:MapPath(filename)

Cryptomatte for Fusion is very rigid on how it treats the Filename metadata tag so an entry with a PathMap in it (coming from an EXRIO based tool other then a classical Loader node) causes a Fusion Console error like this to happen:

ERROR: Cannot read image file "Comp:/Media/crypto_vray_balls_test.exr". No such file or directory.

Great addition, fixed in #18.

Cheers Cedric

[AndrewHazelden]

How would you feel about adding a "shim" utility node to the Fusion Cryptomatte toolset that would be a MediaIn node centric metadata Filename injector?

It could be called "CryptoMetadataInjector.fuse" as a working title.

The shim node could literally be just a copy of the SetMetadata.fuse (with a ButtonControl if scope was an issue) that would read the upstream MediaIn node's MediaID tag and the MediaProp filename record. Then the shim fuse could push this Filename, file path, and relative timeline frame number into an inline embedded Metadata Filename record.

If one wanted to get fancy, a "Frame Offset" ScrewControl could help people who turned their MediaIn node Edit page based EXR image sequences into compound footage clips.

FWIW, your "generation 1" ~2017 era Loader node based Cryptomatte fuse had channel reading code that was able to jiggle the channels on a set of Loader nodes when a connected was made to your fuse... so it's not like you don't have the code know-how to nodally scrape that MediaProp info from a Resolve (free) Fusion page node graph stream. :)

LifeSaver.fuse in the early days of the Resolve 15/16 betas was able to act as a Metadata Filename tag injector, so it seems like a nice quality of life thing for Resolve users if Cryptomatte's Fusion offerings had a clean, minimal tool that could do the same task.

Cheers, Andrew

[cedricduriau]

I'll say it just to get it on record again, it's such a shame that we need to cover this considering this works fine for Fusion, yet not for Resolve.

How would you feel about adding a "shim" utility node to the Fusion Cryptomatte toolset that would be a MediaIn node centric metadata Filename injector?

Adding a fuse dedicated for this opens a whole other world of features and bugs. Not so keen on doing this. Now let me clarify, I say this with Cryptomatte as a repository in mind. We can perfectly create a separate fuse and ship it with reactor. I have no problems with that approach.

If one wanted to get fancy, a "Frame Offset" ScrewControl could help people who turned their MediaIn node Edit page based EXR image sequences into compound footage clips.

This reinforces my above statement and fears, it all sounds practical and neat though, don't get me wrong.

What about adding a disabled text edit control to Cryptomatte that drives the filename? By default it could be either empty or the expression to read from the input loader metadata. This way it would be 100% transparent and backwards compatible for Fusion users. Resolve MediaIn users could then enable this and change the expression to be routed to the MediaIn record.

What do you think?

[AndrewHazelden]

Yup. I know what you mean about it being a shame that this is even required.

A TextEdit control would work fine in my opinion... but I didn't want to propose that to you as I still foolishly hope the situation, and this metadata injector concept can be kept as a short term stop-gap workaround.

With short-term being defined as the time until the "powers that be" who interface the Fusion page with the Media Pool, and Edit page in Resolve come to their senses and provide consistent metadata access for Loaders and MediaIn nodes.

[AndrewHazelden]

I'd say yes 💯to doing a Reactor workaround fuse additional package for the metadata injection if you want to keep the official Psyop Crypto repo tidy.