cross platform rendering - referenced gaffer scenes

[mrpurest]

Hello,

This is more of a question / discussion than anything else.

I have a pretty reliable setup already submitting renders from Linux, and rendering on both windows and Linux render nodes.

The only unsolved part currently is gaffer references, since the referenced gaffer files of course also contain paths, and the gafferDeadline path remapping only runs on the submitted file, not the referenced files.

I can think of multiple options for resolving this - but maybe its already been solved?

[ericmehl]

Yes, that's definitely an issue for references, and one I haven't addressed yet because I expect users to have lots of different possible configurations. But I'm glad you opened up the subject as I'm interested to hear what may and may not work for different studios.

Personally I use ${project:rootDirectory} at the start of all my paths and set project:rootDirectory to an environment variable that is customized per-platform. It can be annoying to always make sure that's in the path, and something I plan to implement soon is some combination of :

• A user-runnable script command to search the Gaffer script for possible path mistakes. Possibly done as a hook on script save (not sure if such a hook is available).
• A preSpool hook to automatically convert bad path roots when dispatching. This would only work for the main Gaffer script since the references can't be modified at dispatch time (at least not without overwriting the reference file).

I also try to make my references take as an input parameter a path name that is set from the main script, rather than having it hard-coded on a graph node itself. That way path-mapping the main script works for references as well. But this may not be a good option depending on your setup.

Another option would be for GafferDeadline to apply path mapping to the references as well. I'm a little wary of this for a few reasons :

• I can see this being possibly dangerous if you don't have a system in place to make sure all of the references are unique files per-platform or even per-dispatch. I could see unexpected errors coming up from systems smashing over the same file with their own path mapping, and at the very least the reference file would be left in a random state depending on the last machine to path map it.
• Alternatively GafferDeadline could copy the reference files to the Worker temporary directory and path map those files. Then update the Reference nodes with the new file. Maybe I'm being too cautious, but I'm vaguely anxious about updating reference paths behind the scenes.
• I suppose there could be an off chance that someone has a path name that shouldn't be mapped or a string that happens to match path mapping rules but is not actually a path. This seems somewhat unlikely to me, but one that could be hard for a user to solve since path mapping happens on temporary files. One example might be if 3d scene paths are structured in the same naming system as the file system? Then all the Filter nodes are going to go haywire.

Also notable is a possibility of having some kind of path mapping built in to Gaffer when I get the FileSystemPathPlug merged as part of the wider Windows effort. I don't know what the likelihood of having such a feature accepted into the main project, but it was briefly discussed a while back when we first were talking about Windows vs. Linux path handling.

I'm certainly interested to hear your thoughts and those of anyone else who may be following.

[mrpurest]

For some reason having a ${root} variable on all the paths was not one of the solutions I had thought of, despite it being basically the exact way that I am getting the .usd files to work cross platform.

I don't see any downsides to this approach really. Other than it needing to be setup this way for all the gaffer files.

if GafferDeadline was going to remap references, then I would definitely want them to be copied first. I wouldn't want any files changed in place. remapping like this would be quite nice I think, since it would not require the gaffer scenes to be set up in any specific way.

A false match seems quite low to me - and fairly easy to avoid.