Pools and Groups user overrides during publish

[m-u-r-p-h-y]

Is your feature request related to a problem? Please describe. As mentioned in ynput/OpenPype#3051 PR we can continue improving pools and groups handling during publish

Describe the solution you'd like

We should expose pools and groups in render instances for users so these can be overridden from the default values we set in project settings. (in all hosts where it is actually possible, definitely in Maya, Nuke, Houdini, Blender, etc.)

Render farm resources management can be quite complex. So far we have per-project settings to allow fine-tunning the requirements based on production needs.

Sometimes it is necessary to alter these settings on a scene level to allow user control. For example, we need to restrict the machines with 256GB ram to render or only machines with GPU installed etc.

It will be also valuable to expose these values (the group is hidden in settings now) so it will be much easier to double-check if submission settings are correct.

We should expose pools and groups fields as text fields so it would be possible to add/change the assignment.

Hosts

• [ ] Blender
• [ ] Fusion
• [x] Houdini
• [ ] Max
• [x] Maya
• [x] Nuke

[cuID:OP-3096]

[kalisp]

Question here is if we want to add it to hosts here, or add it when hosts will me move to New Publisher, where it would look same for all hosts (even for AE, Harmony where it is currently not possible for artist to override the values as there are no editable fields for such keys.)

[BigRoy]

add it when hosts will me move to New Publisher, where it would look same for all hosts (even for AE, Harmony where it is currently not possible for artist to override the values as there are no editable fields for such keys.)

Now is the time!

I'd still argue that the default Settings should be able to define a default pool per Host (and maybe even more granular per task?)

Especially because Nuke + 3DsMax + Maya is getting PRs recently involving how Deadline Submissions work. If we get the settings in place correctly now it means we don't need to maintain massive backwards compatibility in the future.

I'd say we want to implement the logic for this in a single way, e.g. as a Attribute Definition on the Collect Pools plug-in so all publishes would directly benefit from it without requiring host specific plug-ins/attribute definitions.

[kalisp]

There is already openpype/modules/deadline/plugins/publish/collect_pools.py that could be enhanced with NP approach (eg. implement there get_attribute_defs). Not sure how this would work with legacy Publish vs DCC.

This collector provides single point to provide pool, it looks for value on instance (in Maya it would come from text field on instance in Maya UI) and if not there it uses configurable value from Settings.

(Is the idea that eventually some/most fields that are currently on instances directly in DCC, for example these pools are directly in Maya UI, will be removed from UI and used as attributes in NP when DCC is transformed from legacy to NP?)

[BigRoy]

For anyone else reading the previous comment, I suspect NP is New Publisher

There is already openpype/modules/deadline/plugins/publish/collect_pools.py that could be enhanced with NP approach (eg. implement there get_attribute_defs). Not sure how this would work with legacy Publish vs DCC.

This is actually done in https://github.com/ynput/OpenPype/pull/4658 currently 👍 That'll need to be tested against the other hosts, like Nuke how it conflicts with logic that might already exist there.

(Is the idea that eventually some/most fields that are currently on instances directly in DCC, for example these pools are directly in Maya UI, will be removed from UI and used as attributes in NP when DCC is transformed from legacy to NP?)

Sort of, yes. The idea at least is that editing the settings would be done through the New Publisher UI with that being the main entry point. Whether it's doable to still have those values accessible/editable in the host I suppose depends on the host but that wouldn't be too trivial to maintain across wide variety of hosts. So yes, I'd say the goal is to unify it so that the logic can be easily maintained and has minimal spread across the codebase (it's contained) while being easily accessible in the New Publisher as a setting.

[BigRoy]

@m-u-r-p-h-y most of these now work fine in AYON host integrations - right? If they are failing in some for you still, could you transfer this issue or report it on ayon-core, or hopefully soon ayon-deadline?

[m-u-r-p-h-y]

Maya and Nuke has it already.

Blender render instance is still missing the options, not sure about the Houdini @MustafaJafar ?

[MustafaJafar]

Maya and Nuke has it already.

Blender render instance is still missing the options, not sure about the Houdini @MustafaJafar ?

I have these options when publishing render from Houdini

[BigRoy]

Transfering this issue since it's still todo apparently for Blender.

Added a checklist to the PR description.