search

ynput / ayon-houdini

Houdini addon for AYON
Apache License 2.0
9 stars 7 forks source link

Create AYON Publish ROP node #119

Open BigRoy opened 4 days ago

BigRoy commented 4 days ago

Please describe the feature you have in mind and explain what the current shortcomings are?

Add a generalized "AYON Publish ROP" for /out context that can submit itself as dependent job to Deadline with the input ROPs as dependencies and their files.

This may seem like it overlaps with #2 - but it's different in that this is about exposing our own ROP node that just "publishes upstream inputs" instead of being imprinted/embedded as logic onto ROP nodes themselves.

So, we want a ROP node in the /out context which takes the inputs and "publishes" those into AYON (as 'existing files'). This way, you can attach the ROP to any upstream dependencies and publish their outputs trivially.

The default deadline submitter (outside of AYON) can already take ROPs in /out and submit them each as individual dependent jobs as needed - so that logic already lives there and is not the primary issue to solve here (we can essentially copy the majority of that logic). It being a ROP in /out should already allows to generate a dependency graph inside of deadline to execute work as a tree.

  1. Create a custom AYON Publish ROP HDA
  2. Allow it to publish the input dependencies.

(Each upstream node could become its own representation; if you need multiple products you can then position multiple AYON publish nodes in the graph for example)

Lyon stated:

we want to publish a scene.

  • first we need to generate and export all the assets.
  • then we need to generate and export the scene.
  • then we need to start export the render.

this could look like this. image

and other use case would be an game asset generation. image

in all those cases we need to build a dependency graph recursively upwards from our Ayon-Deadline node.

It's crucial to understand that we're not here to make this a Deadline submitter node. We want to make it an AYON Publish node - that knows nothing of Deadline; and the goal is that hopefully you can then throw any other submitter to it and have it 'work' out of the box (like using Deadline's own "Deadline Submission ROP").

How would you imagine the implementation of the feature?

Create an AYON publish ROP that can publish its input streams.

  1. Create an AYON Publish ROP for /out context
  2. Allow it to publish just the first input (MVP)
  3. Allow multiple upstream dependencies (nice-to-have).
  4. Allow multiple AYON publish ROPs in a stream to 'break up' what files end up in which product, etc.
BigRoy commented 3 days ago

Just attaching these references/links regarding the soho configuration for output drivers (RopNode) in /out to run a python script on RopNode.render() which is what we'd need:

This seems to be quite thorough in explaining some of it: https://sebastienmarsais.com/houdini-for-td-processing.html Which seems better than the odd Houdini docs about it: https://www.sidefx.com/docs/houdini/render/soho.html And some forum topics regarding Houdini, SOHO and running a Python script: