search

ynput / ayon-deadline

Deadline addon for AYON
Apache License 2.0
3 stars 11 forks source link

Baking chunk size for sequences #51

Closed jrsndl closed 2 weeks ago

jrsndl commented 1 month ago

Changelog Description

Supporting deadline rendering of Nuke baking jobs (Extract Review Intermediate) with multi chunks. The Chunk size is copied from the ayon+settings://deadline/publish/NukeSubmitDeadline/chunk_size

Additional info

Might close https://github.com/ynput/ayon-nuke/issues/3

Testing notes:

  1. Set Extract Review Intermediate to jpg
  2. Make sure you do not have Nuke Slate present (see bug)
  3. Publish Nuke render on Deadline
  4. Check Number of tasks for baking job in Deadline Monitor, it should be > 1
kalisp commented 2 weeks ago

If this PR is about to have chunk size dynamic for Baking job, it will be obsolete by https://github.com/ynput/ayon-deadline/pull/49 where it will be controlled by Profile/could be overridden by artist.

Would it be important to separate chunks/other JobInfo fields per render vs baking job (eg. have different chunk for render, different one for baking)? Currently it is not possible, not sure how the model in Setting would like for that.

BigRoy commented 2 weeks ago

If this PR is about to have chunk size dynamic for Baking job, it will be obsolete by https://github.com/ynput/ayon-deadline/pull/49 where it will be controlled by Profile/could be overridden by artist.

Actually - I think what the intent is.. is that if you were to write out a video file you want the chunk size to be infinitely large because multiple machines couldn't render out to the same file. Something tells me - that is what this is trying to solve.

If media has # in it - allow chunk size to be configurable. If media does not have # it's a single file, and we assume multiple machines can't simultaneously write to.

To easily support that in an abstract plug-in we may need some way to identify that a job MUST be one task or not due to that. Like "singleTaskOnly" or "forceSingleTask" or whatever.

jrsndl commented 2 weeks ago

Actually - I think what the intent is.. is that if you were to write out a video file you want the chunk size to be infinitely large because multiple machines couldn't render out to the same file. Something tells me - that is what this is trying to solve.

Correct. Of course, having confugurable chunk size for baking sequences would be great. Possibly even Deadline Frame Dependencies, so baking can start right when each frame finishes. Just taking the same chunks as the render is easy and should already help quite a bit.

I am trying to solve very slow publishing of long highres shots. Sometimes, particullarly for episodic vfx, we have shots that are one, or two minutes long. Production gets gray hair when publish is on the farm one more hour just to produce baking mov.

BigRoy commented 2 weeks ago

I am trying to solve very slow publishing of long highres shots. Sometimes, particullarly for episodic vfx, we have shots that are one, or two minutes long. Production gets gray hair when publish is on the farm one more hour just to produce baking mov.

Valid use case - and has come up before. I think there was an epic description about that some time ago @antirotor? But definitely quite some dev work.

kalisp commented 2 weeks ago

Closing in favor of https://github.com/ynput/ayon-deadline/pull/49