search

ynput / OpenPype

OpenPype has been surpassed by AYON and is now read only.
https://ayon.ynput.io
MIT License
286 stars 129 forks source link

envvar in deadline dependencies hangs the job #366

Open mkolar opened 4 years ago

mkolar commented 4 years ago

We are sending all dependant files to deadline jobs to make sure they are all synced before the job starts, however sometimes client might use environment variable in the texture path or something similar. When that happens, deadline is unable to resolve this path and the job will stay pending until manually released

We should probably resolve these when storing them into the deadline payload to prevent this behaviousr

[cuID:OP-1031]

BigRoy commented 2 years ago

I believe you're referring to these fileDependencies being passed along to the Deadline job on submission.

It could potentially work to just to os.path.expandvars before passing them along, or even os.path.normpath or os.path.abspath. I'm not sure when/how Deadline tries to resolve those paths that get passed along. If it applies correct dirmapping to that then potentially it would work just making sure we pass along absolute paths using os.path.abspath?

Thoughts?

If you have a setup or case that reproduces this issue then try one of those changes and see if that does what you need it to. Especially good to test it in a cross-OS environment?

BigRoy commented 2 years ago

Also, should we have a validator that validates the file dependencies actually exist - or at least logs a warning if it doesn't?

We've had cases where Jobs wouldn't start to render in Deadline due to someone just having added a temporary file into their scene 'as reference' without actually needing it to render. It's a bit odd - but it's more confusing to the artist that the Job won't render in Deadline at all (stays on "Pending") and it's hard to figure out exactly why (which admittedly is also somewhat down to Deadline UX.) I guess we could also disable the project_settings/deadline/publish/MayaSubmitDeadline/asset_dependencies setting altogether on our end. Thoughts?

I'm assuming we're still considering this "high severity"? I'd be happy to look into it if we know the options we'd want.

tokejepsen commented 2 years ago

+1 for validating the dependencies exists. Had this issue a couple of times now as well.