ynput / OpenPype

OpenPype has been surpassed by AYON and is now read only.
https://ayon.ynput.io
MIT License
286 stars 129 forks source link

Deadline render fails with openpype_console error #3955

Closed m-u-r-p-h-y closed 2 years ago

m-u-r-p-h-y commented 2 years ago

Running version 3.14.4-nightly.2 (latest develop)

Describe the bug rendering on Deadline fails with an error

2022-10-10 11:14:21:  0: PYTHON: Looking for OpenPype at: C:\Program Files (x86)\OpenPype\3.14.3
2022-10-10 11:14:21:  0: PYTHON:   - found: 3.14.3 - C:\Program Files (x86)\OpenPype\3.14.3
2022-10-10 11:14:21:  0: PYTHON: *** Latest available version found is 3.14.4
2022-10-10 11:14:21:  0: PYTHON: *** Latest compatible version found is 3.14.4
2022-10-10 11:14:21:  0: PYTHON: --- OpenPype executable: d:\REPO\3.14.4-nightly.2\openpype_console.exe
2022-10-10 11:14:21:  0: PYTHON: >>> Temporary path: C:\Users\murph\AppData\Local\Temp\20221010091421253885_ecfe4a70-487b-11ed-ba51-00090faa0001.json
2022-10-10 11:14:21:  0: PYTHON: >>> Executing: d:\REPO\3.14.4-nightly.2\openpype_console.exe --headless extractenvironments C:\Users\murph\AppData\Local\Temp\20221010091421253885_ecfe4a70-487b-11ed-ba51-00090faa0001.json --project murphy_22101 --asset sh030 --task lighting --app maya/2022 --envgroup farm
2022-10-10 11:14:22:  0: PYTHON: >>> Exception b''
2022-10-10 11:14:22:  0: PYTHON: Traceback (most recent call last):
2022-10-10 11:14:22:  0: PYTHON:   File "C:\ProgramData\Thinkbox\Deadline10\workers\WS01\plugins\633fe432db251310d48c54f5\GlobalJobPreLoad.py", line 198, in inject_openpype_environment
2022-10-10 11:14:22:  0: PYTHON:     env=env)
2022-10-10 11:14:22:  0: PYTHON:   File "C:\Program Files\Thinkbox\Deadline10\bin\python3\lib\subprocess.py", line 411, in check_output
2022-10-10 11:14:22:  0: PYTHON:     **kwargs).stdout
2022-10-10 11:14:22:  0: PYTHON:   File "C:\Program Files\Thinkbox\Deadline10\bin\python3\lib\subprocess.py", line 512, in run
2022-10-10 11:14:22:  0: PYTHON:     output=stdout, stderr=stderr)
2022-10-10 11:14:22:  0: PYTHON: subprocess.CalledProcessError: Command '['d:\\REPO\\3.14.4-nightly.2\\openpype_console.exe', '--headless', 'extractenvironments', 'C:\\Users\\murph\\AppData\\Local\\Temp\\20221010091421253885_ecfe4a70-487b-11ed-ba51-00090faa0001.json', '--project', 'murphy_22101', '--asset', 'sh030', '--task', 'lighting', '--app', 'maya/2022', '--envgroup', 'farm']' returned non-zero exit status 1.
2022-10-10 11:14:22:  0: PYTHON: !!! Injection failed.
2022-10-10 11:14:22:  0: 'C:\Users\murph\AppData\Local\Thinkbox\Deadline10\pythonAPIs\2022-07-22T224802.0000000Z' already exists. Skipping extraction of PythonSync.

running the failing command d:\REPO\3.14.4-nightly.2\openpype_console.exe --headless extractenvironments C:\Users\murph\AppData\Local\Temp\20221010091421253885_ecfe4a70-487b-11ed-ba51-00090faa0001.json --project murphy_22101 --asset sh030 --task lighting --app maya/2022 --envgroup farm

in cmd does not fail, so it is very hard to tell why is it failing when run in deadline (non-zero exit status 1.)

[cuID:]

BigRoy commented 2 years ago

This line:

>>> Exception b''

Hints that the subprocess does not output anything - not even any errors.

What deadline version are you running and is it set to use Python 3? If using Py2 I believe this could potentially be an issue where the env passed to subprocess contains non-string values.

m-u-r-p-h-y commented 2 years ago

it is the latest 10.1.23.6

2022-10-10 11:14:21: 0: INFO: Plugin execution sandbox using Python version 3

m-u-r-p-h-y commented 2 years ago

Does it work for you @BigRoy with the latest develop? (we have confirmed the issue on several machines already)

BigRoy commented 2 years ago

Yes it does, but I'm not entirely sure we've rolled out the very latest Deadline plug-in to the repository.

At least we are on the same Deadline version: Deadline Client Version: 10.1.23.6 Release (773a6289d)

EDIT

Had a quick look, this is the only real difference that we have before the injection:

        # We don't care to much about the checksums being checked for the
        # OpenPype console. The downsides outweigh the benefits since it's
        # so much slower when it needs to validate. And even IF the version
        # is invalid it'd still just error.
        env["OPENPYPE_DONT_VALIDATE_VERSION"] = "0"

Living dangerous here.

m-u-r-p-h-y commented 2 years ago

So according to the team, the issue only happens when you are running OP from the code, not from the build.

m-u-r-p-h-y commented 2 years ago

the issue was identified as two separate issues #3998 and #3999

so closing this one