Our BOOMX fails on very peculiar issue related to unpickling which isn't very clear. Wondering if anybody has faced this and what the resolution was?
jsonPayload: {
exception: "Traceback (most recent call last):
File "/root/.pyenv/versions/3.10.10/lib/python3.10/site-packages/cog/server/runner.py", line 113, in handle_error
raise error
File "/root/.pyenv/versions/3.10.10/lib/python3.10/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/root/.pyenv/versions/3.10.10/lib/python3.10/site-packages/cog/server/runner.py", line 334, in predict
return _predict(
File "/root/.pyenv/versions/3.10.10/lib/python3.10/site-packages/cog/server/runner.py", line 370, in _predict
for event in worker.predict(input_dict, poll=0.1):
File "/root/.pyenv/versions/3.10.10/lib/python3.10/site-packages/cog/server/worker.py", line 118, in _wait
ev = self._events.recv()
File "/root/.pyenv/versions/3.10.10/lib/python3.10/multiprocessing/connection.py", line 251, in recv
return _ForkingPickler.loads(buf.getbuffer())
_pickle.UnpicklingError: invalid load key, '\x00'."
logger: "server.runner"
message: "caught exception while running prediction"
timestamp: "2023-04-05T08:19:51.798365Z"
}
For context, we're calling a command line utility from subprocess.run() inside the BOOMX. Wondering if it could cause issues. Furthermore, this issue is intermittent... so very sneaky to debug. Thanks!
rookasrudzianskas
commented
1 year ago
Our BOOMX fails on very peculiar issue related to unpickling which isn't very clear. Wondering if anybody has faced this and what the resolution was?
For context, we're calling a command line utility from subprocess.run() inside the BOOMX. Wondering if it could cause issues. Furthermore, this issue is intermittent... so very sneaky to debug. Thanks!