search

Distributive-Network / Bifrost

An SHM/memmap based python and node communication/evaluation tool.
Other
7 stars 2 forks source link

Cloudpickle error, update support to 2.1.0 #43

Open Jonah1234567 opened 2 years ago

Jonah1234567 commented 2 years ago

While testing Bifrost in a local python environment (Node 16.15.0, Ubunutu 20.04, Bifrost 0.5.11 and Python 3.8.10), I ran into an error in which a job would not return results properly and yielded an error:

7 : Slice Error : Error [PythonError]: Traceback (most recent call last):
File "/lib/python3.9/site-packages/_pyodide/_base.py", line 415, in eval_code
CodeRunner(
File "/lib/python3.9/site-packages/_pyodide/_base.py", line 296, in run
coroutine = eval(self.code, globals, locals)
File "<exec>", line 66, in <module>
AttributeError: Can't get attribute '_make_function' on <module 'cloudpickle.cloudpickle' from '/lib/python3.9/site-packages/cloudpickle/cloudpickle.py'>
at new_error (eval at initializePackage (eval at <anonymous> (https://portal.distributed.computer/dcp-client/libexec/sandbox/bravojs-env.js?1655212463038:43:15)), <anonymous>:16:247183)
at wasm://wasm/02438b46:wasm-function[308]:0xe6eb0
at wasm://wasm/02438b46:wasm-function[309]:0xe6fae
at Module._pythonexc2js (eval at initializePackage (eval at <anonymous> (https://portal.distributed.computer/dcp-client/libexec/sandbox/bravojs-env.js?1655212463038:43:15)), <anonymous>:16:944230)
at Module.callPyObjectKwargs (eval at initializePackage (eval at <anonymous> (https://portal.distributed.computer/dcp-client/libexec/sandbox/bravojs-env.js?1655212463038:43:15)), <anonymous>:3:10739)
at Module.callPyObject (eval at initializePackage (eval at <anonymous> (https://portal.distributed.computer/dcp-client/libexec/sandbox/bravojs-env.js?1655212463038:43:15)), <anonymous>:3:10851)
at PyProxyClass.apply (eval at initializePackage (eval at <anonymous> (https://portal.distributed.computer/dcp-client/libexec/sandbox/bravojs-env.js?1655212463038:43:15)), <anonymous>:3:18318)
at Object.apply (eval at initializePackage (eval at <anonymous> (https://portal.distributed.computer/dcp-client/libexec/sandbox/bravojs-env.js?1655212463038:43:15)), <anonymous>:3:17372)
at Object.runPython (eval at initializePackage (eval at <anonymous> (https://portal.distributed.computer/dcp-client/libexec/sandbox/bravojs-env.js?1655212463038:43:15)), <anonymous>:3:22324)
at workFunction (eval at mainModule (https://portal.distributed.computer/dcp-client/libexec/sandbox/bravojs-env.js?1655212463038:1:1), <anonymous>:292:19) {
__error_address: 9333320
}
7 : Python Log : [
'// PYTHON LOG START //',
'Python initialization complete',
'// PYTHON LOG ERROR //'
]
Deploy Job Error : TypeError: Reduce of empty array with no initial value
at Array.reduce (<anonymous>)
at dcpPost (evalmachine.<anonymous>:252:45)
at runNextTicks (node:internal/process/task_queues:61:5)
at processImmediate (node:internal/timers:437:9)
at async evalmachine.<anonymous>:338:21
at async Evaluator.evaluate (/home/jonah/.local/lib/python3.8/site-packages/bifrost/main.js:210:9)
at async Transform.inputStream._transform (/home/jonah/.local/lib/python3.8/site-packages/bifrost/main.js:239:9)
Traceback (most recent call last):
  File "main.py", line 20, in <module>
    result_set = job.exec()
  File "/home/jonah/.local/lib/python3.8/site-packages/bifrost/Job.py", line 417, in exec
    results = self.__dcp_run()
  File "/home/jonah/.local/lib/python3.8/site-packages/bifrost/Job.py", line 367, in __dcp_run
    result_set = node_output['jobOutput']
KeyError: 'jobOutput'

The cause of this error is Bifrost not supporting cloudpickle 2.1.0 so downgrading the package to 2.0.0 would allow the job to work. In the future we need to upgrade support for cloudpickle to 2.1.0

ccmcintyre-kingsds commented 2 years ago

Keeping open, re-labelled as enhancement, as while the failure condition is resolved by adding the explicit version dependency, it still remains the goal to expand functionality to support 2.1.0.

ccmcintyre-kingsds commented 2 years ago

Cloudpickle was deactivated broadly as part of the Pyodide 21 and dependency automation updates, due to increasingly frequest failures across various platforms. Requires further investigation and support.