ucl-pond / pySuStaIn

Subtype and Stage Inference (SuStaIn) algorithm with an example using simulated data.
MIT License
112 stars 62 forks source link

Parallelization fails -- TypeError: cannot pickle '_abc._abc_data' object #41

Open katrinaCode opened 1 year ago

katrinaCode commented 1 year ago

Hi pySustain team,

I am running into the following error when setting parallelization = True in ZscoreSustain: "TypeError: cannot pickle '_abc._abc_data' object". This error is picked up ~60 times, always tracing to the pickle.py or the _dill.py files within the pysustain package. This has happened using both Jupyter Notebook and Spyder, however I can run Sustain fine when parallelization is set to False.

Error Traceback: ```js File "[pysustain package location]/lib/python3.10/site-packages/spyder_kernels/py3compat.py", line 356, in compat_exec exec(code, globals, locals) File [notebook], line 231, in prob_subtype_stage = sustain_input.run_sustain_algorithm() File "[pysustain package location]/lib/python3.10/site-packages/pySuStaIn/AbstractSustain.py", line 164, in run_sustain_algorithm ml_likelihood_mat_EM = self._estimate_ml_sustain_model_nplus1_clusters(self.__sustainData, ml_sequence_prev_EM, ml_f_prev_EM) #self.__estimate_ml_sustain_model_nplus1_clusters(self.__data, ml_sequence_prev_EM, ml_f_prev_EM) File "[pysustain package location]/lib/python3.10/site-packages/pySuStaIn/AbstractSustain.py", line 615, in _estimate_ml_sustain_model_nplus1_clusters ml_likelihood_mat = self._find_ml(sustainData) File "[pysustain package location]/lib/python3.10/site-packages/pySuStaIn/AbstractSustain.py", line 704, in _find_ml pool_output_list = self.pool.map(partial_iter, seed_sequences.spawn(self.N_startpoints)) File "[pysustain package location]/lib/python3.10/site-packages/pathos/multiprocessing.py", line 135, in map return _pool.map(star(f), zip(*args)) # chunksize File "[pysustain package location]/lib/python3.10/site-packages/multiprocess/pool.py", line 367, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "[pysustain package location]/lib/python3.10/site-packages/multiprocess/pool.py", line 774, in get raise self._value File "[pysustain package location]/lib/python3.10/site-packages/multiprocess/pool.py", line 540, in _handle_tasks put(task) File "[pysustain package location]/lib/python3.10/site-packages/multiprocess/connection.py", line 214, in send self._send_bytes(_ForkingPickler.dumps(obj)) File "[pysustain package location]/lib/python3.10/site-packages/multiprocess/reduction.py", line 54, in dumps cls(buf, protocol, *args, **kwds).dump(obj) File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 394, in dump StockPickler.dump(self, obj) File "[pysustain package location]/lib/python3.10/pickle.py", line 487, in dump self.save(obj) File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 388, in save StockPickler.save(self, obj, save_persistent_id) File "[pysustain package location]/lib/python3.10/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "[pysustain package location]/lib/python3.10/pickle.py", line 902, in save_tuple save(element) File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 388, in save StockPickler.save(self, obj, save_persistent_id) File "[pysustain package location]/lib/python3.10/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "[pysustain package location]/lib/python3.10/pickle.py", line 887, in save_tuple save(element) File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 388, in save StockPickler.save(self, obj, save_persistent_id) File "[pysustain package location]/lib/python3.10/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "[pysustain package location]/lib/python3.10/pickle.py", line 887, in save_tuple save(element) File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 388, in save StockPickler.save(self, obj, save_persistent_id) File "[pysustain package location]/lib/python3.10/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 1824, in save_function _save_with_postproc(pickler, (_create_function, ( File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 1089, in _save_with_postproc pickler.save_reduce(*reduction) File "[pysustain package location]/lib/python3.10/pickle.py", line 692, in save_reduce save(args) File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 388, in save StockPickler.save(self, obj, save_persistent_id) File "[pysustain package location]/lib/python3.10/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "[pysustain package location]/lib/python3.10/pickle.py", line 887, in save_tuple save(element) File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 388, in save StockPickler.save(self, obj, save_persistent_id) File "[pysustain package location]/lib/python3.10/pickle.py", line 603, in save self.save_reduce(obj=obj, *rv) File "[pysustain package location]/lib/python3.10/pickle.py", line 692, in save_reduce save(args) File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 388, in save StockPickler.save(self, obj, save_persistent_id) File "[pysustain package location]/lib/python3.10/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "[pysustain package location]/lib/python3.10/pickle.py", line 887, in save_tuple save(element) File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 388, in save StockPickler.save(self, obj, save_persistent_id) File "[pysustain package location]/lib/python3.10/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 1427, in save_instancemethod0 pickler.save_reduce(MethodType, (obj.__func__, obj.__self__), obj=obj) File "[pysustain package location]/lib/python3.10/pickle.py", line 692, in save_reduce save(args) File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 388, in save StockPickler.save(self, obj, save_persistent_id) File "[pysustain package location]/lib/python3.10/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "[pysustain package location]/lib/python3.10/pickle.py", line 887, in save_tuple save(element) File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 388, in save StockPickler.save(self, obj, save_persistent_id) File "[pysustain package location]/lib/python3.10/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 1824, in save_function _save_with_postproc(pickler, (_create_function, ( File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 1084, in _save_with_postproc pickler._batch_setitems(iter(source.items())) File "[pysustain package location]/lib/python3.10/pickle.py", line 998, in _batch_setitems save(v) File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 388, in save StockPickler.save(self, obj, save_persistent_id) File "[pysustain package location]/lib/python3.10/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 1698, in save_type _save_with_postproc(pickler, (_create_type, ( File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 1070, in _save_with_postproc pickler.save_reduce(*reduction, obj=obj) File "[pysustain package location]/lib/python3.10/pickle.py", line 692, in save_reduce save(args) File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 388, in save StockPickler.save(self, obj, save_persistent_id) File "[pysustain package location]/lib/python3.10/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "[pysustain package location]/lib/python3.10/pickle.py", line 902, in save_tuple save(element) File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 388, in save StockPickler.save(self, obj, save_persistent_id) File "[pysustain package location]/lib/python3.10/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 1186, in save_module_dict StockPickler.save_dict(pickler, obj) File "[pysustain package location]/lib/python3.10/pickle.py", line 972, in save_dict self._batch_setitems(obj.items()) File "[pysustain package location]/lib/python3.10/pickle.py", line 998, in _batch_setitems save(v) File "[pysustain package location]/lib/python3.10/site-packages/dill/_dill.py", line 388, in save StockPickler.save(self, obj, save_persistent_id) File "[pysustain package location]/lib/python3.10/pickle.py", line 578, in save rv = reduce(self.proto) TypeError: cannot pickle '_abc._abc_data' object ```
System and package info: - MacBook Pro 2021 - Python 3.10.6 Packages: alabaster @ file:///home/ktietz/src/ci/alabaster_1611921544520/work anyio==3.6.2 applaunchservices @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_96v71vcny2/croots/recipe/applaunchservices_1661854626389/work appnope==0.1.3 argon2-cffi==21.3.0 argon2-cffi-bindings==21.2.0 arrow @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_bc9ine8jfo/croot/arrow_1666726871970/work astroid @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_9fsa1cbbec/croots/recipe/astroid_1659023133872/work asttokens==2.0.8 atomicwrites==1.4.0 attrs==22.1.0 autopep8 @ file:///opt/conda/conda-bld/autopep8_1650463822033/work awkde @ git+https://github.com/noxtoby/awkde.git@1c31e55fe54c0cad80ab423a9605fc9ddfb2614c Babel==2.10.3 backcall @ file:///home/ktietz/src/ci/backcall_1611930011877/work beautifulsoup4 @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_croot-cdiouih5/beautifulsoup4_1650462164803/work binaryornot @ file:///tmp/build/80754af9/binaryornot_1617751525010/work black @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_d0nhgmyc3l/croots/recipe/black_1660237813406/work bleach==5.0.1 brotlipy==0.7.0 certifi @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_0ek9yztvu3/croot/certifi_1665076692562/work/certifi cffi @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_62rp5d8fd4/croots/recipe/cffi_1659598655556/work chardet @ file:///Users/builder/ci_310/chardet_1642531418028/work charset-normalizer==2.1.1 click @ file:///opt/concourse/worker/volumes/live/2d66025a-4d79-47c4-43be-6220928b6c82/volume/click_1646056610594/work cloudpickle @ file:///tmp/build/80754af9/cloudpickle_1632508026186/work colorama @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_b8ecd5af-5e60-48b8-80ac-92164ecb9b9bxkf0tkfp/croots/recipe/colorama_1657009097162/work contourpy==1.0.5 cookiecutter @ file:///opt/conda/conda-bld/cookiecutter_1649151442564/work cryptography @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_3evwafgyg8/croot/cryptography_1665612651044/work cycler==0.11.0 debugpy==1.6.3 decorator @ file:///opt/conda/conda-bld/decorator_1643638310831/work defusedxml @ file:///tmp/build/80754af9/defusedxml_1615228127516/work diff-match-patch @ file:///Users/ktietz/demo/mc3/conda-bld/diff-match-patch_1630511840874/work dill @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_88dxe9g1aq/croot/dill_1667919544494/work docutils @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_10cfb287-0327-45ef-a38e-53dffd30cef1nwpvy20e/croots/recipe/docutils_1657175439973/work entrypoints @ file:///opt/concourse/worker/volumes/live/5eb4850e-dcbc-41ad-5f22-922bac778f70/volume/entrypoints_1649926457041/work executing==1.1.1 fastjsonschema @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_b5c1gee32t/croots/recipe/python-fastjsonschema_1661368622875/work flake8 @ file:///opt/conda/conda-bld/flake8_1648129545443/work fonttools==4.38.0 future==0.18.2 idna @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_00jf0h4zbt/croot/idna_1666125573348/work imagesize @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_4a6ed1be-fe30-4d6a-91d4-f867600caa0be5_dxzvt/croots/recipe/imagesize_1657179500955/work importlib-metadata @ file:///opt/concourse/worker/volumes/live/a8740f82-0523-4b08-5bb5-afa0c929f5e0/volume/importlib-metadata_1648562424930/work inflection==0.5.1 intervaltree @ file:///Users/ktietz/demo/mc3/conda-bld/intervaltree_1630511889664/work ipykernel==6.16.2 ipython==8.5.0 ipython-genutils @ file:///tmp/build/80754af9/ipython_genutils_1606773439826/work isort @ file:///tmp/build/80754af9/isort_1628603791788/work jedi @ file:///opt/concourse/worker/volumes/live/18b71546-5bde-4add-72d1-7d16b76f0f7a/volume/jedi_1644315243726/work jellyfish @ file:///opt/concourse/worker/volumes/live/d045b25f-e3af-4008-4edc-a00aeffb8b33/volume/jellyfish_1647962558521/work Jinja2 @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_6adj7x0ejx/croot/jinja2_1666908137966/work jinja2-time @ file:///opt/conda/conda-bld/jinja2-time_1649251842261/work joblib==1.2.0 json5==0.9.10 jsonschema @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_d832da7jx3/croots/recipe/jsonschema_1663375475386/work jupyter-server==1.21.0 jupyter_client==7.4.4 jupyter_core @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_fc_0us_ta7/croot/jupyter_core_1668084443574/work jupyterlab==3.5.0 jupyterlab-pygments==0.2.2 jupyterlab_server==2.16.1 jupyterthemes==0.20.0 kde-ebm @ git+https://github.com/ucl-pond/kde_ebm.git@26ee48f7f723a82e4ff740e59b9745aa7def3daa keyring @ file:///Users/builder/ci_310/keyring_1642616528347/work kiwisolver==1.4.4 lazy-object-proxy @ file:///Users/builder/ci_310/lazy-object-proxy_1642533824465/work lesscpy==0.15.1 lxml @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_1902c961-4bd2-4871-a3c5-70b7317a6521kpj7nz2o/croots/recipe/lxml_1657545138937/work MarkupSafe @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_d4a9444f-bd4c-4043-b47d-cede33979b0fve7bm42r/croots/recipe/markupsafe_1654597878200/work matplotlib==3.6.0 matplotlib-inline @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_9ddl71oqte/croots/recipe/matplotlib-inline_1662014471815/work mccabe @ file:///opt/conda/conda-bld/mccabe_1644221741721/work mistune==2.0.4 multiprocess==0.70.14 mypy-extensions==0.4.3 nbclassic==0.4.5 nbclient==0.7.0 nbconvert==7.2.5 nbformat==5.7.0 nest-asyncio==1.5.6 notebook==6.5.1 notebook_shim==0.2.0 numpy==1.23.4 numpydoc @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_adnyzxppoz/croot/numpydoc_1668085907252/work p2j==1.3.2 packaging @ file:///tmp/build/80754af9/packaging_1637314298585/work pandas==1.5.1 pandocfilters @ file:///opt/conda/conda-bld/pandocfilters_1643405455980/work parso @ file:///opt/conda/conda-bld/parso_1641458642106/work pathos==0.3.0 pathspec @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_e2t1r2kdq7/croots/recipe/pathspec_1659627124303/work pexpect @ file:///tmp/build/80754af9/pexpect_1605563209008/work pickleshare @ file:///tmp/build/80754af9/pickleshare_1606932040724/work Pillow==9.2.0 platformdirs @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_7fs8_2xgrm/croots/recipe/platformdirs_1662711383474/work pluggy @ file:///opt/concourse/worker/volumes/live/8277900c-164a-49c8-6f2a-f55c3c0154be/volume/pluggy_1648042581708/work ply==3.11 pox==0.3.2 poyo @ file:///tmp/build/80754af9/poyo_1617751526755/work ppft==1.7.6.6 prometheus-client==0.15.0 prompt-toolkit==3.0.31 psutil==5.9.3 ptyprocess @ file:///tmp/build/80754af9/ptyprocess_1609355006118/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl pure-eval==0.2.2 pybind11==2.10.0 pycodestyle @ file:///tmp/build/80754af9/pycodestyle_1636635402688/work pycparser @ file:///tmp/build/80754af9/pycparser_1636541352034/work pydocstyle @ file:///tmp/build/80754af9/pydocstyle_1621600989141/work pyflakes @ file:///tmp/build/80754af9/pyflakes_1636644436481/work Pygments==2.13.0 pylint @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_e75_4ydew9/croots/recipe/pylint_1659110352634/work pyls-spyder==0.4.0 pyobjc-core @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_e7giy3a869/croots/recipe/pyobjc-core_1661848172499/work pyobjc-framework-Cocoa @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_4c2umern3y/croots/recipe/pyobjc-framework-cocoa_1661850714385/work pyobjc-framework-CoreServices @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_4717m_ngol/croots/recipe/pyobjc-framework-coreservices_1661853392396/work pyobjc-framework-FSEvents @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_5atkr691rb/croots/recipe/pyobjc-framework-fsevents_1661852390555/work pyOpenSSL @ file:///opt/conda/conda-bld/pyopenssl_1643788558760/work pyparsing @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_3a17y2delq/croots/recipe/pyparsing_1661452538853/work PyQt5-sip==12.11.0 pyrsistent==0.18.1 PySocks @ file:///Users/builder/ci_310/pysocks_1642536366386/work pySuStaIn @ git+https://github.com/ucl-pond/pySuStaIn@564f07617a2a11477a18aec0b24d5d80825b0371 python-dateutil @ file:///tmp/build/80754af9/python-dateutil_1626374649649/work python-lsp-black @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_14xl6hg757/croots/recipe/python-lsp-black_1661852036282/work python-lsp-jsonrpc==1.0.0 python-lsp-server @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_6cu9im5n5w/croots/recipe/python-lsp-server_1661813818984/work python-slugify @ file:///tmp/build/80754af9/python-slugify_1620405669636/work pytz==2022.5 PyYAML==6.0 pyzmq==24.0.1 QDarkStyle @ file:///tmp/build/80754af9/qdarkstyle_1617386714626/work qstylizer @ file:///tmp/build/80754af9/qstylizer_1617713584600/work/dist/qstylizer-0.1.10-py2.py3-none-any.whl QtAwesome @ file:///tmp/build/80754af9/qtawesome_1637160816833/work qtconsole @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_552cqm7spz/croots/recipe/qtconsole_1662018258355/work QtPy @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_4e5ppuhz0f/croots/recipe/qtpy_1662014536017/work requests @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_516b78ce-034d-4395-b9b5-1d78c2847384qtnol99l/croots/recipe/requests_1657734628886/work rope @ file:///opt/conda/conda-bld/rope_1643788605236/work Rtree @ file:///Users/builder/ci_310/rtree_1642537064369/work scikit-learn==1.1.3 scipy==1.9.3 seaborn==0.12.1 Send2Trash==1.8.0 sip @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_88z1zrsfrf/croots/recipe/sip_1659012373083/work six @ file:///tmp/build/80754af9/six_1644875935023/work sklearn==0.0 sniffio==1.3.0 snowballstemmer @ file:///tmp/build/80754af9/snowballstemmer_1637937080595/work sortedcontainers @ file:///tmp/build/80754af9/sortedcontainers_1623949099177/work soupsieve @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_14fb2zs6e3/croot/soupsieve_1666296397588/work Sphinx @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_5d9f8d69-b80c-4ca1-8876-1698c70b1faeqe461tx8/croots/recipe/sphinx_1657784127805/work sphinxcontrib-applehelp @ file:///home/ktietz/src/ci/sphinxcontrib-applehelp_1611920841464/work sphinxcontrib-devhelp @ file:///home/ktietz/src/ci/sphinxcontrib-devhelp_1611920923094/work sphinxcontrib-htmlhelp @ file:///tmp/build/80754af9/sphinxcontrib-htmlhelp_1623945626792/work sphinxcontrib-jsmath @ file:///home/ktietz/src/ci/sphinxcontrib-jsmath_1611920942228/work sphinxcontrib-qthelp @ file:///home/ktietz/src/ci/sphinxcontrib-qthelp_1611921055322/work sphinxcontrib-serializinghtml @ file:///tmp/build/80754af9/sphinxcontrib-serializinghtml_1624451540180/work spyder @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_97gv8v17po/croots/recipe/spyder_1663056808858/work spyder-kernels @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_1c7pyd81si/croots/recipe/spyder-kernels_1662457889999/work stack-data==0.5.1 tdt==0.5.4 terminado==0.17.0 text-unidecode @ file:///Users/ktietz/demo/mc3/conda-bld/text-unidecode_1629401354553/work textdistance @ file:///tmp/build/80754af9/textdistance_1612461398012/work threadpoolctl==3.1.0 three-merge @ file:///tmp/build/80754af9/three-merge_1607553261110/work tinycss @ file:///tmp/build/80754af9/tinycss_1617713798712/work tinycss2 @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_56dshjmms6/croot/tinycss2_1668168824483/work toml @ file:///tmp/build/80754af9/toml_1616166611790/work tomli @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_90762ba4-f339-47e8-bd29-416854a59b233d27hku_/croots/recipe/tomli_1657175507767/work tomlkit @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_5fgtm9if1m/croots/recipe/tomlkit_1658946891645/work tornado @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_1fimz6o0gc/croots/recipe/tornado_1662061695695/work tqdm==4.64.1 traitlets==5.5.0 typing_extensions @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_ff5_5nqr6l/croots/recipe/typing_extensions_1659638832447/work ujson @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_cf44fbd5-5db0-48cf-86c4-c8d4e74d1cbbwhgckc99/croots/recipe/ujson_1657544919410/work Unidecode @ file:///tmp/build/80754af9/unidecode_1614712377438/work urllib3 @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_7f7kb5tudl/croot/urllib3_1666298941688/work watchdog @ file:///Users/builder/ci_310/watchdog_1642516765439/work wcwidth @ file:///Users/ktietz/demo/mc3/conda-bld/wcwidth_1629357192024/work webencodings==0.5.1 websocket-client==1.4.1 whatthepatch @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_0aabmq0ph3/croots/recipe/whatthepatch_1661795995892/work wrapt @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_1ade1f68-8354-4db8-830b-ff3072015779vd_2hm7k/croots/recipe/wrapt_1657814407132/work wurlitzer @ file:///Users/builder/ci_310/wurlitzer_1642539193810/work yapf @ file:///tmp/build/80754af9/yapf_1615749224965/work zipp @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_b279673d-f037-44c7-8773-c5a6b6f51037d3wfr9cq/croots/recipe/zipp_1652341773612/work

Thank you for your help! Katrina

sea-shunned commented 1 year ago

Hi Katrina,

Many thanks for reporting this to us. I couldn't see the exact dill version in the output that you provided, but locally I've seen that this error occurs with dill==0.3.6 and not with a previous version. Does downgrading via e.g. pip install dill==0.3.5.1 fix this issue for you?

jadenecke commented 1 year ago

dill==0.3.5.1 this did the trick for me, however I was warned by pip and then also adjusted multiprocess==0.70.13 and pathos==0.2.9. It did not throw an error right away with the newer versions, but I also did not let the script finish.

sea-shunned commented 1 year ago

I also got the same, and it hasn't caused any problems (yet).

We are working on a longer term solution, but I am glad this helped in the interm. When the full solution has been integrated, I'll let you know here and close this issue!

Thanks for reporting the problem :)

katrinaCode commented 1 year ago

Hi all, thank you! Doing the three-package version downgrade as suggested by jadenecke worked and it isn't throwing the error anymore. 🥳