materialsproject / jobflow

jobflow is a library for writing computational workflows.
https://materialsproject.github.io/jobflow
Other
95 stars 24 forks source link

BUG: Jobflow not compatible with maggma 0.38.0+ #68

Closed Andrew-S-Rosen closed 2 years ago

Andrew-S-Rosen commented 2 years ago

Just reporting what I shared via Slack. Jobflow works fine with maggma==0.36.0, but not via the new maggma version in #61.

import jobflow # in a clean Python 3.8 conda env (Windows) after pip install jobflow
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\asros\Anaconda3\envs\cms2\lib\site-packages\jobflow\__init__.py", line 11, in <module>
    from jobflow.settings import JobflowSettings
  File "C:\Users\asros\Anaconda3\envs\cms2\lib\site-packages\jobflow\settings.py", line 15, in <module>
    class JobflowSettings(BaseSettings):
  File "pydantic\main.py", line 299, in pydantic.main.ModelMetaclass.__new__
  File "pydantic\fields.py", line 411, in pydantic.fields.ModelField.infer

  File "pydantic\fields.py", line 342, in pydantic.fields.ModelField.__init__
  File "pydantic\fields.py", line 445, in pydantic.fields.ModelField.prepare
  File "pydantic\fields.py", line 473, in pydantic.fields.ModelField._set_default_and_type
  File "pydantic\fields.py", line 345, in pydantic.fields.ModelField.get_default
  File "C:\Users\asros\Anaconda3\envs\cms2\lib\site-packages\jobflow\settings.py", line 69, in <lambda>
    default_factory=lambda: JobStore(MemoryStore()),
TypeError: Can't instantiate abstract class JobStore with abstract methods _collection

Edit: Actually, even with maggma==0.36.0, things aren't working when I try to access the store (looks like a maggma issue). The traceback below is in a clean Python 3.8 environment (Linux) after doing pip install jobflow but setting maggma==0.36.0 and running run_locally().

Traceback (most recent call last):
  File "flow_test.py", line 71, in <module>
    responses = run_locally(flow, create_folders=True)
  File "/global/homes/r/rosen/software/miniconda/envs/cms2/lib/python3.8/site-packages/jobflow/managers/local.py", line 62, in run_locally
    store.connect()
  File "/global/homes/r/rosen/software/miniconda/envs/cms2/lib/python3.8/site-packages/jobflow/core/store.py", line 103, in connect
    self.docs_store.connect(force_reset=force_reset)
  File "/global/homes/r/rosen/software/miniconda/envs/cms2/lib/python3.8/site-packages/maggma/stores/mongolike.py", line 168, in connect
    db.authenticate(self.username, self.password)
  File "/global/homes/r/rosen/software/miniconda/envs/cms2/lib/python3.8/site-packages/pymongo/collection.py", line 2579, in __call__
    raise TypeError("'Collection' object is not callable. If you "
TypeError: 'Collection' object is not callable. If you meant to call the 'authenticate' method on a 'Database' object it is failing because no such method exists.
Andrew-S-Rosen commented 2 years ago

@utf, do you know of a current workaround for the above? Unfortunately, I haven't been able to store results from any Jobflow jobs and am struggling to find the combination of packages that gets it working again, even if temporarily...

utf commented 2 years ago

I just pushed a fixed to the main branch. If all tests pass I'll update the dependencies and release a new version. I think maggma changed the abstract Store interface.

utf commented 2 years ago

Fixed in v0.1.6