search

MolSSI / QCFractal

A distributed compute and database platform for quantum chemistry.
https://molssi.github.io/QCFractal/
BSD 3-Clause "New" or "Revised" License
144 stars 47 forks source link

Conda installation doesn't work #711

Closed keceli closed 11 months ago

keceli commented 2 years ago

Describe the bug Conda installation fails with either The environment is inconsistent, please check the package plan carefully or it proceeds with installing an older version (0.11.0) of QCFractal, which fails with ImportError: cannot import name 'Schema' from 'pydantic' (/usr/local/lib/python3.7/site-packages/pydantic/__init__.cpython-37m-x86_64-linux-gnu.so) when you import qcfractal. To Reproduce

molssi> docker run -i -t conda/miniconda3 /bin/bash
root@3f6c36b66f82:/# conda --version
conda 4.6.11
root@3f6c36b66f82:/# python --version
Python 3.7.3
root@9e7e7a8475a2:/# conda install qcfractal -c conda-forge
WARNING: The conda.compat module is deprecated and will be removed in a future release.
Collecting package metadata: done
Solving environment: | 
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:

  - defaults/linux-64::libedit==3.1.20181209=hc058e9b_0
  - defaults/linux-64::ruamel_yaml==0.15.46=py37h14c3975_0
  - defaults/linux-64::pysocks==1.6.8=py37_0
  - defaults/linux-64::sqlite==3.27.2=h7b6447c_0
  - defaults/linux-64::six==1.12.0=py37_0
  - defaults/linux-64::requests==2.21.0=py37_0
  - defaults/linux-64::ncurses==6.1=he6710b0_1
  - defaults/linux-64::openssl==1.1.1b=h7b6447c_1
  - defaults/linux-64::wheel==0.32.3=py37_0
  - defaults/linux-64::certifi==2019.3.9=py37_0
  - defaults/linux-64::pycosat==0.6.3=py37h14c3975_0
  - defaults/linux-64::urllib3==1.24.1=py37_0
  - defaults/linux-64::yaml==0.1.7=had09818_2
  - defaults/linux-64::pycparser==2.19=py37_0
  - defaults/linux-64::pip==18.1=py37_0
  - defaults/linux-64::conda==4.6.11=py37_0
  - defaults/linux-64::libffi==3.2.1=hd88cf55_4
  - defaults/linux-64::readline==7.0=h7b6447c_5
  - defaults/linux-64::libgcc-ng==8.2.0=hdf63c60_1
  - defaults/linux-64::cryptography==2.4.2=py37h1ba5d50_0
  - defaults/linux-64::setuptools==40.6.3=py37_0
  - defaults/linux-64::cffi==1.11.5=py37he75722e_1
  - defaults/linux-64::python==3.7.3=h0371630_0
  - defaults/linux-64::idna==2.8=py37_0
  - defaults/linux-64::chardet==3.0.4=py37_1
  - defaults/linux-64::asn1crypto==0.24.0=py37_0
  - defaults/linux-64::pyopenssl==18.0.0=py37_0
  - defaults/linux-64::zlib==1.2.11=h7b6447c_3
  - defaults/linux-64::tk==8.6.8=hbc83047_0
  - defaults/linux-64::xz==5.2.4=h14c3975_4

Expected behavior

conda installation doesn't have a prerequisite, so this was expected to run out of the box. I tried on Linux and Mac and got similar errors. Additional context

This issue is probably related to issue #695.

loriab commented 2 years ago

The interim solution is to specify the new version of qcfractal in the conda install command. I don't know why the environment solution is favoring the less-encumbered old qcf 0.11, but it is for now.

keceli commented 2 years ago

Thank you Lori. The installation with conda create -n qcf711_38 python=3.8 qcfractal=0.15.8.1 plotly=5.6.0 -c conda-forge worked. But, some unit tests (pytest --pyargs qcfractal) give error or fail. I should also note that adding geometric and psi4 to the conda environment creates even more problems, i.e. more tests fail since conda downgrades/upgrades various dependencies. Should I create a new issue for these problems?

On Darwin Kernel Version 21.3.0

qcelemental               0.24.0             pyhd8ed1ab_0    conda-forge
qcengine                  0.23.0             pyhd8ed1ab_0    conda-forge
qcfractal                 0.15.8.1         py38h50d1736_0    conda-forge
qcfractal-core            0.15.8.1         py38h50d1736_0    conda-forge
> conda --version
conda 4.12.0
> python --version
Python 3.8.12
> pytest --pyargs qcfractal
================================================================================ short test summary info ================================================================================
ERROR tests/test_collections.py::test_dataset_contributed_units[download_view] - AttributeError: module 'distutils' has no attribute 'version'
ERROR tests/test_collections.py::test_dataset_contributed_mixed_values[download_view] - AttributeError: module 'distutils' has no attribute 'version'
ERROR tests/test_collections.py::test_contributed_dataset_values_subset[download_view-True] - AttributeError: module 'distutils' has no attribute 'version'
ERROR tests/test_collections.py::test_contributed_dataset_values_subset[download_view-False] - AttributeError: module 'distutils' has no attribute 'version'
ERROR tests/test_collections.py::test_contributed_dataset_list_get_values[download_view] - AttributeError: module 'distutils' has no attribute 'version'
ERROR tests/test_collections.py::test_contributed_dataset_view_identical[download_view] - AttributeError: module 'distutils' has no attribute 'version'
ERROR tests/test_collections.py::test_contributed_dataset_plaintextview_write[download_view] - AttributeError: module 'distutils' has no attribute 'version'
ERROR tests/test_collections.py::test_dataset_contributed_units[remote_view] - ValueError: Remote view is not available.
ERROR tests/test_collections.py::test_dataset_contributed_mixed_values[remote_view] - ValueError: Remote view is not available.
ERROR tests/test_collections.py::test_contributed_dataset_values_subset[remote_view-True] - ValueError: Remote view is not available.
ERROR tests/test_collections.py::test_contributed_dataset_values_subset[remote_view-False] - ValueError: Remote view is not available.
ERROR tests/test_collections.py::test_contributed_dataset_list_get_values[remote_view] - ValueError: Remote view is not available.
ERROR tests/test_collections.py::test_contributed_dataset_view_identical[remote_view] - ValueError: Remote view is not available.
ERROR tests/test_collections.py::test_contributed_dataset_plaintextview_write[remote_view] - ValueError: Remote view is not available.

On CentOS Linux release 7.9

qcelemental               0.24.0             pyhd8ed1ab_0    conda-forge
qcengine                  0.23.0             pyhd8ed1ab_0    conda-forge
qcfractal                 0.15.8.1         py38h578d9bd_0    conda-forge
qcfractal-core            0.15.8.1         py38h578d9bd_0    conda-forge
> conda --version
conda 4.10.3
> python --version
Python 3.8.13
> pytest --pyargs qcfractal
================================================================================ short test summary info ================================================================================
FAILED tests/test_server.py::test_snowflakehandler_log - ConnectionRefusedError: [Errno Snowflake instance did not boot properly, try increasing the timeout.
FAILED tests/test_storage.py::test_results_pagination - AssertionError: assert 33 == (25 + 5)
FAILED tests/test_storage.py::test_procedure_pagination - sqlalchemy.exc.IntegrityError: (psycopg2.errors.ForeignKeyViolation) update or delete on table "molecule" violates foreign k...
FAILED tests/test_storage.py::test_mol_pagination - AssertionError: assert 1 == 0
FAILED tests/test_storage.py::test_mol_formula - AssertionError: assert 1 == 0
FAILED tests/test_storage.py::test_collections_include_exclude - sqlalchemy.exc.IntegrityError: (psycopg2.errors.ForeignKeyViolation) update or delete on table "molecule" violates fo...
ERROR tests/test_collections.py::test_dataset_contributed_units[download_view] - AttributeError: module 'distutils' has no attribute 'version'
ERROR tests/test_collections.py::test_dataset_contributed_mixed_values[download_view] - AttributeError: module 'distutils' has no attribute 'version'
ERROR tests/test_collections.py::test_contributed_dataset_values_subset[download_view-True] - AttributeError: module 'distutils' has no attribute 'version'
ERROR tests/test_collections.py::test_contributed_dataset_values_subset[download_view-False] - AttributeError: module 'distutils' has no attribute 'version'
ERROR tests/test_collections.py::test_contributed_dataset_list_get_values[download_view] - AttributeError: module 'distutils' has no attribute 'version'
ERROR tests/test_collections.py::test_contributed_dataset_view_identical[download_view] - AttributeError: module 'distutils' has no attribute 'version'
ERROR tests/test_collections.py::test_contributed_dataset_plaintextview_write[download_view] - AttributeError: module 'distutils' has no attribute 'version'
ERROR tests/test_collections.py::test_dataset_contributed_units[remote_view] - ValueError: Remote view is not available.
ERROR tests/test_collections.py::test_dataset_contributed_mixed_values[remote_view] - ValueError: Remote view is not available.
ERROR tests/test_collections.py::test_contributed_dataset_values_subset[remote_view-True] - ValueError: Remote view is not available.
ERROR tests/test_collections.py::test_contributed_dataset_values_subset[remote_view-False] - ValueError: Remote view is not available.
ERROR tests/test_collections.py::test_contributed_dataset_list_get_values[remote_view] - ValueError: Remote view is not available.
ERROR tests/test_collections.py::test_contributed_dataset_view_identical[remote_view] - ValueError: Remote view is not available.
ERROR tests/test_collections.py::test_contributed_dataset_plaintextview_write[remote_view] - ValueError: Remote view is not available.
ERROR tests/test_storage.py::test_server_log - AssertionError: assert 1 == 0

If I do conda create -n qcf01581 python=3.8 qcfractal=0.15.8.1 plotly=5.6.0 geometric rdkit psi4 -c psi4 -c conda-forge on my mac, I get:

(qcf01581) > python -c "from qcfractal import FractalSnowflakeHandler; server = FractalSnowflakeHandler()"
/Users/keceli/miniconda3/envs/qcf01581/lib/python3.8/site-packages/numpy/__init__.py:159: UserWarning: mkl-service package failed to import, therefore Intel(R) MKL initialization ensuring its correct out-of-the box operation under condition when Gnu OpenMP had already been loaded by Python process is not assured. Please install mkl-service package, see http://github.com/IntelPython/mkl-service
  from . import _distributor_init
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/keceli/tmp/molssi/QCFractal/qcfractal/__init__.py", line 5, in <module>
    from . import interface
  File "/Users/keceli/tmp/molssi/QCFractal/qcfractal/interface/__init__.py", line 5, in <module>
    from . import collections, data, models, util
  File "/Users/keceli/tmp/molssi/QCFractal/qcfractal/interface/collections/__init__.py", line 7, in <module>
    from .dataset_view import DatasetView, HDF5View, PlainTextView, RemoteView
  File "/Users/keceli/tmp/molssi/QCFractal/qcfractal/interface/collections/dataset_view.py", line 14, in <module>
    import h5py
  File "/Users/keceli/miniconda3/envs/qcf01581/lib/python3.8/site-packages/h5py/__init__.py", line 33, in <module>
    from . import version
  File "/Users/keceli/miniconda3/envs/qcf01581/lib/python3.8/site-packages/h5py/version.py", line 15, in <module>
    from . import h5 as _h5
  File "h5py/h5.pyx", line 1, in init h5py.h5
ImportError: dlopen(/Users/keceli/miniconda3/envs/qcf01581/lib/python3.8/site-packages/h5py/defs.cpython-38-darwin.so, 0x0002): Symbol not found: _H5Pget_fapl_ros3
  Referenced from: /Users/keceli/miniconda3/envs/qcf01581/lib/python3.8/site-packages/h5py/defs.cpython-38-darwin.so
  Expected in: /Users/keceli/miniconda3/envs/qcf01581/lib/libhdf5.103.dylib
MiloszGrabski commented 2 years ago

Using conda install -c conda-forge qcfractal=0.15.8.1on windows gives: PackagesNotFoundError: The following packages are not available from current channels: image

bennybp commented 11 months ago

I think all the conda packages are working (both v0.15 and the new v0.50).

qcfractal does not support windows (and in v0.50 it's explicitly marked that way). However, qcportal should be fine on windows.

(Also, mamba is usually the way to go, and I can't wait until it becomes the default in conda :)