search

MolSSI / QCFractal

A distributed compute and database platform for quantum chemistry.
https://molssi.github.io/QCFractal/
BSD 3-Clause "New" or "Revised" License
144 stars 47 forks source link

Missing dependencies in Quickstart notebook environment #695

Closed lilyminium closed 12 months ago

lilyminium commented 2 years ago

Describe the bug

This is maybe more of an issue for requests and alembic, but using the default provided environment in the Quickstart environment I can't run the notebook on my Mac OS Catalina.

Quickstart also installs QCelemental 0.23.0 for QCFractal 0.15.6 which doesn't work. (#690)

Also, Python <= 3.7 is pinned in the environment. This is about to become unsupported by newer numpy releases (https://numpy.org/neps/nep-0029-deprecation_policy.html) from Dec 26 2021.

To Reproduce

conda env create qcarchive/qcfractal-snowflake -n snowflake
conda activate snowflake conda && python -m ipykernel install --user --name snowflake --display-name "Python (snowflake)"

chardet

>>> from qcfractal import FractalSnowflakeHandler

~/anaconda3/envs/snowflake/lib/python3.7/site-packages/qcfractal/interface/collections/dataset.py in <module>
     10 import numpy as np
     11 import pandas as pd
---> 12 import requests
     13 from pydantic import Field, validator
     14 from qcelemental import constants

~/.local/lib/python3.7/site-packages/requests/__init__.py in <module>
     42 
     43 import urllib3
---> 44 import chardet
     45 import warnings
     46 from .exceptions import RequestsDependencyWarning

ModuleNotFoundError: No module named 'chardet'

importlib_resources

The error message is not actually helpful, but if you print stderr (below) you can see alembic. I'm not sure how to get the logger to log inside Jupyter or the terminal.

>>> server = FractalSnowflakeHandler()

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/var/folders/rv/j6lbln6j0kvb5svxj8wflc400000gn/T/ipykernel_72793/3909637800.py in <module>
----> 1 server = FractalSnowflakeHandler()
      2 server

~/anaconda3/envs/snowflake/lib/python3.7/site-packages/qcfractal/snowflake.py in __init__(self, ncores)
    214         self._running = False
    215         self._qcfractal_proc = None
--> 216         self._storage = TemporaryPostgres()
    217         self._storage_uri = self._storage.database_uri(safe=False)
    218         self._qcfdir = None

~/anaconda3/envs/snowflake/lib/python3.7/site-packages/qcfractal/postgres_harness.py in __init__(self, database_name, tmpdir, quiet, logger)
    487         self.psql = PostgresHarness(self.config)
    488         self.psql.initialize_postgres()
--> 489         self.psql.init_database()
    490 
    491         atexit.register(self.stop)

~/anaconda3/envs/snowflake/lib/python3.7/site-packages/qcfractal/postgres_harness.py in init_database(self)
    385         if ret["retcode"] != 0:
    386             self.logger(ret)
--> 387             raise ValueError("\nFailed to Stamp the database with current version.\n")
    388 
    389     def backup_database(self, filename: Optional[str] = None) -> None:

ValueError: 
Failed to Stamp the database with current version.

Stderr:

  File "/Users/lily/anaconda3/envs/snowflake/lib/python3.7/site-packages/alembic/util/compat.py", line 33, in <module>
    import importlib_resources  # type:ignore[no-redef] # noqa
ModuleNotFoundError: No module named 'importlib_resources'

Perhaps a more informative error message would attach stderr to the ValueError, e.g.:

- raise ValueError("\nFailed to Stamp the database with current version.\n")
+ raise ValueError("Failed to Stamp the database with current version. "
+                  f"stderr: {ret['stderr']}")

Expected behavior

Additional context

Initial environment before installing missing deps:

# packages in environment at /Users/lily/anaconda3/envs/snowflake:
#
# Name                    Version                   Build  Channel
alembic                   1.7.4              pyhd3eb1b0_0
ambit                     0.5.1                he7184a3_1    psi4
appnope                   0.1.2           py37hecd8cb5_1001
argcomplete               1.12.3             pyhd3eb1b0_0
argon2-cffi               20.1.0           py37h9ed2024_1
arrow-cpp                 0.13.0           py37h8cfbac2_0
asn1crypto                1.4.0                      py_0
async_generator           1.10             py37h28b3542_0
attrs                     21.2.0             pyhd3eb1b0_0
backcall                  0.2.0              pyhd3eb1b0_0
bcrypt                    3.2.0            py37haf1e3a3_0
blas                      1.0                         mkl
bleach                    4.0.0              pyhd3eb1b0_0
boost                     1.68.0          py37h9888f84_1001    conda-forge
boost-cpp                 1.68.0            h6f8c590_1000    conda-forge
bottleneck                1.3.2            py37hf1fa96c_1
brotli                    1.0.9                hb1e8313_2
brotlipy                  0.7.0           py37h9ed2024_1003
bzip2                     1.0.8                h1de35cc_0
ca-certificates           2021.10.26           hecd8cb5_2
cached-property           1.5.2                      py_0
cairo                     1.16.0            h0ab9d94_1001    conda-forge
certifi                   2021.10.8        py37hecd8cb5_0
cffi                      1.14.0           py37hb5b8e2f_0
charset-normalizer        2.0.4              pyhd3eb1b0_0
chemps2                   1.8.9                he7184a3_2    psi4
cryptography              2.3.1            py37hdbc3d79_0
dbus                      1.13.18              h18a8e69_0
debugpy                   1.4.1            py37h23ab428_0
decorator                 5.1.0              pyhd3eb1b0_0
defusedxml                0.7.1              pyhd3eb1b0_0
dftd3                     3.2.1                h4f947d3_1    psi4
dkh                       1.2                  h97561c3_4    psi4
double-conversion         3.1.5                haf313ee_1
entrypoints               0.3                      py37_0
expat                     2.4.1                h23ab428_2
fontconfig                2.13.1               ha9ee91d_0
freetype                  2.11.0               hd8bbffd_0
gau2grid                  2.0.7                h35c211d_1    conda-forge
gcp                       2.0.2                h0e1e685_0    psi4
gdma                      2.2.6                h97561c3_6    psi4
geometric                 0.9.7.2                    py_0    conda-forge
gettext                   0.21.0               h7535e17_0
gflags                    2.2.2                h0a44026_0
giflib                    5.2.1                haf1e3a3_0
glib                      2.63.1               hd977a24_0
glog                      0.5.0                h23ab428_0
h5py                      3.2.1            py37h90fc2a2_0
hdf5                      1.10.6               hdbbcd12_0
icu                       58.2                 h0a44026_3
idna                      3.2                pyhd3eb1b0_0
importlib-metadata        4.8.1            py37hecd8cb5_0
importlib_metadata        4.8.1                hd3eb1b0_0
iniconfig                 1.1.1              pyhd3eb1b0_0
intel-openmp              2021.3.0          hecd8cb5_3375
ipykernel                 6.4.1            py37hecd8cb5_1
ipython                   7.27.0           py37h01d92e1_0
ipython_genutils          0.2.0                    py37_0
ipywidgets                7.6.5              pyhd3eb1b0_1
jedi                      0.18.0           py37hecd8cb5_1
jinja2                    3.0.1              pyhd3eb1b0_0
jpeg                      9d                   h9ed2024_0
jsonschema                3.2.0                    py37_1
jupyter                   1.0.0                    py37_7
jupyter_client            7.0.1              pyhd3eb1b0_0
jupyter_console           6.4.0              pyhd3eb1b0_0
jupyter_core              4.8.1            py37hecd8cb5_0
jupyterlab_pygments       0.1.2                      py_0
jupyterlab_widgets        1.0.0              pyhd3eb1b0_1
krb5                      1.16.1               h24a3359_6
lcms2                     2.12                 hf1fd2bf_0
libboost                  1.67.0               hebc422b_4
libcxx                    12.0.0               h2f01273_0
libedit                   3.1.20210714         h9ed2024_0
libevent                  2.1.8                h58d6694_0
libffi                    3.2.1             h0a44026_1007
libgfortran               3.0.1                h93005f0_2
libiconv                  1.16                 h1de35cc_0
libint2                   2.6.0                h879752b_3    psi4
libpng                    1.6.37               ha441bb4_0
libpq                     10.5                 hf30b1f0_0
libprotobuf               3.6.0                hd9629dc_0
libsodium                 1.0.18               h1de35cc_0
libtiff                   4.1.0                hcb84e12_0
libwebp                   1.2.0                hacca55c_0
libxc                     5.1.7            py37h271585c_0    conda-forge
libxml2                   2.9.12               hcdb78fc_0
llvm-openmp               12.0.0               h0dcd299_1
lz4-c                     1.8.1.2              h1de35cc_0
mako                      1.1.4              pyhd3eb1b0_0
markupsafe                2.0.1            py37h9ed2024_0
matplotlib-inline         0.1.2              pyhd3eb1b0_2
mistune                   0.8.4            py37h1de35cc_0
mkl                       2019.4                      233
mkl-service               2.3.0            py37h9ed2024_0
mkl_fft                   1.3.0            py37ha059aab_0
mkl_random                1.1.1            py37h959d312_0
more-itertools            8.10.0             pyhd3eb1b0_0
msgpack-python            1.0.2            py37hf7b0b51_1
nbclient                  0.5.3              pyhd3eb1b0_0
nbconvert                 6.1.0            py37hecd8cb5_0
nbformat                  5.1.3              pyhd3eb1b0_0
ncurses                   6.2                  h0a44026_1
nest-asyncio              1.5.1              pyhd3eb1b0_0
networkx                  2.6.3              pyhd3eb1b0_0
nglview                   3.0.3              pyh8a188c0_0    conda-forge
notebook                  6.4.5            py37hecd8cb5_0
numexpr                   2.7.3            py37h16bde0e_0
numpy                     1.19.2           py37h456fd55_0
numpy-base                1.19.2           py37hcfb5961_0
olefile                   0.46                     py37_0
openssl                   1.0.2u               h1de35cc_0
packaging                 21.0               pyhd3eb1b0_0
pandas                    1.3.3            py37h5008ddb_0
pandocfilters             1.4.3            py37hecd8cb5_1
parso                     0.8.2              pyhd3eb1b0_0
pcmsolver                 1.2.1.1          py37h23dd00a_0    psi4
pcre                      8.45                 h23ab428_0
pexpect                   4.8.0                    py37_0
pickleshare               0.7.5                    py37_0
pillow                    8.4.0            py37h98e4679_0
pint                      0.18               pyhd8ed1ab_0    conda-forge
pip                       21.2.2           py37hecd8cb5_0
pixman                    0.38.0               h1de35cc_0
plotly                    5.1.0              pyhd3eb1b0_0
pluggy                    0.13.1           py37hecd8cb5_0
postgresql                10.5                 hf30b1f0_0
prometheus_client         0.11.0             pyhd3eb1b0_0
prompt-toolkit            3.0.20             pyhd3eb1b0_0
prompt_toolkit            3.0.20               hd3eb1b0_0
psi4                      1.4.1+cd00f19    py37h9453f11_0    psi4
psutil                    5.8.0            py37h9ed2024_1
psycopg2                  2.7.5            py37hdbc3d79_0
ptyprocess                0.7.0              pyhd3eb1b0_2
py                        1.10.0             pyhd3eb1b0_0
py-cpuinfo                8.0.0              pyhd3eb1b0_1
pyarrow                   0.13.0           py37h0a44026_0
pycairo                   1.19.1           py37h06c6e95_0
pycparser                 2.20                       py_2
pydantic                  1.8.2            py37h9ed2024_0
pygments                  2.10.0             pyhd3eb1b0_0
pyopenssl                 19.0.0                   py37_0
pyparsing                 2.4.7              pyhd3eb1b0_0
pyqt                      5.9.2            py37h655552a_2
pyrsistent                0.17.3           py37haf1e3a3_0
pysocks                   1.7.1            py37hecd8cb5_0
pytest                    6.2.4            py37hecd8cb5_2
python                    3.7.0                hc167b69_0
python-dateutil           2.8.2              pyhd3eb1b0_0
python-editor             1.0.4              pyhd3eb1b0_0
python_abi                3.7                     2_cp37m    conda-forge
pytz                      2021.3             pyhd3eb1b0_0
pyyaml                    5.4.1            py37h9ed2024_1
pyzmq                     22.2.1           py37h23ab428_1
qcelemental               0.23.0             pyhd8ed1ab_0    conda-forge
qcengine                  0.20.1             pyhd8ed1ab_0    conda-forge
qcfractal                 0.15.6           py37hf985489_0    conda-forge
qcfractal-core            0.15.6           py37hf985489_0    conda-forge
qcportal                  0.15.6             pyhd8ed1ab_0    conda-forge
qt                        5.9.7                h468cd18_1
qtconsole                 5.1.1              pyhd3eb1b0_0
qtpy                      1.10.0             pyhd3eb1b0_0
rdkit                     2019.03.2        py37haff9cca_0    conda-forge
re2                       2020.11.01           h23ab428_1
readline                  7.0                  h1de35cc_5
requests                  2.26.0             pyhd3eb1b0_0
send2trash                1.8.0              pyhd3eb1b0_1
setuptools                58.0.4           py37hecd8cb5_0
simint                    0.7                  h9ec563c_2    psi4
sip                       4.19.8           py37h0a44026_0
six                       1.15.0           py37hecd8cb5_0
snappy                    1.1.8                hb1e8313_0
sqlalchemy                1.3.23           py37h9ed2024_0
sqlite                    3.33.0               hffcf06c_0
tenacity                  8.0.1            py37hecd8cb5_0
terminado                 0.9.4            py37hecd8cb5_0
testpath                  0.5.0              pyhd3eb1b0_0
thrift-cpp                0.11.0               h443fa2d_3
tk                        8.6.11               h7bc2e8c_0
toml                      0.10.2             pyhd3eb1b0_0
tornado                   6.1              py37h9ed2024_0
tqdm                      4.62.3             pyhd3eb1b0_1
traitlets                 5.1.0              pyhd3eb1b0_0
typing-extensions         3.10.0.2             hd3eb1b0_0
typing_extensions         3.10.0.2           pyh06a4308_0
urllib3                   1.26.7             pyhd3eb1b0_0
wcwidth                   0.2.5              pyhd3eb1b0_0
webencodings              0.5.1                    py37_1
wheel                     0.37.0             pyhd3eb1b0_1
widgetsnbextension        3.5.1                    py37_0
xz                        5.2.5                h1de35cc_0
yaml                      0.2.5                haf1e3a3_0
zeromq                    4.3.4                h23ab428_0
zipp                      3.6.0              pyhd3eb1b0_0
zlib                      1.2.11               h1de35cc_3
zstd                      1.3.7                h5bba6e5_0
bennybp commented 12 months ago

I think this is resolved? If not, please re-open (especially if this applies to the new v0.50 release)