search

Materials-Consortia / optimade-tutorial-exercises

Tutorial exercises for the OPTIMADE API
https://optimade.org
MIT License
15 stars 7 forks source link

Fix pymatgen response fields issue in tutorial #9

Closed ml-evs closed 2 years ago

ml-evs commented 2 years ago

Add temporary workaround for #8 by pinning pymatgen to fixed/forked version.

This PR can be tested at Colab and Binder.

ml-evs commented 2 years ago

Lots of additional diff generated due to using a more recent jupyter version, unfortunately. Basically the only change is adding a cell that does:

# There is a bug with 2020.0.15 and 2020.0.16 this has been fixed in a fork, hopefully soon to be merged and added to next pymatgen release
# If this is not available, install directly from GitHub

# You may not want to run this cell if you are performing the tutorial within your own Python environment
if int(version("pymatgen").split(".")[-1]) < 17:
    !pip install git+https://github.com/ml-evs/pymatgen@fix_response_fields
ml-evs commented 2 years ago

The fix has now been merged into pymatgen, and I have verified that the install process here works. I have changed this PR to pin to master of pymatgen until the next release, and will open an issue tracking it so we can remove this workaround. Thanks again for raising this @JPBergsma, I'll merge now but feel free to take a look.

JPBergsma commented 2 years ago

Hi Matthew,

I just tried it in binder but when I try to execute : results_snls = OptimadeRester("odbx").get_snls(nelements=2, additional_response_fields=["_odbx_thermodynamics"])

I get the error message :

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/tmp/ipykernel_68/91642301.py in <module>
----> 1 results_snls = OptimadeRester("odbx").get_snls(nelements=2, additional_response_fields=["_odbx_thermodynamics"])

TypeError: get_snls() got an unexpected keyword argument 'additional_response_fields'

The pymatgen version is still listed as version(2022.0.16) Below is the feedback from your installation script.

Collecting git+https://github.com/materialsproject/pymatgen@master
  Cloning https://github.com/materialsproject/pymatgen (to revision master) to /tmp/pip-req-build-krpec93y
  Running command git clone --filter=blob:none -q https://github.com/materialsproject/pymatgen /tmp/pip-req-build-krpec93y
  Resolved https://github.com/materialsproject/pymatgen to commit 4fc889f742941fdcc6019c14c93ddd168f27c962
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: pandas in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (1.3.4)
Requirement already satisfied: tabulate in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (0.8.9)
Requirement already satisfied: palettable>=3.1.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (3.3.0)
Requirement already satisfied: ruamel.yaml>=0.15.6 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (0.17.17)
Requirement already satisfied: uncertainties>=3.1.4 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (3.1.6)
Requirement already satisfied: spglib>=1.9.9.44 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (1.16.2)
Requirement already satisfied: requests in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (2.26.0)
Requirement already satisfied: monty>=3.0.2 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (2021.8.17)
Requirement already satisfied: matplotlib>=1.5 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (3.5.0)
Requirement already satisfied: plotly>=4.5.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (5.4.0)
Requirement already satisfied: sympy in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (1.9)
Requirement already satisfied: numpy>=1.20.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (1.21.4)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (4.0.0)
Requirement already satisfied: networkx>=2.2 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (2.6.3)
Requirement already satisfied: scipy>=1.5.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (1.7.3)
Requirement already satisfied: setuptools-scm>=4 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib>=1.5->pymatgen==2022.0.16) (6.3.2)
Requirement already satisfied: packaging>=20.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib>=1.5->pymatgen==2022.0.16) (21.3)
Requirement already satisfied: cycler>=0.10 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib>=1.5->pymatgen==2022.0.16) (0.11.0)
Requirement already satisfied: python-dateutil>=2.7 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib>=1.5->pymatgen==2022.0.16) (2.8.2)
Requirement already satisfied: pillow>=6.2.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib>=1.5->pymatgen==2022.0.16) (8.4.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib>=1.5->pymatgen==2022.0.16) (1.3.2)
Requirement already satisfied: fonttools>=4.22.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib>=1.5->pymatgen==2022.0.16) (4.28.2)
Requirement already satisfied: pyparsing>=2.2.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib>=1.5->pymatgen==2022.0.16) (3.0.6)
Requirement already satisfied: tenacity>=6.2.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from plotly>=4.5.0->pymatgen==2022.0.16) (8.0.1)
Requirement already satisfied: six in /srv/conda/envs/notebook/lib/python3.7/site-packages (from plotly>=4.5.0->pymatgen==2022.0.16) (1.16.0)
Requirement already satisfied: ruamel.yaml.clib>=0.1.2 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ruamel.yaml>=0.15.6->pymatgen==2022.0.16) (0.2.6)
Requirement already satisfied: future in /srv/conda/envs/notebook/lib/python3.7/site-packages (from uncertainties>=3.1.4->pymatgen==2022.0.16) (0.18.2)
Requirement already satisfied: pytz>=2017.3 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pandas->pymatgen==2022.0.16) (2021.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests->pymatgen==2022.0.16) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests->pymatgen==2022.0.16) (2.0.0)
Requirement already satisfied: certifi>=2017.4.17 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests->pymatgen==2022.0.16) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests->pymatgen==2022.0.16) (3.1)
Requirement already satisfied: mpmath>=0.19 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from sympy->pymatgen==2022.0.16) (1.2.1)
Requirement already satisfied: setuptools in /srv/conda/envs/notebook/lib/python3.7/site-packages (from setuptools-scm>=4->matplotlib>=1.5->pymatgen==2022.0.16) (59.1.1)
Requirement already satisfied: tomli>=1.0.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from setuptools-scm>=4->matplotlib>=1.5->pymatgen==2022.0.16) (1.2.2)

When I only load your version,thus skipping the first code blocks.

I get the error below when I execute from pymatgen.ext.optimade import OptimadeRester

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
/tmp/ipykernel_67/3060016688.py in <module>
----> 1 from pymatgen.ext.optimade import OptimadeRester

/srv/conda/envs/notebook/lib/python3.7/site-packages/pymatgen/ext/optimade.py in <module>
     16 from pymatgen.core.periodic_table import DummySpecies
     17 from pymatgen.core.structure import Structure
---> 18 from pymatgen.util.provenance import StructureNL
     19 from pymatgen.util.sequence import PBar
     20 

/srv/conda/envs/notebook/lib/python3.7/site-packages/pymatgen/util/provenance.py in <module>
     15 from monty.json import MontyDecoder, MontyEncoder
     16 from monty.string import remove_non_ascii
---> 17 from pybtex import errors
     18 from pybtex.database.input import bibtex
     19 

ModuleNotFoundError: No module named 'pybtex'

The feed back from the installation is:

Collecting git+https://github.com/materialsproject/pymatgen@master
  Cloning https://github.com/materialsproject/pymatgen (to revision master) to /tmp/pip-req-build-so16_80u
  Running command git clone --filter=blob:none -q https://github.com/materialsproject/pymatgen /tmp/pip-req-build-so16_80u
  Resolved https://github.com/materialsproject/pymatgen to commit 4fc889f742941fdcc6019c14c93ddd168f27c962
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting uncertainties>=3.1.4
  Downloading uncertainties-3.1.6-py2.py3-none-any.whl (98 kB)
     |████████████████████████████████| 98 kB 3.0 MB/s             
Collecting tabulate
  Downloading tabulate-0.8.9-py3-none-any.whl (25 kB)
Collecting pandas
  Downloading pandas-1.3.4-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.3 MB)
     |████████████████████████████████| 11.3 MB 11.9 MB/s            
Collecting plotly>=4.5.0
  Downloading plotly-5.4.0-py2.py3-none-any.whl (25.3 MB)
     |████████████████████████████████| 25.3 MB 36.2 MB/s                     | 12.2 MB 36.2 MB/s eta 0:00:01
Collecting spglib>=1.9.9.44
  Downloading spglib-1.16.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (292 kB)
     |████████████████████████████████| 292 kB 46.2 MB/s            
Collecting palettable>=3.1.1
  Downloading palettable-3.3.0-py2.py3-none-any.whl (111 kB)
     |████████████████████████████████| 111 kB 23.2 MB/s            
Collecting sympy
  Downloading sympy-1.9-py3-none-any.whl (6.2 MB)
     |████████████████████████████████| 6.2 MB 36.8 MB/s            
Requirement already satisfied: numpy>=1.20.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (1.21.4)
Requirement already satisfied: requests in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (2.26.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (4.0.0)
Collecting monty>=3.0.2
  Downloading monty-2021.8.17-py3-none-any.whl (65 kB)
     |████████████████████████████████| 65 kB 2.2 MB/s             
Requirement already satisfied: scipy>=1.5.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (1.7.3)
Requirement already satisfied: matplotlib>=1.5 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (3.5.0)
Collecting networkx>=2.2
  Downloading networkx-2.6.3-py3-none-any.whl (1.9 MB)
     |████████████████████████████████| 1.9 MB 42.8 MB/s            
Requirement already satisfied: ruamel.yaml>=0.15.6 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pymatgen==2022.0.16) (0.17.17)
Requirement already satisfied: fonttools>=4.22.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib>=1.5->pymatgen==2022.0.16) (4.28.2)
Requirement already satisfied: pillow>=6.2.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib>=1.5->pymatgen==2022.0.16) (8.4.0)
Requirement already satisfied: setuptools-scm>=4 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib>=1.5->pymatgen==2022.0.16) (6.3.2)
Requirement already satisfied: python-dateutil>=2.7 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib>=1.5->pymatgen==2022.0.16) (2.8.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib>=1.5->pymatgen==2022.0.16) (1.3.2)
Requirement already satisfied: cycler>=0.10 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib>=1.5->pymatgen==2022.0.16) (0.11.0)
Requirement already satisfied: packaging>=20.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib>=1.5->pymatgen==2022.0.16) (21.3)
Requirement already satisfied: pyparsing>=2.2.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib>=1.5->pymatgen==2022.0.16) (3.0.6)
Requirement already satisfied: six in /srv/conda/envs/notebook/lib/python3.7/site-packages (from plotly>=4.5.0->pymatgen==2022.0.16) (1.16.0)
Collecting tenacity>=6.2.0
  Downloading tenacity-8.0.1-py3-none-any.whl (24 kB)
Requirement already satisfied: ruamel.yaml.clib>=0.1.2 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ruamel.yaml>=0.15.6->pymatgen==2022.0.16) (0.2.6)
Collecting future
  Downloading future-0.18.2.tar.gz (829 kB)
     |████████████████████████████████| 829 kB 29.4 MB/s            
  Preparing metadata (setup.py) ... done
Requirement already satisfied: pytz>=2017.3 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pandas->pymatgen==2022.0.16) (2021.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests->pymatgen==2022.0.16) (2.0.0)
Requirement already satisfied: certifi>=2017.4.17 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests->pymatgen==2022.0.16) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests->pymatgen==2022.0.16) (3.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests->pymatgen==2022.0.16) (1.26.7)
Collecting mpmath>=0.19
  Downloading mpmath-1.2.1-py3-none-any.whl (532 kB)
     |████████████████████████████████| 532 kB 49.7 MB/s            
Requirement already satisfied: setuptools in /srv/conda/envs/notebook/lib/python3.7/site-packages (from setuptools-scm>=4->matplotlib>=1.5->pymatgen==2022.0.16) (59.1.1)
Requirement already satisfied: tomli>=1.0.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from setuptools-scm>=4->matplotlib>=1.5->pymatgen==2022.0.16) (1.2.2)
Building wheels for collected packages: pymatgen, future
  Building wheel for pymatgen (pyproject.toml) ... done
  Created wheel for pymatgen: filename=pymatgen-2022.0.16-cp37-cp37m-linux_x86_64.whl size=3801017 sha256=6ef6d17acdd45bd655eb3e5926067f263334b210cfc8a3ce408b1259791f27bd
  Stored in directory: /tmp/pip-ephem-wheel-cache-dz8mw1sb/wheels/ac/b8/a4/4a4882e39e71c64cc53e1d5f4346f3fb643d5cb28cc1a6d293
  Building wheel for future (setup.py) ... done
  Created wheel for future: filename=future-0.18.2-py3-none-any.whl size=491070 sha256=03cf03c2aa777345fe71b746951da1dd2c539ac84e5d18812ca232a9961f70a2
  Stored in directory: /home/jovyan/.cache/pip/wheels/56/b0/fe/4410d17b32f1f0c3cf54cdfb2bc04d7b4b8f4ae377e2229ba0
Successfully built pymatgen future
Installing collected packages: tenacity, mpmath, future, uncertainties, tabulate, sympy, spglib, plotly, pandas, palettable, networkx, monty, pymatgen
Successfully installed future-0.18.2 monty-2021.8.17 mpmath-1.2.1 networkx-2.6.3 palettable-3.3.0 pandas-1.3.4 plotly-5.4.0 pymatgen-2022.0.16 spglib-1.16.2 sympy-1.9 tabulate-0.8.9 tenacity-8.0.1 uncertainties-3.1.6
ml-evs commented 2 years ago

Thanks for this report @JPBergsma! I wonder if we are just hitting a cache somewhere... it worked in Colab before I merged, and I assumed this would be harder than binder to please...

I'll make a commit that forcibly uninstalls the old version before calling pip install pymatgen@master again, which should definitely fix it.

The pymatgen version is still listed as version(2022.0.16) Below is the feedback from your installation script.

The pymatgen build metadata has not changed, so it will still register 2022.0.16 for now (bit annoying).

  Running command git clone --filter=blob:none -q https://github.com/materialsproject/pymatgen /tmp/pip-req-build-krpec93y
  Resolved https://github.com/materialsproject/pymatgen to commit 4fc889f742941fdcc6019c14c93ddd168f27c962

This looks like it is loading the right version at least, 4fc889 is the most recent commit.

When I only load your version,thus skipping the first code blocks.

I get the error below when I execute from pymatgen.ext.optimade import OptimadeRester

ModuleNotFoundError: No module named 'pybtex'

This is also to be expected, the OptimadeRester uses some non-standard pymatgen dependencies (and we don't want to pull all of the other optional ones), so it is expected that people still run the top cell first.

ml-evs commented 2 years ago

7885a65 definitely works in both Colab and Binder now 😅

JPBergsma commented 2 years ago

It works for me too under Binder.

JPBergsma commented 2 years ago

In Colab I still got this warning:

WARNING: The following packages were previously imported in this runtime: [pymatgen] You must restart the runtime in order to use newly installed versions. So let's hope that the students will notice it too and restart the Runtime.