search

janosh / matbench-discovery

An evaluation framework for machine learning models simulating high-throughput materials discovery.
https://matbench-discovery.materialsproject.org
MIT License
105 stars 17 forks source link

test_plots.py and test_preds.py failing #24

Closed pbenner closed 1 year ago

pbenner commented 1 year ago 
> pytest 
===================================================================================== test session starts =====================================================================================
platform linux -- Python 3.10.8, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/pbenner/Source/tmp/matbench-discovery, configfile: pyproject.toml, testpaths: tests
collected 34 items / 2 errors / 2 deselected / 32 selected                                                                                                                                    

=========================================================================================== ERRORS ============================================================================================
____________________________________________________________________________ ERROR collecting tests/test_plots.py _____________________________________________________________________________
tests/test_plots.py:18: in <module>
    from matbench_discovery.preds import load_df_wbm_with_preds
../../../.local/opt/anaconda3/envs/crysfeat/lib/python3.10/site-packages/matbench_discovery/preds.py:127: in <module>
    df_preds = load_df_wbm_with_preds().round(3)
../../../.local/opt/anaconda3/envs/crysfeat/lib/python3.10/site-packages/matbench_discovery/preds.py:96: in load_df_wbm_with_preds
    df = glob_to_df(PRED_FILES[model_name], pbar=False, **kwargs).set_index(id_col)
../../../.local/opt/anaconda3/envs/crysfeat/lib/python3.10/site-packages/matbench_discovery/data.py:170: in glob_to_df
    raise FileNotFoundError(f"No files matching glob {pattern=}")
E   FileNotFoundError: No files matching glob pattern='/home/pbenner/.local/opt/anaconda3/envs/crysfeat/lib/python3.10/site-packages/models//bowsr/2023-01-23-bowsr-megnet-wbm-IS2RE.csv'
--------------------------------------------------------------------------------------- Captured stderr ---------------------------------------------------------------------------------------
Loading preds:   0%|          | 0/8 [00:00<?, ?it/s, BOWSR + MEGNet]
____________________________________________________________________________ ERROR collecting tests/test_preds.py _____________________________________________________________________________
tests/test_preds.py:6: in <module>
    from matbench_discovery.preds import (
../../../.local/opt/anaconda3/envs/crysfeat/lib/python3.10/site-packages/matbench_discovery/preds.py:127: in <module>
    df_preds = load_df_wbm_with_preds().round(3)
../../../.local/opt/anaconda3/envs/crysfeat/lib/python3.10/site-packages/matbench_discovery/preds.py:96: in load_df_wbm_with_preds
    df = glob_to_df(PRED_FILES[model_name], pbar=False, **kwargs).set_index(id_col)
../../../.local/opt/anaconda3/envs/crysfeat/lib/python3.10/site-packages/matbench_discovery/data.py:170: in glob_to_df
    raise FileNotFoundError(f"No files matching glob {pattern=}")
E   FileNotFoundError: No files matching glob pattern='/home/pbenner/.local/opt/anaconda3/envs/crysfeat/lib/python3.10/site-packages/models//bowsr/2023-01-23-bowsr-megnet-wbm-IS2RE.csv'
--------------------------------------------------------------------------------------- Captured stderr ---------------------------------------------------------------------------------------
Loading preds:   0%|          | 0/8 [00:00<?, ?it/s, BOWSR + MEGNet]
=================================================================================== short test summary info ===================================================================================
ERROR tests/test_plots.py - FileNotFoundError: No files matching glob pattern='/home/pbenner/.local/opt/anaconda3/envs/crysfeat/lib/python3.10/site-packages/models//bowsr/2023-01-23-bowsr-...
ERROR tests/test_preds.py - FileNotFoundError: No files matching glob pattern='/home/pbenner/.local/opt/anaconda3/envs/crysfeat/lib/python3.10/site-packages/models//bowsr/2023-01-23-bowsr-...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
janosh commented 1 year ago

This looks like it's due to having both cloned and pip-installed the repo. If you pip uninstall -y matbench-discovery and pip install -e ./matbench-discovery the clone, I think it should work?

pbenner commented 1 year ago

Ok yes, with the editable install the tests are working. Would be nice to have it more pythonic and not depend on an editable installation.

janosh commented 1 year ago

Unless you clone, there should be no way to run the tests, right? (Except for the fact I'm currently bundling the test files with the package. I tried to exclude them. Not sure why that's not working.)

https://github.com/janosh/matbench-discovery/blob/dabd7d6d5c33d4711bab5716afeac9d9262dc5a8/pyproject.toml#L70-L71

In any case, the main reason for cloning I think would be to have an editable install? And in that case, the tests pass. So I think the problem here is minor. But of course, less potential pitfalls, more good. :)

janosh commented 1 year ago

Closing as I think there's nothing actionable here at the moment. Happy to re-open if something comes up.