Installation causes numpy ModuleNotFoundError

[buddejul]

Bug description

When I add optimagic to environment.yml (either conda-forge or pip), and update the environment I get

ModuleNotFoundError: No module named 'numpy._core.numeric'

in a pd.read_pickle() call (in a task collected by pytask).

Afaik this could point to issues with the numpy installation.

To Reproduce

If it helps I can check this with a smaller example some time soon!

Screenshots/Error messages

System

• OS: Windows 11 10.0.26100
• Version: 0.5.0

---
name: thesis
channels: [conda-forge, r, nodefaults]
dependencies:
  - python ==3.12
  - pip >=21.1

  # Development dependencies
  - conda-lock
  - ipykernel
  - jupyterlab
  - pre-commit

  # Language agnostic template project dependencies
  - pytask >=0.5.0
  - pytask-latex >=0.4.2
  - pytask-parallel >=0.5.0
  - pytest
  - pytest-cov
  - pytest-xdist

  # Python template project dependencies
  - statsmodels
  - numpy
  - pandas >=2.2
  - plotly >=5.2.0,<6

  # R template project dependencies
  # Currently we are not using R hence we do not need to install these dependencies.
  # - pytask-r >=0.4.1
  # - pyreadr
  # - r-plyr
  # - r-precommit
  # - r-yaml
  # - r-forcats

  # Optimization
  # - optimagic

  # Install project
  - pip:
    - -e .
    - pdbp
    - kaleido==0.1.0post1
    - joblib
    - git+https://github.com/buddejul/pyvmte@fix_second_step_error
    - coptpy
    # - optimagic

[timmens]

Hey @buddejul, thanks for opening an issue!

Can you provide a minimal reproducible example? If your hunch is correct, it should be enough to have

1. An environment file with specific versions of numpy, pandas, and optimagic,
2. A code snippet where you save something in pickle format and then try to read it with pandas.

[hmgaudecker]

Looks numpy 2.0-related to me. Instead of updating, can you remove and recreate the environment?

[hmgaudecker]

Or did optimagic cause numpy to be downgraded and now you are trying to read a pickle file created with a more recent version?

[buddejul]

Hi everyone, thanks for the swift replies! Got a small example - looks like @hmgaudecker second guess is right, it's the downgrade. So of course no problem with optimagic. ~But maybe that is something users might be reminded of e.g. in the Installation Docs?~ Thinking about it - I guess it's on the user to be more careful with fixing numpy if they actually need it. In that case I would have caught the error earlier.

---

I guess obvious, but to reproduce:

• python bug_write.py using environment bug_numpy_2
• python bug_read.py using environment bug

I pinned the packages at the versions I found in my original environment (except for numpy in bug_numpy_2).

import pandas as pd # type: ignore[import-untyped]
import numpy as np

if __name__ == "__main__":
    data = pd.DataFrame(np.random.randn(6, 4), columns=list('ABCD'))

    # Save data as pickle
    data.to_pickle('data.pkl')

import pandas as pd # type: ignore[import-untyped]
import numpy as np

if __name__ == "__main__":
    data = pd.read_pickle('data.pkl')

---
name: bug
channels: [conda-forge, r, nodefaults]
dependencies:
  - python==3.12
  - numpy==1.26.4
  - pandas==2.2.3  
  - optimagic==0.5.0

---
name: bug_numpy_2
channels: [conda-forge, r, nodefaults]
dependencies:
  - python==3.12
  - numpy==2.1.2
  - pandas==2.2.3  

[timmens]

Great! If the issue is resolved for you, you can close it!