numpy version incompatibility with module with single model off-the-shelf demo on laptop with only CPU

[rosskempner]

Hi,

I am running run_single.py with the following configurations:

import os
import shutil
from datetime import datetime
from pathlib import Path

from lfads_torch.run_model import run_model

# ---------- OPTIONS -----------
PROJECT_STR = "lfads-torch-example"
DATASET_STR = "nlb_mc_rtt"
RUN_TAG = datetime.now().strftime("%y%m%d") + "_exampleSingle"
RUN_DIR = Path("runs") / PROJECT_STR / DATASET_STR / RUN_TAG
OVERWRITE = True
# ------------------------------

# Overwrite the directory if necessary
if RUN_DIR.exists() and OVERWRITE:
    shutil.rmtree(RUN_DIR)
RUN_DIR.mkdir(parents=True)
# Copy this script into the run directory
shutil.copyfile(__file__, RUN_DIR / Path(__file__).name)
# Switch to the `RUN_DIR` and train the model
os.chdir(RUN_DIR)
run_model(
    overrides={
        "datamodule": "configs/datamodule/nlb_mc_rtt.yaml",
        "model": "configs/model/nlb_mc_rtt.yaml",
    },
    config_path="../configs/single.yaml",
)

And I am getting the following error message:

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.2 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "/xx/xx/xx/xx//Research/neural_computation/lfads_s1dz/testing_demo.py", line 8, in <module>
    from lfads_torch.run_model import run_model
  File "/xx/xx/xx/xx//Research/neural_computation/lfads_s1dz/lfads_torch/run_model.py", line 8, in <module>
    import pytorch_lightning as pl
  File "/xx/xx/opt/anaconda3/envs/lfads-torch/lib/python3.9/site-packages/pytorch_lightning/__init__.py", line 30, in <module>
    from pytorch_lightning.callbacks import Callback  # noqa: E402
  File "/xx/xx/opt/anaconda3/envs/lfads-torch/lib/python3.9/site-packages/pytorch_lightning/callbacks/__init__.py", line 14, in <module>
    from pytorch_lightning.callbacks.base import Callback
  File "/xx/xx/opt/anaconda3/envs/lfads-torch/lib/python3.9/site-packages/pytorch_lightning/callbacks/base.py", line 25, in <module>
    from pytorch_lightning.utilities.types import STEP_OUTPUT
  File "/xx/xx/opt/anaconda3/envs/lfads-torch/lib/python3.9/site-packages/pytorch_lightning/utilities/__init__.py", line 18, in <module>
    from pytorch_lightning.utilities.apply_func import move_data_to_device  # noqa: F401
  File "/xx/xx/opt/anaconda3/envs/lfads-torch/lib/python3.9/site-packages/pytorch_lightning/utilities/apply_func.py", line 41, in <module>
    _CPU_DEVICES = ("cpu", torch.device("cpu"))
/xx/xx/opt/anaconda3/envs/lfads-torch/lib/python3.9/site-packages/pytorch_lightning/utilities/apply_func.py:41: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/torch/csrc/utils/tensor_numpy.cpp:77.)
  _CPU_DEVICES = ("cpu", torch.device("cpu"))
Traceback (most recent call last):
  File "/xx/xx/xx/xx//Research/neural_computation/lfads_s1dz/testing_demo.py", line 8, in <module>
    from lfads_torch.run_model import run_model
  File "/xx/xx/xx/xx//Research/neural_computation/lfads_s1dz/lfads_torch/run_model.py", line 12, in <module>
    from ray import tune
  File "/xx/xx/opt/anaconda3/envs/lfads-torch/lib/python3.9/site-packages/ray/tune/__init__.py", line 2, in <module>
    from ray.tune.tune import run_experiments, run
  File "/xx/xx/opt/anaconda3/envs/lfads-torch/lib/python3.9/site-packages/ray/tune/tune.py", line 13, in <module>
    from ray.tune.analysis import ExperimentAnalysis
  File "/xx/xx/opt/anaconda3/envs/lfads-torch/lib/python3.9/site-packages/ray/tune/analysis/__init__.py", line 1, in <module>
    from ray.tune.analysis.experiment_analysis import ExperimentAnalysis
  File "/xx/xx/opt/anaconda3/envs/lfads-torch/lib/python3.9/site-packages/ray/tune/analysis/experiment_analysis.py", line 33, in <module>
    from ray.tune.execution.trial_runner import (
  File "/xx/xx/opt/anaconda3/envs/lfads-torch/lib/python3.9/site-packages/ray/tune/execution/trial_runner.py", line 25, in <module>
    from ray.tune.execution.ray_trial_executor import (
  File "/xx/xx/opt/anaconda3/envs/lfads-torch/lib/python3.9/site-packages/ray/tune/execution/ray_trial_executor.py", line 25, in <module>
    from ray.tune.logger import NoopLogger
  File "/xx/xx/opt/anaconda3/envs/lfads-torch/lib/python3.9/site-packages/ray/tune/logger/__init__.py", line 10, in <module>
    from ray.tune.logger.tensorboardx import TBXLogger, TBXLoggerCallback
  File "/xx/xx/opt/anaconda3/envs/lfads-torch/lib/python3.9/site-packages/ray/tune/logger/tensorboardx.py", line 25, in <module>
    class TBXLogger(Logger):
  File "/xx/xx/opt/anaconda3/envs/lfads-torch/lib/python3.9/site-packages/ray/tune/logger/tensorboardx.py", line 35, in TBXLogger
    VALID_NP_HPARAMS = (np.bool8, np.float32, np.float64, np.int32, np.int64)
  File "/xx/xx/opt/anaconda3/envs/lfads-torch/lib/python3.9/site-packages/numpy/__init__.py", line 410, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'bool8'

[arsedler9]

Thanks Ross, it looks like this is an incompatibility of ray 2.1.0 with the latest version of numpy (they didn't specify numpy<2 here). Can you try pip install numpy<2 and see if that fixes the problem? If so, I'll add it to requirements.txt.

[rosskempner]

pip install numpy<2 fixes it, thanks!