numpy deprecated `np.float` since `1.24.0`

[dingp]

When trying to build the external software stack for DUNE DAQ with the latest versions of available packages, I tripped over moo since it uses np.float which is deprecated since numpy 1.20.0 and removed in 1.23.0.

     177      File "/cvmfs/dunedaq.opensciencegrid.org/spack/externals/ext-v2.0/spack-0.20.0-gcc-12.1.0/spack
            -0.20.0/opt/spack/linux-almalinux9-zen3/gcc-12.1.0/py-numpy-1.24.3-kzb2fw3rp33kzu6iqegnakmcsjqkqy
            cz/lib/python3.10/site-packages/numpy/__init__.py", line 305, in __getattr__
     178        raise AttributeError(__former_attrs__[attr])
     179    AttributeError: module 'numpy' has no attribute 'float'.
     180    `np.float` was a deprecated alias for the builtin `float`. To avoid this error in existing code,
            use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically w
            anted the numpy scalar type, use `np.float64` here.
     181    The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the origin
            al release note at:
     182        https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations. Did you mean: 'cfloat'?

[brettviren]

Hi @dingp Sorry for delay, just seeing this now.

Can you show a traceback? I don't find any use of np.float or similar in moo.

[dingp]

Thanks @brettviren After having a second look, it seems the version of openpyxl is the culprit. 3.0.5 is the version used in my test.

It would be great to set version constraints on it in the setup.py.

This is the full section of the error message:

[2/14] Generating codegen/include/cmdlib/cmd/Nljs.hpp
FAILED: cmdlib/codegen/include/cmdlib/cmd/Nljs.hpp /scratch/dingpf/dev-fddaq-0822/build/cmdlib/codegen/include/cmdlib/cmd/Nljs.hpp
cd /scratch/dingpf/dev-fddaq-0822/build && /cvmfs/dunedaq-development.opensciencegrid.org/nightly/NB23-08-22/spack-0.18.1-gcc-12.1.0/spack-0.18.1/opt/spack/gcc-12.1.0/py-moo-0.6.3-qsvni3n5shrp6vyooootzonemxmn6piy/bin/moo -T/scratch/dingpf/dev-fddaq-0822/sourcecode/cmdlib/schema -M/scratch/dingpf/dev-fddaq-0822/sourcecode/cmdlib/schema -g /lang:ocpp.jsonnet -A path=dunedaq.cmdlib.cmd -A ctxpath=dunedaq -A os=cmdlib/cmd.jsonnet render -o /scratch/dingpf/dev-fddaq-0822/build/cmdlib/codegen/include/cmdlib/cmd/Nljs.hpp omodel.jsonnet onljs.hpp.j2 && /cvmfs/dunedaq-development.opensciencegrid.org/nightly/NB23-08-22/spack-0.18.1-gcc-12.1.0/spack-0.18.1/opt/spack/gcc-12.1.0/py-moo-0.6.3-qsvni3n5shrp6vyooootzonemxmn6piy/bin/moo -T/scratch/dingpf/dev-fddaq-0822/sourcecode/cmdlib/schema -M/scratch/dingpf/dev-fddaq-0822/sourcecode/cmdlib/schema -g /lang:ocpp.jsonnet -A path=dunedaq.cmdlib.cmd -A ctxpath=dunedaq -A os=cmdlib/cmd.jsonnet render-deps -t cmdlib/codegen/include/cmdlib/cmd/Nljs.hpp -o /scratch/dingpf/dev-fddaq-0822/build/cmdlib/codegen/include/cmdlib/cmd/Nljs.hpp.d omodel.jsonnet onljs.hpp.j2
Traceback (most recent call last):
  File "/cvmfs/dunedaq-development.opensciencegrid.org/nightly/NB23-08-22/spack-0.18.1-gcc-12.1.0/spack-0.18.1/opt/spack/gcc-12.1.0/py-moo-0.6.3-qsvni3n5shrp6vyooootzonemxmn6piy/bin/moo", line 7, in <module>
    from moo.__main__ import main
  File "/cvmfs/dunedaq-development.opensciencegrid.org/nightly/NB23-08-22/spack-0.18.1-gcc-12.1.0/spack-0.18.1/opt/spack/gcc-12.1.0/py-moo-0.6.3-qsvni3n5shrp6vyooootzonemxmn6piy/lib/python3.10/site-packages/moo/__init__.py", line 18, in <module>
    import moo.xls
  File "/cvmfs/dunedaq-development.opensciencegrid.org/nightly/NB23-08-22/spack-0.18.1-gcc-12.1.0/spack-0.18.1/opt/spack/gcc-12.1.0/py-moo-0.6.3-qsvni3n5shrp6vyooootzonemxmn6piy/lib/python3.10/site-packages/moo/xls.py", line 4, in <module>
    from openpyxl import load_workbook
  File "/scratch/dingpf/dev-fddaq-0822/.venv/lib/python3.10/site-packages/openpyxl/__init__.py", line 4, in <module>
    from openpyxl.compat.numbers import NUMPY, PANDAS
  File "/scratch/dingpf/dev-fddaq-0822/.venv/lib/python3.10/site-packages/openpyxl/compat/__init__.py", line 3, in <module>
    from .numbers import NUMERIC_TYPES
  File "/scratch/dingpf/dev-fddaq-0822/.venv/lib/python3.10/site-packages/openpyxl/compat/numbers.py", line 41, in <module>
    numpy.float,
  File "/scratch/dingpf/dev-fddaq-0822/.venv/lib/python3.10/site-packages/numpy/__init__.py", line 284, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'float'. Did you mean: 'cfloat'?

This is how to reproduce the issue:

source /cvmfs/dunedaq.opensciencegrid.org/setup_dunedaq.sh
setup_dbt latest
dbt-create -c -n last_fddaq dev-fddaq-0822
cd dev-fddaq-0822/
source env.sh
pip install numpy==1.24.0
cd sourcecode/
git clone https://github.com/DUNE-DAQ/cmdlib
dbt-build

[dingp]

Upgrading to the latest openpyxl==3.1.2 with numpy==1.24.0 worked. Though I haven't tested the minimum versions of openpyxl needed.

[brettviren]

Hi @dingp

Upgrading to the latest openpyxl==3.1.2 with numpy==1.24.0 worked.

I'm a little unclear how version constraints are meant to be applied here. Your example shows a manual install of a particular version of numpy while, I guess, the remaining Python packages are installed via Spack? Does Spack somehow utilize version constraints in, for example, moo/setup.py?

I guess my question is: do you expect/want moo/setup.py to add, eg, "openpyxl>=3.1.2"?

[dingp]

It turned out that I only had the latest version of py-openpyxl to 3.0.5 in its package.py. Spack will try to install the latest version of a given package if no version specified.

Sorry for noise.