tox fails with pinned numpy dependency

[bra-fsn]

I have an environment where we run tests with tox with pinned dependencies (normally they are listed in a dynamically generated test-requirements.txt and then added as deps = -rtest-requirements.txt in the tox.ini. setup.py has the dependencies listed without version constraints.

Started recently, running these tests fail with:

________________ ERROR collecting bidderbot/tests/test_misc.py _________________
ImportError while importing test module '/packages/python/bidderbot/bidderbot/tests/test_misc.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
__init__.pxd:942: in numpy.import_array
    ???
E   RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xf

During handling of the above exception, another exception occurred:
/usr/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
bidderbot/tests/test_misc.py:5: in <module>
    from bidderbot import utils
[...]
.tox/py/lib/python3.8/site-packages/bidderbot/utils.py:31: in <module>
    import talib as ta
.tox/py/lib/python3.8/site-packages/talib/__init__.py:93: in <module>
    from ._ta_lib import (
talib/_func.pxi:15: in init talib._ta_lib
    ???
__init__.pxd:944: in numpy.import_array
    ???
E   ImportError: numpy.core.multiarray failed to import

It seems that ta-lib is compiled with various (the pinned, 1.22.4 and the most recent 1.23.3) versions and used intermixed during the tests, but I can't see why. If I remove the version constraint from test-requirements.txt, it works, but we have other packages which depend on numpy==1.22.4, so this is not a usable solution here. If I add the version constraint to setup.py, I get the same error.

Sample tox.ini:

[tox]
envlist = py38,lint
toxworkdir={env:TOX_WORK_DIR:.tox}

[testenv]
pip_version = pip==22.2.2
setenv = MAKEFLAGS = -j8
basepython =
    {lint}: {env:TOXPYTHON:python}
    py38: {env:TOXPYTHON:python3.8}
passenv = *
install_command = pip install {opts} {packages}
deps =
    pytest-cov
    pytest-mock
    pytest-pep8
    -rtest-requirements.txt
usedevelop = false
commands = pytest -v .

Requirements file (just to reproduce, not the full one):

numpy==1.22.4
TA-Lib @ https://github.com/mrjbq7/ta-lib/archive/TA_Lib-0.4.25.zip
Cython==0.29.30

setup.py's requirements list:

install_requires = [
    "numpy",
    "TA-Lib",
]

Originally I had a simple TA-Lib, but switched to the archive in order not to install binaries (first I thought this is the problem). And ideas how this could be solved along with keeping the pinned versions?

[mrjbq7]

This is due to my including the generated C files so users don't need Cython...

Should I be building them against older numpy?

Is there a compatibility flag I can include to build against the older API but with newer numpy?

[mrjbq7]

I would imagine our new automatic wheels builder will have this issue also?

[bra-fsn]

This is due to my including the generated C files so users don't need Cython...

Should I be building them against older numpy?

Is there a compatibility flag I can include to build against the older API but with newer numpy?

I don't know about numpy API internals, but I can see there are a lot of API versions. Not sure if they have to match or they are forward-compatible. If adding cython as a dependency helps, that would be great to have that as an extra, so anybody in this situation could just install TA-Lib[cython].