Maybe we should consider avoiding dependence on nipy

ganow commented 1 year ago

While creating a PR for new dependency management, including extra dependencies, I noticed that the nipy installation fails with Python>=3.10.

c.f. https://github.com/KamitaniLab/bdpy/pull/67#issuecomment-1600789516

A similar issue has already been discussed on the nipy GitHub issue, but unfortunately according to this discussion, nipy is "essentially unmaintained." In fact, it appears that the developers of the other packages that initiated this discussion have decided to replace the nipy-dependent portions of their codebase.

As far as I know at this time, bdpy currently uses at least the following features of nipy:

nipy.load_image
nipy.modalities.fmri.experimental_paradigm.{BlockParadigm,EventRelatedParadigm}

I don't know at this time how many resources would be needed to replace these codebases.

ganow commented 1 year ago

Why does `nipy` installation via `pyproject.toml` fail?

First, we assume the following structure for pyproject.toml:

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
...
dependencies = ["numpy<1.24", "nipy"]

nipy is a tool that depends on numpy. If you try to install it in the project folder with Python 3.10 or later using the command:

$ pip install .
...
Collecting nipy
  Using cached nipy-0.5.0.tar.gz (2.4 MB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
 ...
      ModuleNotFoundError: No module named 'numpy'

The installation fails. The cause of this error originates from the following procedure:

Since nipy is not provided in wheel format for Python 3.10 or later, the build tool attempts to build from source code in sdist format.
Because there's a statement from numpy.distutils.misc_util import appendpath in setup.py, an exception is thrown if numpy is not installed in the runtime before this line is evaluated.

This issue can be circumvented by first installing numpy only. In other words, the following installation process, for example, would not cause a problem:

$ pip install 'numpy<1.24'
$ pip install .  # includes installation of nipy

The problem with this approach is that it's not possible to declaratively describe the installation procedure. Also, as mentioned in the original issue, manual building in wheel format, as shown below, will pass without problems as long as numpy is installed beforehand.

$ pip install build
$ pip install 'numpy<1.24'
$ git clone https://github.com/nipy/nipy.git && cd nipy
$ python -m build

ganow commented 1 year ago

Update on 6/26

TL;DR

If we wait about several months to a year, the problem will be solved 🙌.

Detail

I found that a pyproject.toml was added to nipy about ~2mo ago. https://github.com/nipy/nipy/commit/adccc98f365b01123a6aaa59e3f4adeaae815170

The following procedures work on the current nipy master branch.

$ git clone https://github.com/nipy/nipy.git && cd nipy
$ pip install .

When the next version of nipy is released, it will likely be possible to put a declarative dependency on nipy in our pyproject.toml.

ganow commented 1 year ago

The following can solve the problem tentatively:

[project.optional-dependencies]
mri = [
    "nipy ; python_version < '3.10'",
    "nipy @ git+https://github.com/nipy/nipy.git@7eede02 ; python_version >= '3.10'"  # build from recent git commit
]

Update on 6/30: The nipy build does not pass in the first place with the Apple sillicon architecture. The above behavior is confirmed only on linux (x86_64).

KamitaniLab / bdpy