search

marcelm / dnaio

Efficiently read and write sequencing data from Python
https://dnaio.readthedocs.io/
MIT License
62 stars 9 forks source link

installation from source with `pip install` fails #138

Closed boegel closed 3 months ago

boegel commented 3 months ago

Running this command:

python -m pip install --prefix=/software/cutadapt/4.9-GCCcore-12.3.0  --verbose  --no-deps  --ignore-installed  --no-index  --no-build-isolation  .

in the unpacked source directory for dnaio v1.2.1 fails with:

  gcc -DNDEBUG -g -fwrapv -O3 -Wall -O2 -ftree-vectorize -march=native -fno-math-errno -fPIC -I/software/Python/3.11.3-GCCcore-12.3.0/include/python3.11 -c src/dnaio/_core.c -o build/temp.linux-x86_64-cpython-311/src/dnaio/_core.o
  cc1: fatal error: src/dnaio/_core.c: No such file or directory
  compilation terminated.
  error: command '/software/GCCcore/12.3.0/bin/gcc' failed with exit code 1
  error: subprocess-exited-with-error

The error is pretty clear, and based on #83 I know that src/dnaio/_core.c is supposed to be generated automatically, but in this case that's clearly not happening.

I guess this is because the bit in setup.py that is responsible for this isn't being picked up, perhaps pip ignores the setup.py when there's a pyproject.toml around (not sure about that)

Any suggestions here? Is there a way to manually trigger the generation of _core.c before running the pip install command?

marcelm commented 3 months ago

Hi, the .c files are generated by Cython at build time. Cython is therefore listed as a build dependency in pyproject.toml. When you use --no-build-isolation, this is ignored and you need to ensure yourself that the build dependencies are available.

So you would need to either drop --no-build-isolation or install Cython (and setuptools_scm) manually. Note that Cython is not a runtime dependency. pip installs the build dependencies into a temporary virtual environment which is separate from the environment into which the software itself is installed. If you have to keep using --no-build-isolation, I would recommend you do something similar, otherwise your target prefix may be a bit unclean as you would find a cython binary in there.

boegel commented 3 months ago

@marcelm Thanks a lot for the lightning-fast reply!

The problem was indeed that Cython wasn't available. I somehow knew that, and knew it would be necessary, but assumed just making sure that it's available wouldn't be sufficient.

It is though, so case closed.