Cutadapt 3.7 installation issues with python 3.11

fjossandon commented 1 year ago

Hello @marcelm! A colleague tried to pip install Cutadapt 3.7 in a new Linux image using Python 3.11 but it failed:

# pip install cutadapt==3.7
Collecting cutadapt==3.7
  Downloading cutadapt-3.7.tar.gz (221 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 221.8/221.8 kB 600.1 kB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: dnaio>=0.7.1 in /usr/local/lib/python3.11/site-packages (from cutadapt==3.7) (0.10.0)
Requirement already satisfied: xopen>=1.1 in /usr/local/lib/python3.11/site-packages (from cutadapt==3.7) (1.7.0)
Requirement already satisfied: isal>=1.0.0 in /usr/local/lib/python3.11/site-packages (from xopen>=1.1->cutadapt==3.7) (1.1.0)
Building wheels for collected packages: cutadapt
  Building wheel for cutadapt (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for cutadapt (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [33 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-311
      creating build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/modifiers.py -> build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/log.py -> build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/statistics.py -> build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/report.py -> build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/parser.py -> build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/utils.py -> build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/steps.py -> build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/align.py -> build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/filters.py -> build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/tokenizer.py -> build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/__main__.py -> build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/json.py -> build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/_version.py -> build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/__init__.py -> build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/adapters.py -> build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/pipeline.py -> build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/qualtrim.pyi -> build/lib.linux-x86_64-cpython-311/cutadapt
      copying src/cutadapt/_align.pyi -> build/lib.linux-x86_64-cpython-311/cutadapt
      running build_ext
      cythoning src/cutadapt/_align.pyx to src/cutadapt/_align.c
      cythoning src/cutadapt/qualtrim.pyx to src/cutadapt/qualtrim.c
      building 'cutadapt._align' extension
      creating build/temp.linux-x86_64-cpython-311
      creating build/temp.linux-x86_64-cpython-311/src
      creating build/temp.linux-x86_64-cpython-311/src/cutadapt
      gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/include/python3.11 -c src/cutadapt/_align.c -o build/temp.linux-x86_64-cpython-311/src/cutadapt/_align.o
      error: command 'gcc' failed: No such file or directory
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for cutadapt
Failed to build cutadapt
ERROR: Could not build wheels for cutadapt, which is required to install pyproject.toml-based projects

No such file or directory?? Installation of version 4.4 worked though.

I know that Cutadapt v3.7 is somewhat old now, but I have not tested v4.4 enough to confidently migrate yet, so I was wondering if there is some fix that could make it work, or if there has been some kind of breaking change that will make it impossible to work on python 3.11.

Best regards,

rhpvorderman commented 1 year ago

error: command 'gcc' failed: No such file or directory

3.11 did not exist at the time of release of cutadapt 3.7. As such there are no wheels available. If you are installing in an image probably there are no build tools available to build the wheel.

There are several options:

Forego the latest python version. There are wheels for python 3.10.
Do use python:3.11 as base image. That should have the build tools. If not: apt install build-essential should solve your problem (on debian/ubuntu-based images). EDIT: (Python:3.11 image should have the build tools)
Simply use 4.4 and enjoy a significant speedup for adapter trimming workloads.

marcelm commented 1 year ago

Regarding updating to 4.4: The only slightly backwards incompatible change since 3.7 is a change in the alignment algorithm that affects very few reads, see the 4.0 changelog.

fjossandon commented 1 year ago

Hello! Thanks for your replies. @rhpvorderman, thanks for the second option, I tried and that worked! @marcelm, my problem is that there is a considerable loss of reads in my workflow when trying 4.4 compared to 3.7, I'm not sure if it's related to that or is something different, but that is a different matter, so I close this one and open a new ticket very soon. :)

marcelm / cutadapt

Cutadapt 3.7 installation issues with python 3.11 #714