search

maxplanck-ie / snakepipes

Customizable workflows based on snakemake and python for the analysis of NGS data
http://snakepipes.readthedocs.io
MIT License
374 stars 85 forks source link

snakePipes installation fails with python 3.12.0 #939

Closed katsikora closed 5 months ago

katsikora commented 9 months ago

Failed to build datrie ERROR: Could not build wheels for datrie, which is required to install pyproject.toml-based projects

works with python 3.11.3

katsikora commented 5 months ago

This is no longer an issue.

sunta3iouxos commented 3 weeks ago

Hi @katsikora , This is still an issue: From the devel branch: git clone -b develop https://github.com/maxplanck-ie/snakepipes.git snakepipes_develop

$mamba create -n snakePipes_devel pip
$mamba activate snakePipes_devel
$pip install --upgrade .
Processing /home/tgeorgom/snakepipes_develop_folder
  Preparing metadata (setup.py) ... done
Collecting snakemake<7.19.0,>=6.2.1 (from snakePipes==2.9.0)
  Using cached snakemake-7.18.2-py3-none-any.whl.metadata (2.2 kB)
Collecting psutil (from snakePipes==2.9.0)
  Downloading psutil-6.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (21 kB)
Collecting pandas (from snakePipes==2.9.0)
  Downloading pandas-2.2.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)
Collecting thefuzz (from snakePipes==2.9.0)
  Using cached thefuzz-0.22.1-py3-none-any.whl.metadata (3.9 kB)
Collecting pyyaml>=5.1 (from snakePipes==2.9.0)
  Using cached PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
Collecting tabulate<0.9 (from snakePipes==2.9.0)
  Using cached tabulate-0.8.10-py3-none-any.whl.metadata (25 kB)
Collecting wrapt (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached wrapt-1.16.0-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting requests (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Downloading requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Collecting throttler (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached throttler-1.2.2-py3-none-any.whl.metadata (7.4 kB)
Collecting configargparse (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached ConfigArgParse-1.7-py3-none-any.whl.metadata (23 kB)
Collecting appdirs (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached appdirs-1.4.4-py2.py3-none-any.whl.metadata (9.0 kB)
Collecting datrie (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached datrie-0.8.2.tar.gz (63 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting jsonschema (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Downloading jsonschema-4.23.0-py3-none-any.whl.metadata (7.9 kB)
Collecting docutils (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Downloading docutils-0.21.2-py3-none-any.whl.metadata (2.8 kB)
Collecting gitpython (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Downloading GitPython-3.1.43-py3-none-any.whl.metadata (13 kB)
Collecting nbformat (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Downloading nbformat-5.10.4-py3-none-any.whl.metadata (3.6 kB)
Collecting toposort (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached toposort-1.10-py3-none-any.whl.metadata (4.1 kB)
Collecting connection-pool>=0.0.3 (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached connection_pool-0.0.3-py3-none-any.whl
Collecting pulp>=2.0 (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached PuLP-2.8.0-py3-none-any.whl.metadata (5.4 kB)
Collecting smart-open>=3.0 (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Downloading smart_open-7.0.4-py3-none-any.whl.metadata (23 kB)
Collecting stopit (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached stopit-1.1.2-py3-none-any.whl
Collecting yte<2.0,>=1.0 (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached yte-1.5.4-py3-none-any.whl.metadata (3.4 kB)
Collecting jinja2<4.0,>=3.0 (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Downloading jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB)
Collecting reretry (from snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached reretry-0.11.8-py2.py3-none-any.whl.metadata (5.5 kB)
Collecting numpy>=1.26.0 (from pandas->snakePipes==2.9.0)
  Downloading numpy-2.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.9/60.9 kB 579.1 kB/s eta 0:00:00
Collecting python-dateutil>=2.8.2 (from pandas->snakePipes==2.9.0)
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Collecting pytz>=2020.1 (from pandas->snakePipes==2.9.0)
  Using cached pytz-2024.1-py2.py3-none-any.whl.metadata (22 kB)
Collecting tzdata>=2022.7 (from pandas->snakePipes==2.9.0)
  Using cached tzdata-2024.1-py2.py3-none-any.whl.metadata (1.4 kB)
Collecting rapidfuzz<4.0.0,>=3.0.0 (from thefuzz->snakePipes==2.9.0)
  Downloading rapidfuzz-3.9.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (12 kB)
Collecting MarkupSafe>=2.0 (from jinja2<4.0,>=3.0->snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached MarkupSafe-2.1.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
Collecting six>=1.5 (from python-dateutil>=2.8.2->pandas->snakePipes==2.9.0)
  Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Collecting dpath<3.0,>=2.1 (from yte<2.0,>=1.0->snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Downloading dpath-2.2.0-py3-none-any.whl.metadata (15 kB)
Collecting plac<2.0.0,>=1.3.4 (from yte<2.0,>=1.0->snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached plac-1.4.3-py2.py3-none-any.whl.metadata (5.9 kB)
Collecting gitdb<5,>=4.0.1 (from gitpython->snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached gitdb-4.0.11-py3-none-any.whl.metadata (1.2 kB)
Collecting attrs>=22.2.0 (from jsonschema->snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached attrs-23.2.0-py3-none-any.whl.metadata (9.5 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema->snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached jsonschema_specifications-2023.12.1-py3-none-any.whl.metadata (3.0 kB)
Collecting referencing>=0.28.4 (from jsonschema->snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Downloading referencing-0.35.1-py3-none-any.whl.metadata (2.8 kB)
Collecting rpds-py>=0.7.1 (from jsonschema->snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Downloading rpds_py-0.19.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.1 kB)
Collecting fastjsonschema>=2.15 (from nbformat->snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Downloading fastjsonschema-2.20.0-py3-none-any.whl.metadata (2.1 kB)
Collecting jupyter-core!=5.0.*,>=4.12 (from nbformat->snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached jupyter_core-5.7.2-py3-none-any.whl.metadata (3.4 kB)
Collecting traitlets>=5.1 (from nbformat->snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Downloading traitlets-5.14.3-py3-none-any.whl.metadata (10 kB)
Collecting charset-normalizer<4,>=2 (from requests->snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached charset_normalizer-3.3.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests->snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Downloading idna-3.7-py3-none-any.whl.metadata (9.9 kB)
Collecting urllib3<3,>=1.21.1 (from requests->snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Downloading urllib3-2.2.2-py3-none-any.whl.metadata (6.4 kB)
Collecting certifi>=2017.4.17 (from requests->snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Downloading certifi-2024.7.4-py3-none-any.whl.metadata (2.2 kB)
Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->gitpython->snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Using cached smmap-5.0.1-py3-none-any.whl.metadata (4.3 kB)
Collecting platformdirs>=2.5 (from jupyter-core!=5.0.*,>=4.12->nbformat->snakemake<7.19.0,>=6.2.1->snakePipes==2.9.0)
  Downloading platformdirs-4.2.2-py3-none-any.whl.metadata (11 kB)
Using cached PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (724 kB)
Using cached snakemake-7.18.2-py3-none-any.whl (351 kB)
Using cached tabulate-0.8.10-py3-none-any.whl (29 kB)
Downloading pandas-2.2.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.7/12.7 MB 34.9 MB/s eta 0:00:00
Downloading psutil-6.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (290 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 290.5/290.5 kB 5.2 MB/s eta 0:00:00
Using cached thefuzz-0.22.1-py3-none-any.whl (8.2 kB)
Downloading jinja2-3.1.4-py3-none-any.whl (133 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.3/133.3 kB 1.8 MB/s eta 0:00:00
Downloading numpy-2.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (19.0 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.0/19.0 MB 26.2 MB/s eta 0:00:00
Using cached PuLP-2.8.0-py3-none-any.whl (17.7 MB)
Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Using cached pytz-2024.1-py2.py3-none-any.whl (505 kB)
Downloading rapidfuzz-3.9.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 18.4 MB/s eta 0:00:00
Downloading smart_open-7.0.4-py3-none-any.whl (61 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 kB 562.7 kB/s eta 0:00:00
Using cached tzdata-2024.1-py2.py3-none-any.whl (345 kB)
Using cached yte-1.5.4-py3-none-any.whl (7.7 kB)
Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Using cached ConfigArgParse-1.7-py3-none-any.whl (25 kB)
Downloading docutils-0.21.2-py3-none-any.whl (587 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 587.4/587.4 kB 7.6 MB/s eta 0:00:00
Downloading GitPython-3.1.43-py3-none-any.whl (207 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 207.3/207.3 kB 3.3 MB/s eta 0:00:00
Downloading jsonschema-4.23.0-py3-none-any.whl (88 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 88.5/88.5 kB 1.4 MB/s eta 0:00:00
Downloading nbformat-5.10.4-py3-none-any.whl (78 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.5/78.5 kB 1.3 MB/s eta 0:00:00
Downloading requests-2.32.3-py3-none-any.whl (64 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.9/64.9 kB 980.9 kB/s eta 0:00:00
Using cached reretry-0.11.8-py2.py3-none-any.whl (5.6 kB)
Using cached throttler-1.2.2-py3-none-any.whl (7.6 kB)
Using cached toposort-1.10-py3-none-any.whl (8.5 kB)
Using cached wrapt-1.16.0-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (87 kB)
Using cached attrs-23.2.0-py3-none-any.whl (60 kB)
Downloading certifi-2024.7.4-py3-none-any.whl (162 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 163.0/163.0 kB 2.5 MB/s eta 0:00:00
Using cached charset_normalizer-3.3.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (141 kB)
Downloading dpath-2.2.0-py3-none-any.whl (17 kB)
Downloading fastjsonschema-2.20.0-py3-none-any.whl (23 kB)
Using cached gitdb-4.0.11-py3-none-any.whl (62 kB)
Downloading idna-3.7-py3-none-any.whl (66 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.8/66.8 kB 1.2 MB/s eta 0:00:00
Using cached jsonschema_specifications-2023.12.1-py3-none-any.whl (18 kB)
Using cached jupyter_core-5.7.2-py3-none-any.whl (28 kB)
Using cached MarkupSafe-2.1.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (28 kB)
Using cached plac-1.4.3-py2.py3-none-any.whl (22 kB)
Downloading referencing-0.35.1-py3-none-any.whl (26 kB)
Downloading rpds_py-0.19.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (359 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 359.1/359.1 kB 4.4 MB/s eta 0:00:00
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Downloading traitlets-5.14.3-py3-none-any.whl (85 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 85.4/85.4 kB 1.7 MB/s eta 0:00:00
Downloading urllib3-2.2.2-py3-none-any.whl (121 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.4/121.4 kB 2.0 MB/s eta 0:00:00
Downloading platformdirs-4.2.2-py3-none-any.whl (18 kB)
Using cached smmap-5.0.1-py3-none-any.whl (24 kB)
Building wheels for collected packages: snakePipes, datrie
  Building wheel for snakePipes (setup.py) ... done
  Created wheel for snakePipes: filename=snakePipes-2.9.0-py3-none-any.whl size=19154147 sha256=e28a33e9d3c5af7c49cf8d4184137cb124141b2e1671f2ca149a7ca213d1cf99
  Stored in directory: /scratch/tgeorgom_temp/pip-ephem-wheel-cache-kxkle2og/wheels/c9/12/6d/7395d055990319edf828d5f416e3776630690f3117e0fdd498
  Building wheel for datrie (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for datrie (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [74 lines of output]
      running bdist_wheel
      running build
      running build_clib
      building 'datrie' library
      creating build
      creating build/temp.linux-x86_64-cpython-312
      creating build/temp.linux-x86_64-cpython-312/libdatrie
      creating build/temp.linux-x86_64-cpython-312/libdatrie/datrie
      .
      .
      .
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for datrie
Successfully built snakePipes
Failed to build datrie
ERROR: Could not build wheels for datrie, which is required to install pyproject.toml-based projects

The default will set the python 3.12.4, but I get the same errors with python 3.11 (which is the latest, 3.11.9 I think) and python 3.11.3 that you have mentioned. Note that I never had this issue beofre Apr 12, as of #1007