mgalardini / pyseer

SEER, reimplemented in python 🐍🔮
http://pyseer.readthedocs.io
Apache License 2.0
104 stars 25 forks source link

Fatal glmnet error when trying to fit an elastic net model on training set #252

Closed vijuvi closed 7 months ago

vijuvi commented 8 months ago

Hello!

I'm trying to use pyseer's elastic net feature to assess the effects of different variant groups on the phenotype, i.e. to find biologically plausible combinations of variants that predict the phenotype of the test set particularly well. I have randomly divided the set into training (80%) and test sets (20%). When I fit elastic net to the whole set, it works fine. When I try to train a model with the training set, it throws an error.

In other words, this command pyseer --kmers output_unitigs.pyseer --uncompressed --phenotypes phenotypes/phenotypes.salt --wg enet --save-vars output/salt_unitigs --save-model salt_unitigs.ridge --cpu 12 --alpha 0.01 > salt_selected_unitigs.txt works fine, but when I'm trying to fit a model to the training set pyseer --kmers output_unitigs.pyseer --uncompressed --phenotypes train80_salt.pheno --wg enet --load-vars output/salt_unitigs --save-model salt_unitigs.ridge --cpu 12 --alpha 0.01 > salt_selected_unitigs.txt, glmnet throws an error like this:

Fitting elastic net to top 0 variants
Traceback (most recent call last):

    File "~/miniconda3/envs/pyseer/bin/pyseer", line 10, in <module>
        sys.exit(main())

    File "~/miniconda3/envs/pyseer/lib/python3.10/site-packages/pyseer/__main__.py", line 657, in main
        enet_betas = fit_enet(p, all_vars, cov, weights,

    File "~/miniconda3/envs/pyseer/lib/python3.10/site-packages/pyseer/enet.py", line 173, in fit_enet
        enet_fit = cvglmnet(x = variants, y = p.values.astype('float64'), family = regression_type,

    File "~/miniconda3/envs/pyseer/lib/python3.10/site-packages/glmnet_python/cvglmnet.py", line 244, in cvglmnet
        glmfit = glmnet(x = x, y = y, family = family, **options)    

    File "~/miniconda3/envs/pyseer/lib/python3.10/site-packages/glmnet_python/glmnet.py", line 449, in glmnet
        fit = elnet(x, is_sparse, irs, pcs, y, weights, offset, gtype, parm, 

    File "~/miniconda3/envs/pyseer/lib/python3.10/site-packages/glmnet_python/elnet.py", line 171, in elnet
        raise ValueError("Fatal glmnet error in library call : error code = ", jerr_r.value)

ValueError: ('Fatal glmnet error in library call : error code = ', 10000)

Why does it say Fitting elastic net to top 0 variants when with the whole set it says Fitting elastic net to top 324665 variants? An unitig compatibility issue?

I'm using a conda environment with python 3.10.11 and these package versions:

pyseer 1.3.11
numpy 1.26.0
scipy 1.11.3
pandas 2.1.1
scikit-learn 1.3.1
pysam 0.21.0
glmnet_py 0.1.0b2
matplotlib 3.8.0
DendroPy 4.6.1
pybedtools 0.9.1
bedtools 2.31.0
bedops 2.4.41
mgalardini commented 8 months ago

Can you try the following command: pyseer --kmers output_unitigs.pyseer --uncompressed --phenotypes train80_salt.pheno --wg enet --kmers output_unitigs.pyseer --save-model salt_unitigs.ridge --cpu 12 --alpha 0.01 > salt_selected_unitigs.txt, in which you are reading the kmers from scratch. I suspect that might cause the problem (which should throw a better error if that is the case). I would also perhaps generate k-mers/unitigs from the training set only, and not from the whole dataset, so that there is a stronger separation.

vijuvi commented 8 months ago

I tried leaving --load-vars out as instructed and now numpy throws a error. At least now it tries to fit elastic net on some variants.

Fitting elastic net to top 317265 variants
Traceback (most recent call last):
    File "~/miniconda3/envs/pyseer/bin/pyseer", line 10, in <module>
       sys.exit(main())
    File "~/miniconda3/envs/pyseer/lib/python3.10/site-packages/pyseer/__main__.py", line 657, in main
       enet_betas = fit_enet(p, all_vars, cov, weights,
    File "~/miniconda3/envs/pyseer/lib/python3.10/site-packages/pyseer/enet.py", line 173, in fit_enet
       enet_fit = cvglmnet(x = variants, y = p.values.astype('float64'), family = regression_type,
    File "~/miniconda3/envs/pyseer/lib/python3.10/site-packages/glmnet_python/cvglmnet.py", line 260, in cvglmnet
       ma = scipy.tile(scipy.arange(nfolds), [1, scipy.floor(nobs/nfolds)])
    File "~/miniconda3/envs/pyseer/lib/python3.10/site-packages/scipy/_lib/deprecation.py", line 20, in call
       return fun(*args, **kwargs)
    File "~/miniconda3/envs/pyseer/lib/python3.10/site-packages/numpy/lib/shape_base.py", line 1274, in tile
       return c.reshape(shape_out)
TypeError: 'numpy.float64' object cannot be interpreted as an integer
mgalardini commented 8 months ago

thanks, I will need to look into this a bit more closely. in the meantime can you also try to run the command with a phenotype file that contains only the training set samples?

vijuvi commented 8 months ago

Do you mean the unitig file? The phenotype file already contains only the training set.

mgalardini commented 8 months ago

Ah I see. I would try to provide input files that only contain the samples you are using to train the model; the enet code is a generally less robust to mismatches (still not 100% sure that is the issue though!)

vijuvi commented 7 months ago

I got this command pyseer --kmers train80_unitigs.pyseer --uncompressed --phenotypes train80_salt.pheno --wg enet --load-vars output/salt_unitigs --save-model salt_unitigs.ridge --cpu 12 --alpha 0.01 > salt_selected_unitigs.tx to work by making the unitig file train80_unitigs.pyseer out of the training set strains only. I even managed to predict the phenotypes of the test set quite accurately with the command enet_predict_pyseer --kmers all_unitigs.pyseer --uncompressed --lineage-clusters lineage_files/lineages_CC.txt --true-values test20_salt.pheno salt_unitigs.ridge.pkl test20_strains.txt > salt_ridge_predictions.txt where all_unitigs.pyseer contains the unitigs called from the combined strain set.

Is it now this unitig file after --kmers option that I can use to input different combinations of variants to test their effect on the phenotype?

mgalardini commented 7 months ago

Glad to hear that you managed to run your analysis. This tells us that we need to make that code path a bit more robust for mismatches in samples.

Regarding your last question, I am not sure I understand what exactly your plan is, but you can certainly fit the trained model with a made-up combinations of unitigs and see what the prediction will be based on them.

Will close the issue for now

vijuvi commented 7 months ago

The problem came back, when I changed to another computer with a newer Ubuntu version, so I dug around a little. I got pyseer elastic net to work without a problem by simply using an even older version (3.7.6) of python with conda like this

conda create -n <NAME> python=3.7.6

and installing pyseer in that environment

conda install pyseer.

Older versions of python seem to have a problem with HTSlib not being found, and the very newest python versions throw that numpy error above. Has something drastically changed in the newer numpy packages?

In this python3.7.6 environment the installed package versions are as follows:

pyseer 1.3.11
numpy 1.21.6
scipy 1.7.3
pandas 1.3.5
scikit-learn 1.0.2
pysam 0.16.0
glmnet_py - the one wrapped with pyseer
matplotlib 3.5.3
DendroPy 4.6.1
pybedtools 0.9.0
bedtools 2.31.
bedops 2.4.41
mgalardini commented 7 months ago

can you indicate the package versions (including python itself) for the environments that give you errors? I have a test environment with the following packages and it works on the test suite:

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
bedops                    2.4.41               h9f5acd7_0    bioconda
bedtools                  2.31.0               hf5e1c6e_2    bioconda
brotli                    1.0.9                h166bdaf_8    conda-forge
brotli-bin                1.0.9                h166bdaf_8    conda-forge
brotlipy                  0.7.0           py310h5764c6d_1005    conda-forge
bwa                       0.7.17              he4a0461_11    bioconda
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.19.0               hd590300_0    conda-forge
ca-certificates           2023.7.22            hbcca054_0    conda-forge
capnproto                 0.10.2               h6239696_0    conda-forge
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
cffi                      1.15.1          py310h255011f_3    conda-forge
charset-normalizer        3.1.0              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
contourpy                 1.0.7           py310hdf3cbec_0    conda-forge
cryptography              39.0.0          py310h65dfdc0_0    conda-forge
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
dendropy                  4.6.0              pyh7cba7a3_0    bioconda
fonttools                 4.39.4          py310h2372a71_0    conda-forge
freetype                  2.12.1               hca18f0e_1    conda-forge
glmnet_py                 1.0.2           py310h7d7f266_5    conda-forge
gsl                       2.7                  he838d99_0    conda-forge
htslib                    1.17                 h6bc39ce_1    bioconda
idna                      3.4                pyhd8ed1ab_0    conda-forge
joblib                    1.2.0              pyhd8ed1ab_0    conda-forge
jpeg                      9e                   h0b41bf4_3    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.4           py310hbf28c38_1    conda-forge
krb5                      1.20.1               hf9c8cef_0    conda-forge
lcms2                     2.14                 h6ed2654_0    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libblas                   3.9.0           16_linux64_openblas    conda-forge
libbrotlicommon           1.0.9                h166bdaf_8    conda-forge
libbrotlidec              1.0.9                h166bdaf_8    conda-forge
libbrotlienc              1.0.9                h166bdaf_8    conda-forge
libcblas                  3.9.0           16_linux64_openblas    conda-forge
libcurl                   7.87.0               h6312ad2_0    conda-forge
libdeflate                1.13                 h166bdaf_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 12.2.0              h65d4601_19    conda-forge
libgfortran-ng            12.2.0              h69a702a_19    conda-forge
libgfortran5              12.2.0              h337968e_19    conda-forge
libgomp                   12.2.0              h65d4601_19    conda-forge
liblapack                 3.9.0           16_linux64_openblas    conda-forge
libnghttp2                1.51.0               hdcd2b5c_0    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libopenblas               0.3.21          pthreads_h78a6416_3    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
libsqlite                 3.42.0               h2797004_0    conda-forge
libssh2                   1.10.0               haa6b8db_3    conda-forge
libstdcxx-ng              12.2.0              h46fd767_19    conda-forge
libtiff                   4.4.0                h0e0dad5_3    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp-base              1.3.0                h0b41bf4_0    conda-forge
libxcb                    1.13              h7f98852_1004    conda-forge
libzlib                   1.2.13               h166bdaf_4    conda-forge
mash                      2.3                  hd3113c8_6    bioconda
matplotlib-base           3.7.1           py310he60537e_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
ncurses                   6.3                  h27087fc_1    conda-forge
numpy                     1.24.3          py310ha4c1d20_0    conda-forge
openjpeg                  2.5.0                h7d73246_1    conda-forge
openssl                   1.1.1w               hd590300_0    conda-forge
packaging                 23.1               pyhd8ed1ab_0    conda-forge
pandas                    2.0.1           py310h7cbd5c2_1    conda-forge
patsy                     0.5.3              pyhd8ed1ab_0    conda-forge
perl                      5.32.1          2_h7f98852_perl5    conda-forge
pillow                    9.2.0           py310h454ad03_3    conda-forge
pip                       23.1.2             pyhd8ed1ab_0    conda-forge
platformdirs              3.5.1              pyhd8ed1ab_0    conda-forge
pooch                     1.7.0              pyha770c72_3    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
pybedtools                0.9.0           py310h2b6aa90_2    bioconda
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pyopenssl                 23.1.1             pyhd8ed1ab_0    conda-forge
pyparsing                 3.0.9              pyhd8ed1ab_0    conda-forge
pysam                     0.21.0          py310hff46b53_0    bioconda
pyseer                    1.3.11             pyh7cba7a3_0    bioconda
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.10.8          h257c98d_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-tzdata             2023.3             pyhd8ed1ab_0    conda-forge
python_abi                3.10                    3_cp310    conda-forge
pytz                      2023.3             pyhd8ed1ab_0    conda-forge
readline                  8.2                  h8228510_1    conda-forge
requests                  2.29.0             pyhd8ed1ab_0    conda-forge
samtools                  1.17                 hd87286a_1    bioconda
scikit-learn              1.2.2           py310h41b6a48_1    conda-forge
scipy                     1.10.1          py310ha4c1d20_3    conda-forge
setuptools                67.7.2             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
statsmodels               0.14.0          py310h278f3c1_1    conda-forge
threadpoolctl             3.1.0              pyh8a188c0_0    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
tqdm                      4.65.0             pyhd8ed1ab_1    conda-forge
typing-extensions         4.5.0                hd8ed1ab_0    conda-forge
typing_extensions         4.5.0              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
unicodedata2              15.0.0          py310h5764c6d_0    conda-forge
urllib3                   1.26.15            pyhd8ed1ab_0    conda-forge
wheel                     0.40.0             pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.9                h7f98852_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zlib                      1.2.13               h166bdaf_4    conda-forge
zstd                      1.5.2                h3eb15da_6    conda-forge

It is using python 3.10.8 as you can see

vijuvi commented 7 months ago

This work computer with Ubuntu 23.04 is the one from the original post. I installed the python3.7.6 environment on this computer and now it works fine. The original environment has python 3.10.11 and has the following packages installed (sorry, there are other bioinformatics packages not related to pyseer mixed up in there):

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
armadillo                 12.6.4               h37e7c65_1    conda-forge
arpack                    3.8.0           nompi_h0baa96a_101    conda-forge
atk-1.0                   2.38.0               hd4edc92_1    conda-forge
bedops                    2.4.41               h4ac6f70_1    bioconda
bedtools                  2.31.0               hf5e1c6e_3    bioconda
bifrost                   1.2.1                h43eeafb_0    bioconda
blast                     2.14.1          pl5321h6f7f691_0    bioconda
brotli                    1.1.0                hd590300_1    conda-forge
brotli-bin                1.1.0                hd590300_1    conda-forge
bwa                       0.7.17              he4a0461_11    bioconda
bzip2                     1.0.8                h7b6447c_0  
c-ares                    1.20.1               hd590300_0    conda-forge
ca-certificates           2023.08.22           h06a4308_0  
cairo                     1.16.0               hb05425b_5  
capnproto                 1.0.1                hfc55251_0    conda-forge
cd-hit                    4.8.1                h43eeafb_9    bioconda
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
contourpy                 1.1.1           py310hd41b1e2_1    conda-forge
curl                      7.87.0               h6312ad2_0    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
dendropy                  4.6.1              pyhdfd78af_0    bioconda
entrez-direct             16.2                 he881be0_1    bioconda
expat                     2.5.0                hcb278e6_1    conda-forge
fasttree                  2.1.11               h031d066_2    bioconda
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.1               h4c34cd2_2  
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.43.1          py310h2372a71_0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
fribidi                   1.0.10               h36c2ea0_0    conda-forge
fsm-lite                  1.0                  h4ac6f70_5    bioconda
gdk-pixbuf                2.42.10              h5eee18b_0  
gettext                   0.21.1               h27087fc_0    conda-forge
giflib                    5.2.1                h0b41bf4_3    conda-forge
glib                      2.78.0               hfc55251_0    conda-forge
glib-tools                2.78.0               hfc55251_0    conda-forge
glmnet-py                 0.1.0b2                  pypi_0    pypi
glmnet_py                 1.0.2           py310h7d7f266_5    conda-forge
graphite2                 1.3.13            h58526e2_1001    conda-forge
graphviz                  7.1.0                h2e5815a_0    conda-forge
gsl                       2.7                  he838d99_0    conda-forge
gtk2                      2.24.33              h90689f9_2    conda-forge
gts                       0.7.6                h977cf35_4    conda-forge
harfbuzz                  6.0.0                h8e241bc_0    conda-forge
hdf5                      1.10.3               hc401514_2    conda-forge
htslib                    1.17                 h6bc39ce_1    bioconda
icu                       70.1                 h27087fc_0    conda-forge
joblib                    1.3.2              pyhd8ed1ab_0    conda-forge
jpeg                      9e                   h166bdaf_2    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.5           py310hd41b1e2_1    conda-forge
krb5                      1.20.1               hf9c8cef_0    conda-forge
lcms2                     2.14                 h6ed2654_0    conda-forge
ld_impl_linux-64          2.38                 h1181459_1  
lerc                      4.0.0                h27087fc_0    conda-forge
libblas                   3.9.0           18_linux64_openblas    conda-forge
libbrotlicommon           1.1.0                hd590300_1    conda-forge
libbrotlidec              1.1.0                hd590300_1    conda-forge
libbrotlienc              1.1.0                hd590300_1    conda-forge
libcblas                  3.9.0           18_linux64_openblas    conda-forge
libcurl                   7.87.0               h6312ad2_0    conda-forge
libdb                     6.2.32               h9c3ff4c_0    conda-forge
libdeflate                1.13                 h166bdaf_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.4                h6a678d5_0  
libgcc                    7.2.0                h69d50b8_2    conda-forge
libgcc-ng                 13.2.0               h807b86a_2    conda-forge
libgd                     2.3.3                h18fbbfe_3    conda-forge
libgfortran               3.0.0                         1    conda-forge
libgfortran-ng            13.2.0               h69a702a_2    conda-forge
libgfortran5              13.2.0               ha4646dd_2    conda-forge
libglib                   2.78.0               hebfc3b9_0    conda-forge
libgomp                   13.2.0               h807b86a_2    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
libidn2                   2.3.4                h166bdaf_0    conda-forge
libjpeg-turbo             2.1.4                h166bdaf_0    conda-forge
liblapack                 3.9.0           18_linux64_openblas    conda-forge
libnghttp2                1.51.0               hdcd2b5c_0    conda-forge
libnsl                    2.0.0                hd590300_1    conda-forge
libopenblas               0.3.24          pthreads_h413a1c8_0    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
librsvg                   2.54.4               h7abd40a_0    conda-forge
libssh2                   1.10.0               haa6b8db_3    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_2    conda-forge
libtiff                   4.4.0                h0e0dad5_3    conda-forge
libtool                   2.4.7                h27087fc_0    conda-forge
libunistring              0.9.10               h7f98852_0    conda-forge
libuuid                   1.41.5               h5eee18b_0  
libwebp                   1.3.2                h11a3e52_0  
libwebp-base              1.3.2                hd590300_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libxml2                   2.10.3               hca2bb57_4    conda-forge
libxslt                   1.1.37               h873f0b0_0    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
mafft                     7.520                h031d066_2    bioconda
mash                      1.1                           0    bioconda
matplotlib-base           3.8.0           py310h62c0568_1    conda-forge
mcl                       14.137               h470a237_3    bioconda
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
ncbi-vdb                  3.0.8                hdbdd923_0    bioconda
ncurses                   6.4                  h6a678d5_0  
numpy                     1.26.0          py310hb13e2d6_0    conda-forge
openjpeg                  2.5.0                h7d73246_1    conda-forge
openssl                   1.1.1w               hd590300_0    conda-forge
ossuuid                   1.6.2             hf484d3e_1000    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pandas                    2.1.1           py310hcc13569_1    conda-forge
pango                     1.50.12              hd33c08f_1    conda-forge
parallel                  20230922             ha770c72_0    conda-forge
patsy                     0.5.3              pyhd8ed1ab_0    conda-forge
pcre                      8.45                 h9c3ff4c_0    conda-forge
pcre2                     10.40                hc3806b6_0    conda-forge
perl                      5.22.0.1                      0    conda-forge
perl-aceperl              1.92                          0    bioconda
perl-algorithm-diff       1.1903               pl5.22.0_1    bioconda
perl-algorithm-munkres    0.08                          0    bioconda
perl-apache-test          1.40                          0    bioconda
perl-app-cpanminus        1.7043               pl5.22.0_0    bioconda
perl-appconfig            1.71                          0    bioconda
perl-archive-tar          2.18                 pl5.22.0_2    bioconda
perl-array-compare        2.11                          0    bioconda
perl-array-utils          0.5                           0    bioconda
perl-b                    1.48                 pl5.22.0_0    bioconda
perl-b-hooks-endofscope   0.21                 pl5.22.0_0    bioconda
perl-bio-asn1-entrezgene  1.72                          1    bioconda
perl-bio-featureio        1.6.905                       0    bioconda
perl-bio-phylo            0.58                          0    bioconda
perl-bio-samtools         1.43                          0    bioconda
perl-bioperl              1.6.924              pl5.22.0_7    bioconda
perl-bioperl-core         1.6.924              pl5.22.0_2    bioconda
perl-bioperl-run          1.006900                      2    bioconda
perl-capture-tiny         0.36                          0    bioconda
perl-carp                 1.38                 pl5.22.0_0    bioconda
perl-cgi                  4.22                          3    bioconda
perl-class-data-inheritable 0.08                          0    bioconda
perl-class-inspector      1.28                          0    bioconda
perl-class-method-modifiers 2.11                          1    bioconda
perl-clone                0.38                          0    bioconda
perl-common-sense         3.74                          0    bioconda
perl-compress-raw-bzip2   2.069                         1    bioconda
perl-compress-raw-zlib    2.069                         3    bioconda
perl-convert-binary-c     0.78                          0    bioconda
perl-convert-binhex       1.125                         0    bioconda
perl-cpan-meta-check      0.012                         1    bioconda
perl-crypt-rc4            2.02                          0    bioconda
perl-data-dumper          2.161                pl5.22.0_0    bioconda
perl-data-optlist         0.110                         0    bioconda
perl-data-stag            0.14                          0    bioconda
perl-datetime             1.42                 pl5.22.0_0    bioconda
perl-datetime-locale      1.12                 pl5.22.0_0    bioconda
perl-datetime-timezone    2.09                 pl5.22.0_0    bioconda
perl-db-file              1.835                         5    bioconda
perl-dbd-sqlite           1.50                          1    bioconda
perl-dbi                  1.641                pl5.22.0_0    bioconda
perl-devel-globaldestruction 0.13                          1    bioconda
perl-devel-stacktrace     2.03                 pl5.22.0_0    bioconda
perl-digest-hmac          1.03                 pl5.22.0_1    bioconda
perl-digest-md5-file      0.08                 pl5.22.0_0    bioconda
perl-digest-perl-md5      1.9                           0    bioconda
perl-dist-checkconflicts  0.11                          0    bioconda
perl-email-date-format    1.005                         0    bioconda
perl-encode-locale        1.05                 pl5.22.0_4    bioconda
perl-env-path             0.19                          0    bioconda
perl-error                0.17024                       0    bioconda
perl-eval-closure         0.14                 pl5.22.0_0    bioconda
perl-exception-class      1.40                          0    bioconda
perl-exporter             5.72                 pl5.22.0_0    bioconda
perl-exporter-tiny        0.042                         1    bioconda
perl-extutils-makemaker   7.24                 pl5.22.0_1    bioconda
perl-file-find-rule       0.34                          3    bioconda
perl-file-grep            0.02                          0    bioconda
perl-file-listing         6.04                          0    bioconda
perl-file-slurp-tiny      0.004                         0    bioconda
perl-file-slurper         0.008                         0    bioconda
perl-file-sort            1.01                 pl5.22.0_1    bioconda
perl-file-which           1.20                          0    bioconda
perl-font-afm             1.20                          0    bioconda
perl-gd                   2.56            pl522h470a237_9    bioconda
perl-getopt-long          2.50                 pl5.22.0_0    bioconda
perl-graph                0.9704                        0    bioconda
perl-graph-readwrite      2.09                          0    bioconda
perl-graphviz             2.20                          1    bioconda
perl-html-element-extended 1.18                          0    bioconda
perl-html-entities-numbered 0.04                          0    bioconda
perl-html-formatter       2.14                          0    bioconda
perl-html-parser          3.72                 pl5.22.0_1    bioconda
perl-html-tableextract    2.13                          0    bioconda
perl-html-tagset          3.20                 pl5.22.0_1    bioconda
perl-html-tidy            1.56                          1    bioconda
perl-html-tree            5.03                          0    bioconda
perl-html-treebuilder-xpath 0.14                          0    bioconda
perl-http-cookies         6.01                          0    bioconda
perl-http-daemon          6.01                          0    bioconda
perl-http-date            6.02                 pl5.22.0_1    bioconda
perl-http-message         6.11                          0    bioconda
perl-http-negotiate       6.01                          0    bioconda
perl-image-info           1.38                          0    bioconda
perl-image-size           3.300                         0    bioconda
perl-importer             0.024                         0    bioconda
perl-io-compress          2.069                pl5.22.0_2    bioconda
perl-io-html              1.001                pl5.22.0_1    bioconda
perl-io-sessiondata       1.03                          0    bioconda
perl-io-socket-ssl        2.024                         0    bioconda
perl-io-string            1.08                 pl5.22.0_1    bioconda
perl-io-stringy           2.111                         0    bioconda
perl-io-tty               1.12                          0    bioconda
perl-io-zlib              1.10                          1    bioconda
perl-ipc-run              0.94                          0    bioconda
perl-jcode                2.07                          0    bioconda
perl-json                 2.90                          1    bioconda
perl-json-xs              2.34                          0    bioconda
perl-libwww-perl          6.15                          0    bioconda
perl-libxml-perl          0.08                          0    bioconda
perl-list-moreutils       0.428                pl5.22.0_0    bioconda
perl-list-util            1.38                 pl5.22.0_0    bioconda
perl-log-log4perl         1.47                          1    bioconda
perl-lwp-mediatypes       6.02                 pl5.22.0_1    bioconda
perl-lwp-protocol-https   6.06                          2    bioconda
perl-lwp-simple           6.15                          3    bioconda
perl-mailtools            2.14                          0    bioconda
perl-math-cdf             0.1                           3    bioconda
perl-math-derivative      0.04                          0    bioconda
perl-math-random          0.72                          0    bioconda
perl-math-spline          0.02                          0    bioconda
perl-mime-lite            3.030                         0    bioconda
perl-mime-tools           5.507                         0    bioconda
perl-mime-types           2.12                          0    bioconda
perl-mldbm                2.05                          0    bioconda
perl-module-build         0.4224               pl5.22.0_0    bioconda
perl-module-implementation 0.09                          0    bioconda
perl-module-runtime       0.014                         1    bioconda
perl-moo                  2.001000                      1    bioconda
perl-moose                2.2009               pl5.22.0_0    bioconda
perl-namespace-autoclean  0.28                 pl5.22.0_0    bioconda
perl-namespace-clean      0.27                 pl5.22.0_0    bioconda
perl-net-http             6.09                          0    bioconda
perl-net-ssleay           1.74                          0    bioconda
perl-ntlm                 1.09                          1    bioconda
perl-number-compare       0.03                 pl5.22.0_1    bioconda
perl-ole-storage_lite     0.19                          0    bioconda
perl-package-stash        0.37                 pl5.22.0_0    bioconda
perl-params-util          1.07                          1    bioconda
perl-params-validationcompiler 0.23                 pl5.22.0_0    bioconda
perl-parse-recdescent     1.967013                      0    bioconda
perl-parse-yapp           1.05                          0    bioconda
perl-pathtools            3.73                 h470a237_2    bioconda
perl-pdf-api2             2.025                         2    bioconda
perl-perlio-utf8_strict   0.006                         0    bioconda
perl-postscript           0.06                          0    bioconda
perl-role-tiny            2.000001                      1    bioconda
perl-scalar-list-utils    1.45                          2    bioconda
perl-set-scalar           1.29                          0    bioconda
perl-soap-lite            1.19                          0    bioconda
perl-sort-naturally       1.03                          0    bioconda
perl-specio-exporter      0.36                 pl5.22.0_0    bioconda
perl-spreadsheet-parseexcel 0.65                          0    bioconda
perl-spreadsheet-writeexcel 2.40                          0    bioconda
perl-statistics-descriptive 3.0612                        0    bioconda
perl-sub-exporter         0.987                         0    bioconda
perl-sub-exporter-progressive 0.001011                      1    bioconda
perl-sub-identify         0.12                 pl5.22.0_0    bioconda
perl-sub-install          0.928                         0    bioconda
perl-sub-uplevel          0.25                          2    bioconda
perl-svg                  2.64                          0    bioconda
perl-svg-graph            0.02                          0    bioconda
perl-task-weaken          1.04                          0    bioconda
perl-template-toolkit     2.26                          0    bioconda
perl-test-builder-tester  1.23_002                      0    bioconda
perl-test-deep            1.120                         1    bioconda
perl-test-differences     0.64                          0    bioconda
perl-test-exception       0.43                          0    bioconda
perl-test-fatal           0.014                         1    bioconda
perl-test-files           0.14                          0    bioconda
perl-test-leaktrace       0.16                 pl5.22.0_0    bioconda
perl-test-more            1.001002             pl5.22.0_0    bioconda
perl-test-most            0.34                          1    bioconda
perl-test-output          1.03                          0    bioconda
perl-test-pod             1.51                          0    bioconda
perl-test-simple          1.302075             pl5.22.0_0    bioconda
perl-test-warn            0.30                          0    bioconda
perl-test-warnings        0.021                         1    bioconda
perl-test2                1.302075             pl5.22.0_0    bioconda
perl-test2-suite          0.000061                      0    bioconda
perl-text-csv             1.33                          0    bioconda
perl-text-diff            1.44                          0    bioconda
perl-text-glob            0.11                 pl5.22.0_0    bioconda
perl-threaded             5.32.1               hdfd78af_1    bioconda
perl-tie-ixhash           1.23                 pl5.22.0_1    bioconda
perl-timedate             2.30                          0    bioconda
perl-tree-dag_node        1.29                          0    bioconda
perl-try-tiny             0.24                 pl5.22.0_2    bioconda
perl-type-tiny            1.000005                      0    bioconda
perl-unicode-map          0.112                         0    bioconda
perl-uri                  1.71                 pl5.22.0_1    bioconda
perl-variable-magic       0.61                 pl5.22.0_0    bioconda
perl-warnings-register    1.03                 pl5.22.0_0    bioconda
perl-www-robotrules       6.02                          0    bioconda
perl-xml-dom              1.45                          0    bioconda
perl-xml-dom-xpath        0.14                          0    bioconda
perl-xml-filter-buffertext 1.01                          0    bioconda
perl-xml-libxml           2.0124                        0    bioconda
perl-xml-libxslt          1.94                          0    bioconda
perl-xml-namespacesupport 1.11                          0    bioconda
perl-xml-parser           2.44                          4    bioconda
perl-xml-regexp           0.04                          0    bioconda
perl-xml-sax              0.99                          0    bioconda
perl-xml-sax-base         1.08                          0    bioconda
perl-xml-sax-expat        0.51                          0    bioconda
perl-xml-sax-writer       0.56                          0    bioconda
perl-xml-simple           2.22                          0    bioconda
perl-xml-twig             3.49                          0    bioconda
perl-xml-writer           0.625                         0    bioconda
perl-xml-xpath            1.33                          0    bioconda
perl-xml-xpathengine      0.14                          0    bioconda
perl-xsloader             0.22                 pl5.22.0_0    bioconda
perl-yaml                 1.24                    pl522_1    bioconda
pillow                    10.0.1          py310ha6cbd5a_0  
pip                       23.2.1          py310h06a4308_0  
pixman                    0.42.2               h59595ed_0    conda-forge
prank                     v.170427             h4ac6f70_7    bioconda
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
pybedtools                0.9.1           py310h2b6aa90_0    bioconda
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pysam                     0.21.0          py310hff46b53_0    bioconda
pyseer                    1.3.11             pyh7cba7a3_0    bioconda
python                    3.10.11              h7a1cb2a_2  
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-tzdata             2023.3             pyhd8ed1ab_0    conda-forge
python_abi                3.10                    2_cp310    conda-forge
pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
readline                  8.2                  h5eee18b_0  
roary                     3.12.0               pl5.22.0_0    bioconda
samtools                  1.18                 hd87286a_0    bioconda
scikit-learn              1.3.1           py310h1fdf081_1    conda-forge
scipy                     1.11.3          py310hb13e2d6_1    conda-forge
sdsl-lite                 2.1.1             h00ab1b0_1002    conda-forge
seer                      1.1.4                hb75bb0b_0    bioconda
setuptools                68.0.0          py310h06a4308_0  
six                       1.16.0             pyh6c4a22f_0    conda-forge
sqlite                    3.41.2               h5eee18b_0  
statsmodels               0.14.0          py310h1f7b6fc_2    conda-forge
superlu                   5.2.2                h00795ac_0    conda-forge
threadpoolctl             3.2.0              pyha21a80b_0    conda-forge
tidyp                     1.04                 hec16e2b_4    bioconda
tk                        8.6.12               h1ccaba5_0  
tqdm                      4.66.1             pyhd8ed1ab_0    conda-forge
tzdata                    2023c                h04d1e81_0  
unicodedata2              15.1.0          py310h2372a71_0    conda-forge
unitig-caller             1.3.0           py310ha320341_2    bioconda
unitig-counter            1.1.0                h56fc30b_0    bioconda
wget                      1.20.3               ha56f1ee_1    conda-forge
wheel                     0.41.2          py310h06a4308_0  
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xz                        5.2.10               h5eee18b_1  
zlib                      1.2.13               hd590300_5    conda-forge
zstd                      1.5.5                hfc55251_0    conda-forge

I got pyseer to work temporarily on my home computer with an older Ubuntu. I'll update this post related to that when I get home.

mgalardini commented 7 months ago

Ok, if possible please see if you can get a clean environment which reproduces the error

vijuvi commented 7 months ago

Okay, I don't know what's the problem anymore , because It works fine in this kind of clean environment:

# packages in environment at ~/miniconda3/envs/pyseer_clean_3.10.11:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
bedops                    2.4.41               h4ac6f70_1    bioconda
bedtools                  2.31.1               hf5e1c6e_0    bioconda
brotli                    1.1.0                hd590300_1    conda-forge
brotli-bin                1.1.0                hd590300_1    conda-forge
bwa                       0.7.17              he4a0461_11    bioconda
bzip2                     1.0.8                hd590300_5    conda-forge
c-ares                    1.21.0               hd590300_0    conda-forge
ca-certificates           2023.7.22            hbcca054_0    conda-forge
capnproto                 0.9.1                ha19adfc_4    conda-forge
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
contourpy                 1.2.0           py310hd41b1e2_0    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
dendropy                  4.6.1              pyhdfd78af_0    bioconda
fonttools                 4.44.0          py310h2372a71_0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
glmnet_py                 1.0.2           py310h7d7f266_5    conda-forge
gsl                       2.7                  he838d99_0    conda-forge
htslib                    1.18                 h81da01d_0    bioconda
joblib                    1.3.2              pyhd8ed1ab_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.5           py310hd41b1e2_1    conda-forge
krb5                      1.21.2               h659d440_0    conda-forge
lcms2                     2.15                 h7f713cb_2    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libblas                   3.9.0           19_linux64_openblas    conda-forge
libbrotlicommon           1.1.0                hd590300_1    conda-forge
libbrotlidec              1.1.0                hd590300_1    conda-forge
libbrotlienc              1.1.0                hd590300_1    conda-forge
libcblas                  3.9.0           19_linux64_openblas    conda-forge
libcurl                   8.4.0                hca28451_0    conda-forge
libdeflate                1.18                 h0b41bf4_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 13.2.0               h807b86a_2    conda-forge
libgfortran-ng            13.2.0               h69a702a_2    conda-forge
libgfortran5              13.2.0               ha4646dd_2    conda-forge
libgomp                   13.2.0               h807b86a_2    conda-forge
libjpeg-turbo             2.1.5.1              hd590300_1    conda-forge
liblapack                 3.9.0           19_linux64_openblas    conda-forge
libnghttp2                1.55.1               h47da74e_0    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libopenblas               0.3.24          pthreads_h413a1c8_0    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
libsqlite                 3.44.0               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_2    conda-forge
libtiff                   4.6.0                h8b53f26_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp-base              1.3.2                hd590300_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
mash                      2.3                  ha9a2dd8_3    bioconda
matplotlib-base           3.8.1           py310h62c0568_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
ncurses                   6.4                  h59595ed_2    conda-forge
numpy                     1.26.0          py310hb13e2d6_0    conda-forge
openjpeg                  2.5.0                h488ebb8_3    conda-forge
openssl                   3.1.4                hd590300_0    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pandas                    2.1.2           py310hcc13569_0    conda-forge
patsy                     0.5.3              pyhd8ed1ab_0    conda-forge
perl                      5.32.1          4_hd590300_perl5    conda-forge
pillow                    10.0.1          py310h29da1c1_1    conda-forge
pip                       23.3.1             pyhd8ed1ab_0    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
pybedtools                0.9.1           py310h2b6aa90_0    bioconda
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pysam                     0.22.0          py310h41dec4a_0    bioconda
pyseer                    1.3.11             pyh7cba7a3_0    bioconda
python                    3.10.11         he550d4f_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-tzdata             2023.3             pyhd8ed1ab_0    conda-forge
python_abi                3.10                    4_cp310    conda-forge
pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
readline                  8.2                  h8228510_1    conda-forge
samtools                  1.18                 h50ea8bc_1    bioconda
scikit-learn              1.3.2           py310h1fdf081_1    conda-forge
scipy                     1.11.3          py310hb13e2d6_1    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
statsmodels               0.14.0          py310h1f7b6fc_2    conda-forge
threadpoolctl             3.2.0              pyha21a80b_0    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tqdm                      4.66.1             pyhd8ed1ab_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
unicodedata2              15.1.0          py310h2372a71_0    conda-forge
wheel                     0.41.3             pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zlib                      1.2.13               hd590300_5    conda-forge
zstd                      1.5.5                hfc55251_0    conda-forge
mgalardini commented 7 months ago

I can see that in the environment that caused the error you have two versions of glmnet installed:

glmnet-py                 0.1.0b2                  pypi_0    pypi
glmnet_py                 1.0.2           py310h7d7f266_5    conda-forge

That's probably it

vijuvi commented 7 months ago

Well damn, thank you!