Closed vijuvi closed 7 months ago
Can you try the following command: pyseer --kmers output_unitigs.pyseer --uncompressed --phenotypes train80_salt.pheno --wg enet --kmers output_unitigs.pyseer --save-model salt_unitigs.ridge --cpu 12 --alpha 0.01 > salt_selected_unitigs.txt
, in which you are reading the kmers from scratch. I suspect that might cause the problem (which should throw a better error if that is the case). I would also perhaps generate k-mers/unitigs from the training set only, and not from the whole dataset, so that there is a stronger separation.
I tried leaving --load-vars out as instructed and now numpy throws a error. At least now it tries to fit elastic net on some variants.
Fitting elastic net to top 317265 variants
Traceback (most recent call last):
File "~/miniconda3/envs/pyseer/bin/pyseer", line 10, in <module>
sys.exit(main())
File "~/miniconda3/envs/pyseer/lib/python3.10/site-packages/pyseer/__main__.py", line 657, in main
enet_betas = fit_enet(p, all_vars, cov, weights,
File "~/miniconda3/envs/pyseer/lib/python3.10/site-packages/pyseer/enet.py", line 173, in fit_enet
enet_fit = cvglmnet(x = variants, y = p.values.astype('float64'), family = regression_type,
File "~/miniconda3/envs/pyseer/lib/python3.10/site-packages/glmnet_python/cvglmnet.py", line 260, in cvglmnet
ma = scipy.tile(scipy.arange(nfolds), [1, scipy.floor(nobs/nfolds)])
File "~/miniconda3/envs/pyseer/lib/python3.10/site-packages/scipy/_lib/deprecation.py", line 20, in call
return fun(*args, **kwargs)
File "~/miniconda3/envs/pyseer/lib/python3.10/site-packages/numpy/lib/shape_base.py", line 1274, in tile
return c.reshape(shape_out)
TypeError: 'numpy.float64' object cannot be interpreted as an integer
thanks, I will need to look into this a bit more closely. in the meantime can you also try to run the command with a phenotype file that contains only the training set samples?
Do you mean the unitig file? The phenotype file already contains only the training set.
Ah I see. I would try to provide input files that only contain the samples you are using to train the model; the enet
code is a generally less robust to mismatches (still not 100% sure that is the issue though!)
I got this command pyseer --kmers train80_unitigs.pyseer --uncompressed --phenotypes train80_salt.pheno --wg enet --load-vars output/salt_unitigs --save-model salt_unitigs.ridge --cpu 12 --alpha 0.01 > salt_selected_unitigs.tx
to work by making the unitig file train80_unitigs.pyseer
out of the training set strains only. I even managed to predict the phenotypes of the test set quite accurately with the command enet_predict_pyseer --kmers all_unitigs.pyseer --uncompressed --lineage-clusters lineage_files/lineages_CC.txt --true-values test20_salt.pheno salt_unitigs.ridge.pkl test20_strains.txt > salt_ridge_predictions.txt
where all_unitigs.pyseer
contains the unitigs called from the combined strain set.
Is it now this unitig file after --kmers option that I can use to input different combinations of variants to test their effect on the phenotype?
Glad to hear that you managed to run your analysis. This tells us that we need to make that code path a bit more robust for mismatches in samples.
Regarding your last question, I am not sure I understand what exactly your plan is, but you can certainly fit the trained model with a made-up combinations of unitigs and see what the prediction will be based on them.
Will close the issue for now
The problem came back, when I changed to another computer with a newer Ubuntu version, so I dug around a little. I got pyseer elastic net to work without a problem by simply using an even older version (3.7.6) of python with conda like this
conda create -n <NAME> python=3.7.6
and installing pyseer in that environment
conda install pyseer
.
Older versions of python seem to have a problem with HTSlib not being found, and the very newest python versions throw that numpy error above. Has something drastically changed in the newer numpy packages?
In this python3.7.6 environment the installed package versions are as follows:
pyseer 1.3.11
numpy 1.21.6
scipy 1.7.3
pandas 1.3.5
scikit-learn 1.0.2
pysam 0.16.0
glmnet_py - the one wrapped with pyseer
matplotlib 3.5.3
DendroPy 4.6.1
pybedtools 0.9.0
bedtools 2.31.
bedops 2.4.41
can you indicate the package versions (including python itself) for the environments that give you errors? I have a test environment with the following packages and it works on the test suite:
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
bedops 2.4.41 h9f5acd7_0 bioconda
bedtools 2.31.0 hf5e1c6e_2 bioconda
brotli 1.0.9 h166bdaf_8 conda-forge
brotli-bin 1.0.9 h166bdaf_8 conda-forge
brotlipy 0.7.0 py310h5764c6d_1005 conda-forge
bwa 0.7.17 he4a0461_11 bioconda
bzip2 1.0.8 h7f98852_4 conda-forge
c-ares 1.19.0 hd590300_0 conda-forge
ca-certificates 2023.7.22 hbcca054_0 conda-forge
capnproto 0.10.2 h6239696_0 conda-forge
certifi 2023.7.22 pyhd8ed1ab_0 conda-forge
cffi 1.15.1 py310h255011f_3 conda-forge
charset-normalizer 3.1.0 pyhd8ed1ab_0 conda-forge
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
contourpy 1.0.7 py310hdf3cbec_0 conda-forge
cryptography 39.0.0 py310h65dfdc0_0 conda-forge
cycler 0.11.0 pyhd8ed1ab_0 conda-forge
dendropy 4.6.0 pyh7cba7a3_0 bioconda
fonttools 4.39.4 py310h2372a71_0 conda-forge
freetype 2.12.1 hca18f0e_1 conda-forge
glmnet_py 1.0.2 py310h7d7f266_5 conda-forge
gsl 2.7 he838d99_0 conda-forge
htslib 1.17 h6bc39ce_1 bioconda
idna 3.4 pyhd8ed1ab_0 conda-forge
joblib 1.2.0 pyhd8ed1ab_0 conda-forge
jpeg 9e h0b41bf4_3 conda-forge
keyutils 1.6.1 h166bdaf_0 conda-forge
kiwisolver 1.4.4 py310hbf28c38_1 conda-forge
krb5 1.20.1 hf9c8cef_0 conda-forge
lcms2 2.14 h6ed2654_0 conda-forge
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
lerc 4.0.0 h27087fc_0 conda-forge
libblas 3.9.0 16_linux64_openblas conda-forge
libbrotlicommon 1.0.9 h166bdaf_8 conda-forge
libbrotlidec 1.0.9 h166bdaf_8 conda-forge
libbrotlienc 1.0.9 h166bdaf_8 conda-forge
libcblas 3.9.0 16_linux64_openblas conda-forge
libcurl 7.87.0 h6312ad2_0 conda-forge
libdeflate 1.13 h166bdaf_0 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 h516909a_1 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc-ng 12.2.0 h65d4601_19 conda-forge
libgfortran-ng 12.2.0 h69a702a_19 conda-forge
libgfortran5 12.2.0 h337968e_19 conda-forge
libgomp 12.2.0 h65d4601_19 conda-forge
liblapack 3.9.0 16_linux64_openblas conda-forge
libnghttp2 1.51.0 hdcd2b5c_0 conda-forge
libnsl 2.0.0 h7f98852_0 conda-forge
libopenblas 0.3.21 pthreads_h78a6416_3 conda-forge
libpng 1.6.39 h753d276_0 conda-forge
libsqlite 3.42.0 h2797004_0 conda-forge
libssh2 1.10.0 haa6b8db_3 conda-forge
libstdcxx-ng 12.2.0 h46fd767_19 conda-forge
libtiff 4.4.0 h0e0dad5_3 conda-forge
libuuid 2.38.1 h0b41bf4_0 conda-forge
libwebp-base 1.3.0 h0b41bf4_0 conda-forge
libxcb 1.13 h7f98852_1004 conda-forge
libzlib 1.2.13 h166bdaf_4 conda-forge
mash 2.3 hd3113c8_6 bioconda
matplotlib-base 3.7.1 py310he60537e_0 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
ncurses 6.3 h27087fc_1 conda-forge
numpy 1.24.3 py310ha4c1d20_0 conda-forge
openjpeg 2.5.0 h7d73246_1 conda-forge
openssl 1.1.1w hd590300_0 conda-forge
packaging 23.1 pyhd8ed1ab_0 conda-forge
pandas 2.0.1 py310h7cbd5c2_1 conda-forge
patsy 0.5.3 pyhd8ed1ab_0 conda-forge
perl 5.32.1 2_h7f98852_perl5 conda-forge
pillow 9.2.0 py310h454ad03_3 conda-forge
pip 23.1.2 pyhd8ed1ab_0 conda-forge
platformdirs 3.5.1 pyhd8ed1ab_0 conda-forge
pooch 1.7.0 pyha770c72_3 conda-forge
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
pybedtools 0.9.0 py310h2b6aa90_2 bioconda
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pyopenssl 23.1.1 pyhd8ed1ab_0 conda-forge
pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge
pysam 0.21.0 py310hff46b53_0 bioconda
pyseer 1.3.11 pyh7cba7a3_0 bioconda
pysocks 1.7.1 pyha2e5f31_6 conda-forge
python 3.10.8 h257c98d_0_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-tzdata 2023.3 pyhd8ed1ab_0 conda-forge
python_abi 3.10 3_cp310 conda-forge
pytz 2023.3 pyhd8ed1ab_0 conda-forge
readline 8.2 h8228510_1 conda-forge
requests 2.29.0 pyhd8ed1ab_0 conda-forge
samtools 1.17 hd87286a_1 bioconda
scikit-learn 1.2.2 py310h41b6a48_1 conda-forge
scipy 1.10.1 py310ha4c1d20_3 conda-forge
setuptools 67.7.2 pyhd8ed1ab_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
statsmodels 0.14.0 py310h278f3c1_1 conda-forge
threadpoolctl 3.1.0 pyh8a188c0_0 conda-forge
tk 8.6.12 h27826a3_0 conda-forge
tqdm 4.65.0 pyhd8ed1ab_1 conda-forge
typing-extensions 4.5.0 hd8ed1ab_0 conda-forge
typing_extensions 4.5.0 pyha770c72_0 conda-forge
tzdata 2023c h71feb2d_0 conda-forge
unicodedata2 15.0.0 py310h5764c6d_0 conda-forge
urllib3 1.26.15 pyhd8ed1ab_0 conda-forge
wheel 0.40.0 pyhd8ed1ab_0 conda-forge
xorg-libxau 1.0.9 h7f98852_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
zlib 1.2.13 h166bdaf_4 conda-forge
zstd 1.5.2 h3eb15da_6 conda-forge
It is using python 3.10.8 as you can see
This work computer with Ubuntu 23.04 is the one from the original post. I installed the python3.7.6 environment on this computer and now it works fine. The original environment has python 3.10.11 and has the following packages installed (sorry, there are other bioinformatics packages not related to pyseer mixed up in there):
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
armadillo 12.6.4 h37e7c65_1 conda-forge
arpack 3.8.0 nompi_h0baa96a_101 conda-forge
atk-1.0 2.38.0 hd4edc92_1 conda-forge
bedops 2.4.41 h4ac6f70_1 bioconda
bedtools 2.31.0 hf5e1c6e_3 bioconda
bifrost 1.2.1 h43eeafb_0 bioconda
blast 2.14.1 pl5321h6f7f691_0 bioconda
brotli 1.1.0 hd590300_1 conda-forge
brotli-bin 1.1.0 hd590300_1 conda-forge
bwa 0.7.17 he4a0461_11 bioconda
bzip2 1.0.8 h7b6447c_0
c-ares 1.20.1 hd590300_0 conda-forge
ca-certificates 2023.08.22 h06a4308_0
cairo 1.16.0 hb05425b_5
capnproto 1.0.1 hfc55251_0 conda-forge
cd-hit 4.8.1 h43eeafb_9 bioconda
certifi 2023.7.22 pyhd8ed1ab_0 conda-forge
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
contourpy 1.1.1 py310hd41b1e2_1 conda-forge
curl 7.87.0 h6312ad2_0 conda-forge
cycler 0.12.1 pyhd8ed1ab_0 conda-forge
dendropy 4.6.1 pyhdfd78af_0 bioconda
entrez-direct 16.2 he881be0_1 bioconda
expat 2.5.0 hcb278e6_1 conda-forge
fasttree 2.1.11 h031d066_2 bioconda
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 hab24e00_0 conda-forge
fontconfig 2.14.1 h4c34cd2_2
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
fonttools 4.43.1 py310h2372a71_0 conda-forge
freetype 2.12.1 h267a509_2 conda-forge
fribidi 1.0.10 h36c2ea0_0 conda-forge
fsm-lite 1.0 h4ac6f70_5 bioconda
gdk-pixbuf 2.42.10 h5eee18b_0
gettext 0.21.1 h27087fc_0 conda-forge
giflib 5.2.1 h0b41bf4_3 conda-forge
glib 2.78.0 hfc55251_0 conda-forge
glib-tools 2.78.0 hfc55251_0 conda-forge
glmnet-py 0.1.0b2 pypi_0 pypi
glmnet_py 1.0.2 py310h7d7f266_5 conda-forge
graphite2 1.3.13 h58526e2_1001 conda-forge
graphviz 7.1.0 h2e5815a_0 conda-forge
gsl 2.7 he838d99_0 conda-forge
gtk2 2.24.33 h90689f9_2 conda-forge
gts 0.7.6 h977cf35_4 conda-forge
harfbuzz 6.0.0 h8e241bc_0 conda-forge
hdf5 1.10.3 hc401514_2 conda-forge
htslib 1.17 h6bc39ce_1 bioconda
icu 70.1 h27087fc_0 conda-forge
joblib 1.3.2 pyhd8ed1ab_0 conda-forge
jpeg 9e h166bdaf_2 conda-forge
keyutils 1.6.1 h166bdaf_0 conda-forge
kiwisolver 1.4.5 py310hd41b1e2_1 conda-forge
krb5 1.20.1 hf9c8cef_0 conda-forge
lcms2 2.14 h6ed2654_0 conda-forge
ld_impl_linux-64 2.38 h1181459_1
lerc 4.0.0 h27087fc_0 conda-forge
libblas 3.9.0 18_linux64_openblas conda-forge
libbrotlicommon 1.1.0 hd590300_1 conda-forge
libbrotlidec 1.1.0 hd590300_1 conda-forge
libbrotlienc 1.1.0 hd590300_1 conda-forge
libcblas 3.9.0 18_linux64_openblas conda-forge
libcurl 7.87.0 h6312ad2_0 conda-forge
libdb 6.2.32 h9c3ff4c_0 conda-forge
libdeflate 1.13 h166bdaf_0 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 h516909a_1 conda-forge
libexpat 2.5.0 hcb278e6_1 conda-forge
libffi 3.4.4 h6a678d5_0
libgcc 7.2.0 h69d50b8_2 conda-forge
libgcc-ng 13.2.0 h807b86a_2 conda-forge
libgd 2.3.3 h18fbbfe_3 conda-forge
libgfortran 3.0.0 1 conda-forge
libgfortran-ng 13.2.0 h69a702a_2 conda-forge
libgfortran5 13.2.0 ha4646dd_2 conda-forge
libglib 2.78.0 hebfc3b9_0 conda-forge
libgomp 13.2.0 h807b86a_2 conda-forge
libiconv 1.17 h166bdaf_0 conda-forge
libidn2 2.3.4 h166bdaf_0 conda-forge
libjpeg-turbo 2.1.4 h166bdaf_0 conda-forge
liblapack 3.9.0 18_linux64_openblas conda-forge
libnghttp2 1.51.0 hdcd2b5c_0 conda-forge
libnsl 2.0.0 hd590300_1 conda-forge
libopenblas 0.3.24 pthreads_h413a1c8_0 conda-forge
libpng 1.6.39 h753d276_0 conda-forge
librsvg 2.54.4 h7abd40a_0 conda-forge
libssh2 1.10.0 haa6b8db_3 conda-forge
libstdcxx-ng 13.2.0 h7e041cc_2 conda-forge
libtiff 4.4.0 h0e0dad5_3 conda-forge
libtool 2.4.7 h27087fc_0 conda-forge
libunistring 0.9.10 h7f98852_0 conda-forge
libuuid 1.41.5 h5eee18b_0
libwebp 1.3.2 h11a3e52_0
libwebp-base 1.3.2 hd590300_0 conda-forge
libxcb 1.15 h0b41bf4_0 conda-forge
libxml2 2.10.3 hca2bb57_4 conda-forge
libxslt 1.1.37 h873f0b0_0 conda-forge
libzlib 1.2.13 hd590300_5 conda-forge
mafft 7.520 h031d066_2 bioconda
mash 1.1 0 bioconda
matplotlib-base 3.8.0 py310h62c0568_1 conda-forge
mcl 14.137 h470a237_3 bioconda
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
ncbi-vdb 3.0.8 hdbdd923_0 bioconda
ncurses 6.4 h6a678d5_0
numpy 1.26.0 py310hb13e2d6_0 conda-forge
openjpeg 2.5.0 h7d73246_1 conda-forge
openssl 1.1.1w hd590300_0 conda-forge
ossuuid 1.6.2 hf484d3e_1000 conda-forge
packaging 23.2 pyhd8ed1ab_0 conda-forge
pandas 2.1.1 py310hcc13569_1 conda-forge
pango 1.50.12 hd33c08f_1 conda-forge
parallel 20230922 ha770c72_0 conda-forge
patsy 0.5.3 pyhd8ed1ab_0 conda-forge
pcre 8.45 h9c3ff4c_0 conda-forge
pcre2 10.40 hc3806b6_0 conda-forge
perl 5.22.0.1 0 conda-forge
perl-aceperl 1.92 0 bioconda
perl-algorithm-diff 1.1903 pl5.22.0_1 bioconda
perl-algorithm-munkres 0.08 0 bioconda
perl-apache-test 1.40 0 bioconda
perl-app-cpanminus 1.7043 pl5.22.0_0 bioconda
perl-appconfig 1.71 0 bioconda
perl-archive-tar 2.18 pl5.22.0_2 bioconda
perl-array-compare 2.11 0 bioconda
perl-array-utils 0.5 0 bioconda
perl-b 1.48 pl5.22.0_0 bioconda
perl-b-hooks-endofscope 0.21 pl5.22.0_0 bioconda
perl-bio-asn1-entrezgene 1.72 1 bioconda
perl-bio-featureio 1.6.905 0 bioconda
perl-bio-phylo 0.58 0 bioconda
perl-bio-samtools 1.43 0 bioconda
perl-bioperl 1.6.924 pl5.22.0_7 bioconda
perl-bioperl-core 1.6.924 pl5.22.0_2 bioconda
perl-bioperl-run 1.006900 2 bioconda
perl-capture-tiny 0.36 0 bioconda
perl-carp 1.38 pl5.22.0_0 bioconda
perl-cgi 4.22 3 bioconda
perl-class-data-inheritable 0.08 0 bioconda
perl-class-inspector 1.28 0 bioconda
perl-class-method-modifiers 2.11 1 bioconda
perl-clone 0.38 0 bioconda
perl-common-sense 3.74 0 bioconda
perl-compress-raw-bzip2 2.069 1 bioconda
perl-compress-raw-zlib 2.069 3 bioconda
perl-convert-binary-c 0.78 0 bioconda
perl-convert-binhex 1.125 0 bioconda
perl-cpan-meta-check 0.012 1 bioconda
perl-crypt-rc4 2.02 0 bioconda
perl-data-dumper 2.161 pl5.22.0_0 bioconda
perl-data-optlist 0.110 0 bioconda
perl-data-stag 0.14 0 bioconda
perl-datetime 1.42 pl5.22.0_0 bioconda
perl-datetime-locale 1.12 pl5.22.0_0 bioconda
perl-datetime-timezone 2.09 pl5.22.0_0 bioconda
perl-db-file 1.835 5 bioconda
perl-dbd-sqlite 1.50 1 bioconda
perl-dbi 1.641 pl5.22.0_0 bioconda
perl-devel-globaldestruction 0.13 1 bioconda
perl-devel-stacktrace 2.03 pl5.22.0_0 bioconda
perl-digest-hmac 1.03 pl5.22.0_1 bioconda
perl-digest-md5-file 0.08 pl5.22.0_0 bioconda
perl-digest-perl-md5 1.9 0 bioconda
perl-dist-checkconflicts 0.11 0 bioconda
perl-email-date-format 1.005 0 bioconda
perl-encode-locale 1.05 pl5.22.0_4 bioconda
perl-env-path 0.19 0 bioconda
perl-error 0.17024 0 bioconda
perl-eval-closure 0.14 pl5.22.0_0 bioconda
perl-exception-class 1.40 0 bioconda
perl-exporter 5.72 pl5.22.0_0 bioconda
perl-exporter-tiny 0.042 1 bioconda
perl-extutils-makemaker 7.24 pl5.22.0_1 bioconda
perl-file-find-rule 0.34 3 bioconda
perl-file-grep 0.02 0 bioconda
perl-file-listing 6.04 0 bioconda
perl-file-slurp-tiny 0.004 0 bioconda
perl-file-slurper 0.008 0 bioconda
perl-file-sort 1.01 pl5.22.0_1 bioconda
perl-file-which 1.20 0 bioconda
perl-font-afm 1.20 0 bioconda
perl-gd 2.56 pl522h470a237_9 bioconda
perl-getopt-long 2.50 pl5.22.0_0 bioconda
perl-graph 0.9704 0 bioconda
perl-graph-readwrite 2.09 0 bioconda
perl-graphviz 2.20 1 bioconda
perl-html-element-extended 1.18 0 bioconda
perl-html-entities-numbered 0.04 0 bioconda
perl-html-formatter 2.14 0 bioconda
perl-html-parser 3.72 pl5.22.0_1 bioconda
perl-html-tableextract 2.13 0 bioconda
perl-html-tagset 3.20 pl5.22.0_1 bioconda
perl-html-tidy 1.56 1 bioconda
perl-html-tree 5.03 0 bioconda
perl-html-treebuilder-xpath 0.14 0 bioconda
perl-http-cookies 6.01 0 bioconda
perl-http-daemon 6.01 0 bioconda
perl-http-date 6.02 pl5.22.0_1 bioconda
perl-http-message 6.11 0 bioconda
perl-http-negotiate 6.01 0 bioconda
perl-image-info 1.38 0 bioconda
perl-image-size 3.300 0 bioconda
perl-importer 0.024 0 bioconda
perl-io-compress 2.069 pl5.22.0_2 bioconda
perl-io-html 1.001 pl5.22.0_1 bioconda
perl-io-sessiondata 1.03 0 bioconda
perl-io-socket-ssl 2.024 0 bioconda
perl-io-string 1.08 pl5.22.0_1 bioconda
perl-io-stringy 2.111 0 bioconda
perl-io-tty 1.12 0 bioconda
perl-io-zlib 1.10 1 bioconda
perl-ipc-run 0.94 0 bioconda
perl-jcode 2.07 0 bioconda
perl-json 2.90 1 bioconda
perl-json-xs 2.34 0 bioconda
perl-libwww-perl 6.15 0 bioconda
perl-libxml-perl 0.08 0 bioconda
perl-list-moreutils 0.428 pl5.22.0_0 bioconda
perl-list-util 1.38 pl5.22.0_0 bioconda
perl-log-log4perl 1.47 1 bioconda
perl-lwp-mediatypes 6.02 pl5.22.0_1 bioconda
perl-lwp-protocol-https 6.06 2 bioconda
perl-lwp-simple 6.15 3 bioconda
perl-mailtools 2.14 0 bioconda
perl-math-cdf 0.1 3 bioconda
perl-math-derivative 0.04 0 bioconda
perl-math-random 0.72 0 bioconda
perl-math-spline 0.02 0 bioconda
perl-mime-lite 3.030 0 bioconda
perl-mime-tools 5.507 0 bioconda
perl-mime-types 2.12 0 bioconda
perl-mldbm 2.05 0 bioconda
perl-module-build 0.4224 pl5.22.0_0 bioconda
perl-module-implementation 0.09 0 bioconda
perl-module-runtime 0.014 1 bioconda
perl-moo 2.001000 1 bioconda
perl-moose 2.2009 pl5.22.0_0 bioconda
perl-namespace-autoclean 0.28 pl5.22.0_0 bioconda
perl-namespace-clean 0.27 pl5.22.0_0 bioconda
perl-net-http 6.09 0 bioconda
perl-net-ssleay 1.74 0 bioconda
perl-ntlm 1.09 1 bioconda
perl-number-compare 0.03 pl5.22.0_1 bioconda
perl-ole-storage_lite 0.19 0 bioconda
perl-package-stash 0.37 pl5.22.0_0 bioconda
perl-params-util 1.07 1 bioconda
perl-params-validationcompiler 0.23 pl5.22.0_0 bioconda
perl-parse-recdescent 1.967013 0 bioconda
perl-parse-yapp 1.05 0 bioconda
perl-pathtools 3.73 h470a237_2 bioconda
perl-pdf-api2 2.025 2 bioconda
perl-perlio-utf8_strict 0.006 0 bioconda
perl-postscript 0.06 0 bioconda
perl-role-tiny 2.000001 1 bioconda
perl-scalar-list-utils 1.45 2 bioconda
perl-set-scalar 1.29 0 bioconda
perl-soap-lite 1.19 0 bioconda
perl-sort-naturally 1.03 0 bioconda
perl-specio-exporter 0.36 pl5.22.0_0 bioconda
perl-spreadsheet-parseexcel 0.65 0 bioconda
perl-spreadsheet-writeexcel 2.40 0 bioconda
perl-statistics-descriptive 3.0612 0 bioconda
perl-sub-exporter 0.987 0 bioconda
perl-sub-exporter-progressive 0.001011 1 bioconda
perl-sub-identify 0.12 pl5.22.0_0 bioconda
perl-sub-install 0.928 0 bioconda
perl-sub-uplevel 0.25 2 bioconda
perl-svg 2.64 0 bioconda
perl-svg-graph 0.02 0 bioconda
perl-task-weaken 1.04 0 bioconda
perl-template-toolkit 2.26 0 bioconda
perl-test-builder-tester 1.23_002 0 bioconda
perl-test-deep 1.120 1 bioconda
perl-test-differences 0.64 0 bioconda
perl-test-exception 0.43 0 bioconda
perl-test-fatal 0.014 1 bioconda
perl-test-files 0.14 0 bioconda
perl-test-leaktrace 0.16 pl5.22.0_0 bioconda
perl-test-more 1.001002 pl5.22.0_0 bioconda
perl-test-most 0.34 1 bioconda
perl-test-output 1.03 0 bioconda
perl-test-pod 1.51 0 bioconda
perl-test-simple 1.302075 pl5.22.0_0 bioconda
perl-test-warn 0.30 0 bioconda
perl-test-warnings 0.021 1 bioconda
perl-test2 1.302075 pl5.22.0_0 bioconda
perl-test2-suite 0.000061 0 bioconda
perl-text-csv 1.33 0 bioconda
perl-text-diff 1.44 0 bioconda
perl-text-glob 0.11 pl5.22.0_0 bioconda
perl-threaded 5.32.1 hdfd78af_1 bioconda
perl-tie-ixhash 1.23 pl5.22.0_1 bioconda
perl-timedate 2.30 0 bioconda
perl-tree-dag_node 1.29 0 bioconda
perl-try-tiny 0.24 pl5.22.0_2 bioconda
perl-type-tiny 1.000005 0 bioconda
perl-unicode-map 0.112 0 bioconda
perl-uri 1.71 pl5.22.0_1 bioconda
perl-variable-magic 0.61 pl5.22.0_0 bioconda
perl-warnings-register 1.03 pl5.22.0_0 bioconda
perl-www-robotrules 6.02 0 bioconda
perl-xml-dom 1.45 0 bioconda
perl-xml-dom-xpath 0.14 0 bioconda
perl-xml-filter-buffertext 1.01 0 bioconda
perl-xml-libxml 2.0124 0 bioconda
perl-xml-libxslt 1.94 0 bioconda
perl-xml-namespacesupport 1.11 0 bioconda
perl-xml-parser 2.44 4 bioconda
perl-xml-regexp 0.04 0 bioconda
perl-xml-sax 0.99 0 bioconda
perl-xml-sax-base 1.08 0 bioconda
perl-xml-sax-expat 0.51 0 bioconda
perl-xml-sax-writer 0.56 0 bioconda
perl-xml-simple 2.22 0 bioconda
perl-xml-twig 3.49 0 bioconda
perl-xml-writer 0.625 0 bioconda
perl-xml-xpath 1.33 0 bioconda
perl-xml-xpathengine 0.14 0 bioconda
perl-xsloader 0.22 pl5.22.0_0 bioconda
perl-yaml 1.24 pl522_1 bioconda
pillow 10.0.1 py310ha6cbd5a_0
pip 23.2.1 py310h06a4308_0
pixman 0.42.2 h59595ed_0 conda-forge
prank v.170427 h4ac6f70_7 bioconda
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
pybedtools 0.9.1 py310h2b6aa90_0 bioconda
pyparsing 3.1.1 pyhd8ed1ab_0 conda-forge
pysam 0.21.0 py310hff46b53_0 bioconda
pyseer 1.3.11 pyh7cba7a3_0 bioconda
python 3.10.11 h7a1cb2a_2
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-tzdata 2023.3 pyhd8ed1ab_0 conda-forge
python_abi 3.10 2_cp310 conda-forge
pytz 2023.3.post1 pyhd8ed1ab_0 conda-forge
readline 8.2 h5eee18b_0
roary 3.12.0 pl5.22.0_0 bioconda
samtools 1.18 hd87286a_0 bioconda
scikit-learn 1.3.1 py310h1fdf081_1 conda-forge
scipy 1.11.3 py310hb13e2d6_1 conda-forge
sdsl-lite 2.1.1 h00ab1b0_1002 conda-forge
seer 1.1.4 hb75bb0b_0 bioconda
setuptools 68.0.0 py310h06a4308_0
six 1.16.0 pyh6c4a22f_0 conda-forge
sqlite 3.41.2 h5eee18b_0
statsmodels 0.14.0 py310h1f7b6fc_2 conda-forge
superlu 5.2.2 h00795ac_0 conda-forge
threadpoolctl 3.2.0 pyha21a80b_0 conda-forge
tidyp 1.04 hec16e2b_4 bioconda
tk 8.6.12 h1ccaba5_0
tqdm 4.66.1 pyhd8ed1ab_0 conda-forge
tzdata 2023c h04d1e81_0
unicodedata2 15.1.0 py310h2372a71_0 conda-forge
unitig-caller 1.3.0 py310ha320341_2 bioconda
unitig-counter 1.1.0 h56fc30b_0 bioconda
wget 1.20.3 ha56f1ee_1 conda-forge
wheel 0.41.2 py310h06a4308_0
xorg-libxau 1.0.11 hd590300_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xz 5.2.10 h5eee18b_1
zlib 1.2.13 hd590300_5 conda-forge
zstd 1.5.5 hfc55251_0 conda-forge
I got pyseer to work temporarily on my home computer with an older Ubuntu. I'll update this post related to that when I get home.
Ok, if possible please see if you can get a clean environment which reproduces the error
Okay, I don't know what's the problem anymore , because It works fine in this kind of clean environment:
# packages in environment at ~/miniconda3/envs/pyseer_clean_3.10.11:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
bedops 2.4.41 h4ac6f70_1 bioconda
bedtools 2.31.1 hf5e1c6e_0 bioconda
brotli 1.1.0 hd590300_1 conda-forge
brotli-bin 1.1.0 hd590300_1 conda-forge
bwa 0.7.17 he4a0461_11 bioconda
bzip2 1.0.8 hd590300_5 conda-forge
c-ares 1.21.0 hd590300_0 conda-forge
ca-certificates 2023.7.22 hbcca054_0 conda-forge
capnproto 0.9.1 ha19adfc_4 conda-forge
certifi 2023.7.22 pyhd8ed1ab_0 conda-forge
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
contourpy 1.2.0 py310hd41b1e2_0 conda-forge
cycler 0.12.1 pyhd8ed1ab_0 conda-forge
dendropy 4.6.1 pyhdfd78af_0 bioconda
fonttools 4.44.0 py310h2372a71_0 conda-forge
freetype 2.12.1 h267a509_2 conda-forge
glmnet_py 1.0.2 py310h7d7f266_5 conda-forge
gsl 2.7 he838d99_0 conda-forge
htslib 1.18 h81da01d_0 bioconda
joblib 1.3.2 pyhd8ed1ab_0 conda-forge
keyutils 1.6.1 h166bdaf_0 conda-forge
kiwisolver 1.4.5 py310hd41b1e2_1 conda-forge
krb5 1.21.2 h659d440_0 conda-forge
lcms2 2.15 h7f713cb_2 conda-forge
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
lerc 4.0.0 h27087fc_0 conda-forge
libblas 3.9.0 19_linux64_openblas conda-forge
libbrotlicommon 1.1.0 hd590300_1 conda-forge
libbrotlidec 1.1.0 hd590300_1 conda-forge
libbrotlienc 1.1.0 hd590300_1 conda-forge
libcblas 3.9.0 19_linux64_openblas conda-forge
libcurl 8.4.0 hca28451_0 conda-forge
libdeflate 1.18 h0b41bf4_0 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 h516909a_1 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc-ng 13.2.0 h807b86a_2 conda-forge
libgfortran-ng 13.2.0 h69a702a_2 conda-forge
libgfortran5 13.2.0 ha4646dd_2 conda-forge
libgomp 13.2.0 h807b86a_2 conda-forge
libjpeg-turbo 2.1.5.1 hd590300_1 conda-forge
liblapack 3.9.0 19_linux64_openblas conda-forge
libnghttp2 1.55.1 h47da74e_0 conda-forge
libnsl 2.0.1 hd590300_0 conda-forge
libopenblas 0.3.24 pthreads_h413a1c8_0 conda-forge
libpng 1.6.39 h753d276_0 conda-forge
libsqlite 3.44.0 h2797004_0 conda-forge
libssh2 1.11.0 h0841786_0 conda-forge
libstdcxx-ng 13.2.0 h7e041cc_2 conda-forge
libtiff 4.6.0 h8b53f26_0 conda-forge
libuuid 2.38.1 h0b41bf4_0 conda-forge
libwebp-base 1.3.2 hd590300_0 conda-forge
libxcb 1.15 h0b41bf4_0 conda-forge
libzlib 1.2.13 hd590300_5 conda-forge
mash 2.3 ha9a2dd8_3 bioconda
matplotlib-base 3.8.1 py310h62c0568_0 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
ncurses 6.4 h59595ed_2 conda-forge
numpy 1.26.0 py310hb13e2d6_0 conda-forge
openjpeg 2.5.0 h488ebb8_3 conda-forge
openssl 3.1.4 hd590300_0 conda-forge
packaging 23.2 pyhd8ed1ab_0 conda-forge
pandas 2.1.2 py310hcc13569_0 conda-forge
patsy 0.5.3 pyhd8ed1ab_0 conda-forge
perl 5.32.1 4_hd590300_perl5 conda-forge
pillow 10.0.1 py310h29da1c1_1 conda-forge
pip 23.3.1 pyhd8ed1ab_0 conda-forge
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
pybedtools 0.9.1 py310h2b6aa90_0 bioconda
pyparsing 3.1.1 pyhd8ed1ab_0 conda-forge
pysam 0.22.0 py310h41dec4a_0 bioconda
pyseer 1.3.11 pyh7cba7a3_0 bioconda
python 3.10.11 he550d4f_0_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-tzdata 2023.3 pyhd8ed1ab_0 conda-forge
python_abi 3.10 4_cp310 conda-forge
pytz 2023.3.post1 pyhd8ed1ab_0 conda-forge
readline 8.2 h8228510_1 conda-forge
samtools 1.18 h50ea8bc_1 bioconda
scikit-learn 1.3.2 py310h1fdf081_1 conda-forge
scipy 1.11.3 py310hb13e2d6_1 conda-forge
setuptools 68.2.2 pyhd8ed1ab_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
statsmodels 0.14.0 py310h1f7b6fc_2 conda-forge
threadpoolctl 3.2.0 pyha21a80b_0 conda-forge
tk 8.6.13 noxft_h4845f30_101 conda-forge
tqdm 4.66.1 pyhd8ed1ab_0 conda-forge
tzdata 2023c h71feb2d_0 conda-forge
unicodedata2 15.1.0 py310h2372a71_0 conda-forge
wheel 0.41.3 pyhd8ed1ab_0 conda-forge
xorg-libxau 1.0.11 hd590300_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
zlib 1.2.13 hd590300_5 conda-forge
zstd 1.5.5 hfc55251_0 conda-forge
I can see that in the environment that caused the error you have two versions of glmnet installed:
glmnet-py 0.1.0b2 pypi_0 pypi
glmnet_py 1.0.2 py310h7d7f266_5 conda-forge
That's probably it
Well damn, thank you!
Hello!
I'm trying to use pyseer's elastic net feature to assess the effects of different variant groups on the phenotype, i.e. to find biologically plausible combinations of variants that predict the phenotype of the test set particularly well. I have randomly divided the set into training (80%) and test sets (20%). When I fit elastic net to the whole set, it works fine. When I try to train a model with the training set, it throws an error.
In other words, this command
pyseer --kmers output_unitigs.pyseer --uncompressed --phenotypes phenotypes/phenotypes.salt --wg enet --save-vars output/salt_unitigs --save-model salt_unitigs.ridge --cpu 12 --alpha 0.01 > salt_selected_unitigs.txt
works fine, but when I'm trying to fit a model to the training setpyseer --kmers output_unitigs.pyseer --uncompressed --phenotypes train80_salt.pheno --wg enet --load-vars output/salt_unitigs --save-model salt_unitigs.ridge --cpu 12 --alpha 0.01 > salt_selected_unitigs.txt
, glmnet throws an error like this:Why does it say
Fitting elastic net to top 0 variants
when with the whole set it saysFitting elastic net to top 324665 variants
? An unitig compatibility issue?I'm using a conda environment with python 3.10.11 and these package versions: