Open shenwei356 opened 1 year ago
Hi @shenwei356, could you please run conda list
on your system? It will tell you the exact conda-versions of all tools in your environment. However, if you have wfmash v0.10.0-4-gcf9bfb0
(this is not a conda-version), it means that you have a version not fully portable that can break in older systems.
I am not a conda
guru, so I can't properly answer about the versioning problem. If you get more information about it, I will be apply to improve the bioconda recipes' for all pggb
's tools.
Why do you recommend specifying the version for cmake
and gcc
in the meta.yaml
? I see that a few tools specify the version of cmake
, but not for gcc
. Do you know examples of that?
Hi Andrea, thanks for your reply.
However, if you have wfmash v0.10.0-4-gcf9bfb0 (this is not a conda-version), it means that you have a version not fully portable that can break in older systems.
wfmash v0.10.0-4-gcf9bfb0
indeed does not match the commit of the v0.10.0 (e7ab130).
Did you draft the release page on Oct 5, 2022 (e7ab130) and tag the release on Oct 28, 2022 (cf9bfb0)? That might be the cause.
$ git clone https://github.com/waveygang/wfmash
cd wfmash
$ git describe --always --tags --long
v0.10.0-9-gcb0ce95
$ git checkout git checkout cf9bfb0ef9fe5b973b91ee3aa649533df7521557
$ git describe --always --tags --long
v0.10.0-4-gcf9bfb0
$ git checkout e7ab130
$ git describe --always --tags --long
v0.10.0-0-ge7ab130
$ which wfmash
/home/shenwei/Public/app/miniconda3/envs/pggb/bin/wfmash
$ wfmash --version
v0.10.0-4-gcf9bfb0
$ ls -lh $(which wfmash)
-rwxrwxr-x 2 shenwei shenwei 3.9M 11月13日 05:45 /home/shenwei/Public/app/miniconda3/envs/pggb/bin/wfmash
$ tar -jtvf pkgs/wfmash-0.10.0-hfdddef0_2.tar.bz2
-rw-r--r-- 1001/1001 11 2022-11-13 05:45 info/files
-rw-r--r-- 1001/1001 34 2022-11-13 05:45 info/test/run_test.sh
-rw-r--r-- 1001/1001 229 2022-11-13 05:45 info/paths.json
-rw-r--r-- 1001/1001 265 2022-11-13 05:45 info/hash_input.json
-rw-r--r-- 1001/1001 391 2022-11-13 05:45 info/index.json
-rw-r--r-- 1001/1001 482 2022-11-13 05:36 info/recipe/build.sh
-rw-r--r-- 1001/1001 824 2022-11-13 05:36 info/recipe/meta.yaml.template
-rw-rw-r-- 1001/1001 1079 2022-10-28 10:26 info/licenses/LICENSE
-rw-r--r-- 1001/1001 3273 2022-11-13 05:45 info/recipe/meta.yaml
-rw-r--r-- 1001/1001 6497 2022-11-13 05:45 info/recipe/conda_build_config.yaml
-rw-r--r-- 1001/1001 9435 2022-11-13 05:45 info/about.json
-rw-r--r-- 1001/1001 0 2022-11-13 05:45 info/git
-rwxrwxr-x 1001/1001 4051256 2022-11-13 05:45 bin/wfmash
The meta.yaml
is the latest. And the compilation environment has a GCC 10.4.
$ cat pkgs/wfmash-0.10.0-hfdddef0_2/info/recipe/meta.yaml
# This file created by conda-build 3.21.8
# meta.yaml template originally from:
# /opt/recipe, last modified Sat Nov 12 21:36:58 2022
# ------------------------------------------------
package:
name: wfmash
version: 0.10.0
source:
sha256: f7635d306f7b436b837f01fcb11aa503d939915a1b18e601e0f50fc496ab6632
url: https://github.com/waveygang/wfmash/releases/download/v0.10.0/wfmash-v0.10.0.tar.gz
build:
number: '2'
string: hfdddef0_2
Why do you recommend specifying the version for cmake and gcc in the meta.yaml? I see that a few tools specify the version of cmake, but not for gcc. Do you know examples of that?
Sorry, I can't find an example via searching the bioconda repo. I'm not familiar with C/C++ ecosystem, but I've met tools that failed to run due to dependencies with versions too new, like sourmash and some other python packages incompatible to Python 3.10 due to the change of stdlib APIs.
I didn't dive into the mechanism of conda. But now I'm pretty sure it caches some files in their server, including binaries (my tools) and packages supposed to be compiled from the source like wfmash
. It may check and compare the local environment with that of the cache server; if they matched, the cached binary is downloaded to avoid compiling from the source. Since the readme explicitly requires "At least GCC version 9.3.0 is required for the compilation", it would be better to specify the version.
I also tried to recreate the conda env and installed the latest cmake and GCC before installing pggb (with conda --use-local), but wfmash still failed to run.
This might be the way: Building conda packages from scratch.
# create a new conda environment with new cmake and GCC
conda create -n pggb
conda activate pggb
mamba install cmake gcc
# copy the build.sh and meta.yaml file to some where
# https://github.com/bioconda/bioconda-recipes/tree/7ba6e417f2b7b31b84c98641214624f961ab2a49/recipes/wfmash
# install conda-build
mamba install conda-build
# cd where build.sh existes.
conda-biuld .
# remove the old one
mamba remove wfmash
# install the locally compiled one
conda install --use-local wfmash
It did compile from scratch. Unfortunately, it still failed to run:
$ ll $(which wfmash)
-rwxrwxr-x 2 shenwei shenwei 3.7M Dec 2 14:36 /home/shenwei/app/miniconda3/envs/pggb/bin/wfmash
$ wfmash --version
v0.10.0-4-gcf9bfb0
$ wfmash -s 500 -l 25000 -p 90 -n 27 -k 19 -H 0.001 -X -t 16 --tmp-base output akk.fasta --approx-map
Segmentation fault (core dumped)
Looking at your first conda list
output, you have the correct bioconda-version of wfmash
, that is hfdddef0_2
. This does not point to the current version of the master branch on GitHub
because bioconda
takes the buildable tarball from the release here. Not sure if this information helps or if it was obvious to you.
Could you please share more information on your system? I am wondering if your CPU architecture is older than Sandy Bridge. Usually, wfmash
breaks during the alignment due to portability issues, but in your case, it breaks earlier, during the mapping. That is more unusual.
The CPU is Intel Xeon Gold 6248 (Cascade Lake), released in April 2, 2019.
Hi all, thanks for developing these pan-genome graph tools.
wfmash v0.10.0-4-gcf9bfb0
was installed as a dependency ofpggb 0.5.1
via Conda, both in my workstation (manjaro/arch with a relatively new environment) and a server (centos 7.9.2009).I've successfully run
pggb
in the workstation with 28 bacterial genomes, while it failed with the same input and command.Then I directly ran the
wfmash
command, and it reported:I also reran with only two genomes (t.fasta.gz), while the same error occurred. The same command with the same input ran successfully and only take 1.5s in the workstation.
The environment of the server
I noticed that the date of the
wfmash
binary file is Nov 13 , not today (date
: Thu Dec 1 23:17:17 CST 2022) nor the day of the bioconda recipe being updated (Nov 15). How's that possible? I doubt thatconda
did not compile from the source at all following the compiling steps.I also tried to compile from the source but failed both in the workstation and server due to the lack of dependencies or other issues.
I'd recommend specifying the version of
cmake
,gcc
in the conda meta.yaml files ofwfmash
,seqwish
and other tools.It would be wonderful if a statically-linked binary file is released along with the source code. :)
Wei