jtamames / SqueezeMeta

A complete pipeline for metagenomic analysis
GNU General Public License v3.0
346 stars 81 forks source link

Issue with step 10 when submitting job on a condor server #772

Closed mwthairu closed 4 months ago

mwthairu commented 5 months ago

Hello, I am running Squeezemeta on a condor server for the first time my runs crash at step 10. I am not sure what is happening. This is my starting command: SqueezeMeta.pl -m sequential -s BW4XX_squeeze/$ID.txt -f fastq_files/ -a spades -assembly_options "--meta" -t 12 -b 15

This the error I get: Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 260. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 260. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 260. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 262. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 266. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 266. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 266. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 271. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 271. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 271. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 271. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 271. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 271. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 271. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 1038. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 1040. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 1045. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 1076. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 1076. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 1076. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 1077. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 1077. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 1077. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 1179. Global symbol "$ENVDIR" requires explicit package name (did you forget to declare "my $ENVDIR"?) at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config_heavy.pl line 1291. Compilation failed in require at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/lib/5.26.2//x86_64-linux-thread-multi/Config.pm line 105. An error occurred during mapping with command /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/SqueezeMeta/bin/bowtie2/bowtie2 -x /var/lib/condor/execute/slot3/dir_1079599/BW4568/data/BW4568.bowtie -1 /var/lib/condor/execute/slot3/dir_1079599/BW4568/temp/BW4568.BW4568.current_1.gz -2 /var/lib/condor/execute/slot3/dir_1079599/BW4568/temp/BW4568.BW4568.current_2.gz --quiet -p 12 -S /var/lib/condor/execute/slot3/dir_1079599/BW4568/data/bam/BW4568.BW4568.sam --very-sensitive-local! at /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/SqueezeMeta/scripts/10.mapsamples.pl line 267, line 0.000000.

I have tried Initially I was using the variable $ENVDIR to activate my SqueezeMeta conda env, but I changed the variable name and still got this error.

I am not sure I how to fix this or what direction go to try and fix it.

I have also attached a syslog from one of my runs syslog.log

fpusan commented 5 months ago

What SqueezeMeta version are you using? Was it installed through conda?

mwthairu commented 5 months ago

v1.6.3, September 2023 and yes I used conda

fpusan commented 5 months ago

There are several options here.

The error seems to be related to a perl library that afaik shouldn't be asking for an $ENVDIR variable. Maybe this is related to condor (no idea, since I am not familiar with it). I would start by discussing this with the administrator.

Alternatively, you can use bwa instead of bowtie for mapping (-map bwa when launching SqueezeMeta). Hopefully that doesn't fail.

Finally, you can try to install bowtie from conda, and see if that somehow works. For that you would need to: 1) Install bowtie2 inside conda conda install bioconda::bowtie2 2) Locate where SqueezeMeta is installed in your cluster. Your syslog file points to /var/lib/condor/execute/slot3/dir_1079599/SqueezeMeta/SqueezeMeta but I suspect this is some sort of temporary directory (not familiar with condor servers). 3) Make SqueezeMeta use conda's bowtie2 instead of the vendored one. For this, edit the SqueezeMeta_conf.pl file inside the SqueezeMeta/scripts directory. Modify the $bowtie_build_soft and $bowtie_x_soft lines so they read like:

$bowtie2_build_soft = "bowtie2-build";
$bowtie2_x_soft     = "bowtie2";

and start a new project (maybe with the test dataset so you can check faster)

fpusan commented 4 months ago

Closing due to lack of activity, feel free to reopen