AMR Finder get error

[minhtrung1997]

Describe the bug Sorry for spam the issues, but when I run pipeline on the newly updated HPC, I got a weird error (because I've run the pipeline successfully with the same dataset last week)

*** ERROR ***
  '/opt/conda/bin/blastp'  -query '2212S024__2.faa' -db /tmp/amrfinder.VikgCk/db/AMRProt  -comp_based_stats 0  -evalue 1e-10  -seg no  -max_target_seqs 10000  -num_threads 4  -mt_mode 1 -outfmt '6 qseqid sseqid qstart qend qlen sstart send slen qseq sseq' -out /tmp/amrfinder.VikgCk/blastp > /dev/null 2> /tmp/amrfinder.VikgCk/blastp-err
  status = 34304
  OMP: Error #179: Function Can't open SHM2 failed:
  OMP: System error #13: Permission denied
  Aborted (core dumped)

To Reproduce Please check the log file nextflow_log_bacannot_amrfinder.txt

Expected behavior If it finishes seamlessly on other computer, I think it's a fault from my own, but I still need a recommendation

Server (please complete the following information):

• OS: ubuntu 20.04.4

[fmalmeida]

On this one I unfortunately cannot be of much help. The only thing I can see from it is that you are probably lacking permissions for some shared libraries that are mounted by nextflow when using singularity.

When googling it I saw that some people said that a simple reboot would fix.

On this error specifically I think it is good to talk with the people taking care of your HPC because I think that it is related to some mis configuration.

Anyways, if in the end you find out that some additional singularity parameters are required, I can then help you create a proper singularity config with your required params.

But I am not of much help on trying to identify the mis config mainly because I don’t have access to HPC/singularity environments thus neither much experience with it 🥲

[minhtrung1997]

it sound sad, for even the admin account with full permission could not passed !

[fmalmeida]

So it sounds like something related to singularity itself.

can you try going to the workdir and starting the singularity container there with the used image. You should find the command executed by Nf to start singularity in the file .command.run in the working directory.

The you could try running the tool from there, to see what happens. Then, also trying some singularity configurations to check if any of them solve it. If so, we could then add this configs to the singularity profile.

amrfinder uses the misc image, so you could probably start singularity with:

singularity run docker://fmalmeida/bacannot:v3.2_misc

[fmalmeida]

Hi @minhtrung1997 , I got hands in a machine with singularity and it ran through with the normal defined profiles for singularity. So indeed there either is some misconfig in your singularity setup, or, this new system require some special options to be additionally passed to singularity. For example, -B $PWD.

Could you check if adding this helps? To try more singularity options one can use a custom config:

process {
    singularity.runOptions = '--writable-tmpfs -B $PWD'
}

and pass to the pipeline with -c custom.config.

If we find what solves it I can then add the changes to the pipeline itself.

I have made this -B $PWD option already available in branch 86-amr-finder-get-error if you wanna quickly try using it with:

nextflow run pipeline <other params> -r 86-amr-finder-get-error -latest

[minhtrung1997]

Yah, thanks, I've fixed the problem with 1-2 commandline to HPC (using ansible) 1: Config the path bind:

 ansible.builtin.shell:
      cmd: /opt//singularity-3.10.5//singularity config global --set "bind path" /etc/resolv.conf
      cmd: /opt//singularity-3.10.5//singularity config global --set "bind path" /scratch

2: Give permission to the /dev/shm in each node sudo chmod 777 /dev/shm Ta-da

[fmalmeida]

Awesome! 😬