Singularity profile - Githubissues

allyssonallan commented 1 week ago

Describe the bug I tried to download the aim-lite via: singularity pull docker://zhandongliulab/aim-lite singularity pull docker://chaozhongliu/aim-lite Unfortunately without success. Also I tried to run with -with-singularity and these .sif and nothing ran... Is there any -profile singularity option, or some workaround to run aim-lite in SLURM?

Expected behavior

./nextflow run Liuzlab/AI_MARRVEL -r nextflow_conversion \
--C $PWD/nextflow.config \
--ref_dir $PWD/AI_MARRVEL/ \
--ref_ver hg38 \
--input_vcf $PWD/S1.deepvariant.vcf.gz \
--input_hpo $PWD/hpo/S1.txt  \
--outdir ./out \
--run_id S1 \
-with-singularity $PWD/aim-lite_latest.sif
-without-docker

Input data vcf - ok (deep variant, freebayes and haplotypecaller tested) hpo - ok (one term below each other) database - ok (download last week) sing cointainer - ok based on aim-lite_latest from docker

Genome build hg38

Screenshots or pipeline output Not applicable

Additional context We are not allowed to run docker in HPC.

DecimalTurn commented 1 week ago

@allyssonallan Yes, this link is indeed part of a malware distribution campaign. Don't download/run what it was suggesting. Related: https://github.com/orgs/community/discussions/136836#discussioncomment-10532718

hyunhwan-bcm commented 1 week ago

@allyssonallan can you share the .nextflow.log? Thanks

allyssonallan commented 1 week ago

Thanks for your help, @DecimalTurn and @hyunhwan-bcm

@allyssonallan can you share the .nextflow.log? Thanks

Sep-02 20:26:58.755 [main] ERROR nextflow.cli.Launcher - @unknown
nextflow.exception.IllegalConfigException: Cannot enable more than one container engine -- Choose either one of: docker, singularity
    at nextflow.Session.memoizedMethodPriv$getContainerConfigString(Session.groovy:1234)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:343)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:328)
    at groovy.lang.MetaClassImpl.doInvokeMethod(MetaClassImpl.java:1333)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1088)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1007)
    at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:645)
    at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:628)
    at org.codehaus.groovy.runtime.InvokerHelper.invokeMethodSafe(InvokerHelper.java:82)
    at nextflow.Session$_closure5.doCall(Session.groovy)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:343)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:328)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:279)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1007)
    at groovy.lang.Closure.call(Closure.java:433)
    at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.lambda$call$0(Memoize.java:137)
    at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:137)
    at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:113)
    at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.call(Memoize.java:136)
    at groovy.lang.Closure.call(Closure.java:422)
    at nextflow.Session.getContainerConfig(Session.groovy)
    at nextflow.Session.getContainerConfig(Session.groovy:1246)
    at nextflow.script.WorkflowMetadata.containerEngine0(WorkflowMetadata.groovy:279)
    at nextflow.script.WorkflowMetadata.<init>(WorkflowMetadata.groovy:263)
    at nextflow.Session.init(Session.groovy:424)
    at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:129)
    at nextflow.cli.CmdRun.run(CmdRun.groovy:372)
    at nextflow.cli.Launcher.run(Launcher.groovy:503)
    at nextflow.cli.Launcher.main(Launcher.groovy:657)

hyunhwan-bcm commented 1 week ago

Thanks for your help, @DecimalTurn and @hyunhwan-bcm

@allyssonallan can you share the .nextflow.log? Thanks

Sep-02 20:26:58.755 [main] ERROR nextflow.cli.Launcher - @unknown
nextflow.exception.IllegalConfigException: Cannot enable more than one container engine -- Choose either one of: docker, singularity
  at nextflow.Session.memoizedMethodPriv$getContainerConfigString(Session.groovy:1234)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.base/java.lang.reflect.Method.invoke(Method.java:567)
  at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:343)
  at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:328)
  at groovy.lang.MetaClassImpl.doInvokeMethod(MetaClassImpl.java:1333)
  at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1088)
  at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1007)
  at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:645)
  at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:628)
  at org.codehaus.groovy.runtime.InvokerHelper.invokeMethodSafe(InvokerHelper.java:82)
  at nextflow.Session$_closure5.doCall(Session.groovy)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.base/java.lang.reflect.Method.invoke(Method.java:567)
  at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:343)
  at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:328)
  at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:279)
  at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1007)
  at groovy.lang.Closure.call(Closure.java:433)
  at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.lambda$call$0(Memoize.java:137)
  at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:137)
  at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:113)
  at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.call(Memoize.java:136)
  at groovy.lang.Closure.call(Closure.java:422)
  at nextflow.Session.getContainerConfig(Session.groovy)
  at nextflow.Session.getContainerConfig(Session.groovy:1246)
  at nextflow.script.WorkflowMetadata.containerEngine0(WorkflowMetadata.groovy:279)
  at nextflow.script.WorkflowMetadata.<init>(WorkflowMetadata.groovy:263)
  at nextflow.Session.init(Session.groovy:424)
  at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:129)
  at nextflow.cli.CmdRun.run(CmdRun.groovy:372)
  at nextflow.cli.Launcher.run(Launcher.groovy:503)
  at nextflow.cli.Launcher.main(Launcher.groovy:657)

Cannot enable more than one container engine -- Choose either one of: docker, singularity

interestingly, it says you are choosing two containers, but I saw you use -without-docker, can you try to once you remove docker.enabled = true in nextflow.config? We are starting to investigate for singularity and slurm. Will update you soon.

Kind regards,

Hwan

allyssonallan commented 1 week ago

Also tried docker.enabled = false, Hwan. Unfortunately no success. I will also work here on my side to see if I am getting some weird results from modules and different nodes.

Thanks for your help, @DecimalTurn and @hyunhwan-bcm

@allyssonallan can you share the .nextflow.log? Thanks

Sep-02 20:26:58.755 [main] ERROR nextflow.cli.Launcher - @unknown
nextflow.exception.IllegalConfigException: Cannot enable more than one container engine -- Choose either one of: docker, singularity
    at nextflow.Session.memoizedMethodPriv$getContainerConfigString(Session.groovy:1234)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:343)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:328)
    at groovy.lang.MetaClassImpl.doInvokeMethod(MetaClassImpl.java:1333)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1088)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1007)
    at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:645)
    at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:628)
    at org.codehaus.groovy.runtime.InvokerHelper.invokeMethodSafe(InvokerHelper.java:82)
    at nextflow.Session$_closure5.doCall(Session.groovy)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:343)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:328)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:279)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1007)
    at groovy.lang.Closure.call(Closure.java:433)
    at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.lambda$call$0(Memoize.java:137)
    at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:137)
    at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:113)
    at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.call(Memoize.java:136)
    at groovy.lang.Closure.call(Closure.java:422)
    at nextflow.Session.getContainerConfig(Session.groovy)
    at nextflow.Session.getContainerConfig(Session.groovy:1246)
    at nextflow.script.WorkflowMetadata.containerEngine0(WorkflowMetadata.groovy:279)
    at nextflow.script.WorkflowMetadata.<init>(WorkflowMetadata.groovy:263)
    at nextflow.Session.init(Session.groovy:424)
    at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:129)
    at nextflow.cli.CmdRun.run(CmdRun.groovy:372)
    at nextflow.cli.Launcher.run(Launcher.groovy:503)
    at nextflow.cli.Launcher.main(Launcher.groovy:657)

Cannot enable more than one container engine -- Choose either one of: docker, singularity

interestingly, it says you are choosing two containers, but I saw you use -without-docker, can you try to once you remove docker.enabled = true in nextflow.config? We are starting to investigate for singularity and slurm. Will update you soon.

Kind regards,

Hwan

hyunhwan-bcm commented 1 week ago

@allyssonallan - for singularity pull docker://chaozhongliu/aim-lite, did you mean that you weren't able to pull the image? I tried my own, and it is working, or you said singularity is not working with the pulled image (i.e., image pulling was successful).

hyunhwan-bcm commented 1 week ago

Hi, I was able to run nextflow with singularity, and here are the log files:

➜  AIM_demo cat .nextflow.log | grep -i singularity
Sep-03 22:51:38.642 [main] DEBUG nextflow.cli.Launcher - $> nextflow run LiuzLab/AI_MARRVEL -r main --ref_dir <redacted> --input_vcf <redacted> --input_hpo <redacted>  --outdir <redacted>  -with-singularity aim-lite_latest.sif -without-docker -with-trace trace_new.txt -with-report report.html -with-timeline timeline.html
Sep-03 22:51:40.868 [main] DEBUG nextflow.config.ConfigBuilder - Enabling execution in Singularity container as requested by command-line option `-with-singularity null`
Sep-03 22:51:44.168 [Actor Thread 47] DEBUG nextflow.container.SingularityCache - Singularity found local store for image=docker://broadinstitute/gatk; path=/home/hwan/AIM_demo/work/singularity/broadinstitute-gatk.img

I guess there might be a problem in your system, can you share with us the full .nextflow.log?

sasidharPasupuleti commented 6 days ago

The issue is being closed since we were able to test singularity on our side successfully. Depending on cluster setup there is a chance the worker nodes are notable to reach internet to pull the image. If possible try to pull image with help of head node. This one of cases I'm assuming that could be an issue.

allyssonallan commented 4 days ago

Thanks @hyunhwan-bcm and @sasidharPasupuleti

That is perfect @sasidharPasupuleti the worker nodes don't reach the internet due to admin issues, now I am trying to allocate nodes and run completely locally. The queue is long, so I have to wait a little. Thanks for your support.

LiuzLab / AI_MARRVEL

Singularity profile #87