CDCgov / phoenix

🔥🐦🔥PHoeNIx: A short-read pipeline for healthcare-associated and antimicrobial resistant pathogens
Apache License 2.0
50 stars 17 forks source link

[BUG] - Undocumented setting docker.userEmulation #169

Closed jfnjdoh closed 1 month ago

jfnjdoh commented 2 months ago

Using: Docker version 26.1.3, build b72abbb Ubuntu 22.04.4 LTS nextflow version 24.04.2.5914

and running nextflow run phoenix/main.nf -profile test,docker -r v2.1.1 -entry PHOENIX --kraken2db ${KDB} where ${KDB} is a valid path, I receive the error

WARN: Undocumented setting `docker.userEmulation` is not supported any more - please remove it from your config
ERROR ~ Error executing process > 'PHOENIX:PHOENIX_EXTERNAL:ASSET_CHECK (1)'

Perhaps this is related to https://github.com/nextflow-io/nextflow/pull/4596

jvhagey commented 2 months ago

What version of docker are you using? It looks like just a warning, but did it kill the whole pipeline from starting?

jfnjdoh commented 2 months ago

Software versions are the first few lines of the original post. It kills the whole pipeline, failing at the PHOENIX:PHOENIX_EXTERNAL:ASSET_CHECK Here's a .nextflow.log file log.txt

slsevilla commented 1 month ago

I ran into the same issue - it's related to nextflow 24.04. I rolled back to 23.10 and the pipeline went on without an issue.

There's a few schema / semantic / variables no longer being used that will eventually have to be addressed, but that'll be a pretty big lift.

jtsteyer93 commented 1 month ago

I wanted to add on that I am also getting this issue, first time trying to run phoenix on my station. Rolling back to nexflow 23.10 fixed the asset part but now I'm getting an error for MLST.

Command: nextflow run cdcgov/phoenix -r v2.0.0 -profile docker,test -entry PHOENIX --kraken2db /home/jts/k2_standard_08gb_20240112

Not sure if I'm missing something in a dependency or just a bug. Thanks!

Screenshot from 2024-05-31 10-52-57

slsevilla commented 1 month ago

Yeah it's not going to work on between 24. There were significant changes made **to nextflow which have not been made to the pipeline.

Where are you doing the work? Personal computer? If so you'll need to uninstall the version you have and install nextflow version 23.X, where x is any number. I know that 23.10 works, but I can't speak to other versions.

jtsteyer93 commented 1 month ago

Yeah, it's a personal computer. at work. I did find a workaround to force the nextflow to be an older version, even if the downloaded version is the most current. This new block is while running v23.10. Thanks for the info for the 23.10 though, I was having such a hard time figuring it out yesterday before finding this thread this morning.

slsevilla commented 1 month ago

what version of phoenix are you running? Looks like the docker container that's being pulled can't detect the flag that is being deployed. I just ran phoenix and didnt see this issue so there may be some disconnect in containers.

if you go into the code go to modules / local / mlst.nf you can get the containerID

process MLST {
    tag "$meta.id"
    label 'process_medium'
    // 2.23.0_07282023 - must edit manually below (line 28)!!!
    container 'quay.io/jvhagey/mlst@sha256:e24cec23ab7300dbb5daf4f4ccd1e4aef9d3befcedae8a14c156cba0bbd952b1'
jvhagey commented 1 month ago

I wanted to add on that I am also getting this issue, first time trying to run phoenix on my station. Rolling back to nexflow 23.10 fixed the asset part but now I'm getting an error for MLST.

Command: nextflow run cdcgov/phoenix -r v2.0.0 -profile docker,test -entry PHOENIX --kraken2db /home/jts/k2_standard_08gb_20240112

Not sure if I'm missing something in a dependency or just a bug. Thanks!

Screenshot from 2024-05-31 10-52-57

Hi, @jtsteyer93 this issue is unrelated to the original post here and is the same as https://github.com/CDCgov/phoenix/issues/115. This is a bug (a singularity argument was hard coded and shouldn't have been so it errors with docker) in v2.0.0 & v2.0.1 so please upgrade >v2.0.2 --> ideally use the latest version 😃

jfnjdoh commented 1 month ago

Turns out the simplest, albeit temporary fix is to to simply prepend a nextflow command with NXF_VER=23.10.1 like so

NXF_VER=23.10.1 nextflow run phoenix/main.nf -profile test,docker -r v2.1.1 -entry PHOENIX --kraken2db ${KDB}
jvhagey commented 1 month ago

@jfnjdoh thanks for finding a temp solution. I am doing some testing to see how to address this without making everyone upgrade nextflow versions.

jtsteyer93 commented 1 month ago

Thanks for the help! I appreciate it.

Sorry for the cross-posting, wasn't sure if it was directly related to the same. I'll look at that issue as well if I continue to have more errors.

jvhagey commented 1 month ago

ok, @jfnjdoh can you try replace L115 with docker.runOptions = '-u $(id -u):$(id -g)' in the nextflow.config and then rerun? Looks like a permissions issue.

jfnjdoh commented 1 month ago

Yeah, that solved it. Thanks @jvhagey

jvhagey commented 1 month ago

great. I will incorporate this in the next release.