search

ComparativeGenomicsToolkit / cactus

Official home of genome aligner based upon notion of Cactus graphs
Other
503 stars 110 forks source link

Error running Cactus on Slurm cluster #1129

Open philge opened 1 year ago

philge commented 1 year ago

Hi,

I am attaching an error am getting while running Cactus on Slurm cluster. Can you please help me to fix the issue.

Thanks Philge cactus_error.txt

glennhickey commented 1 year ago

You're running with --binariesMode docker without having docker installed. 

        FileNotFoundError: [Errno 2] No such file or directory: 'docker'
diekhans commented 1 year ago

It appears you don't have docker installed: FileNotFoundError: [Errno 2] No such file or directory: 'docker'

philge commented 1 year ago

I have docker installed. Still, I get the error. Not sure what is the issue. Should I need to install docker on all nodes?

diekhans commented 1 year ago

Somehow, docker command is not in the PATH when Toil runs; I am not sure how Toil and conda interact. Is the docker command your default PATH?

You might ask on the Toil group: https://github.com/DataBiosphere/toil

Philge Philip @.***> writes:

I have docker installed. Still, I get the error. Not sure what is the issue. image

-- Reply to this email directly or view it on GitHub: https://github.com/ComparativeGenomicsToolkit/cactus/issues/1129#issuecomment-1675730844 You are receiving this because you commented.

Message ID: @.***>

DailyDreaming commented 1 year ago

I'm not sure that this is a Toil issue. I do believe PATH can be overwritten if supplying subprocess.Popen with env vars manually, which cactus seems to do: https://github.com/ComparativeGenomicsToolkit/cactus/blob/master/src/cactus/shared/common.py#L782 . Just a thought. I'd throw some extra debug log statements in that function showing the current env vars, the args supplied to Popen, and maybe a "which docker" to try to see what's going on in the output log and then maybe ask here again if nothing jumps out at you immediately.

adamnovak commented 1 year ago

Should I need to install docker on all nodes?

You definitely need Docker installed on all the nodes in the Slurm cluster for this to work.

philge commented 1 year ago

I'm not sure that this is a Toil issue. I do believe PATH can be overwritten if supplying subprocess.Popen with env vars manually, which cactus seems to do: https://github.com/ComparativeGenomicsToolkit/cactus/blob/master/src/cactus/shared/common.py#L782 . Just a thought. I'd throw some extra debug log statements in that function showing the current env vars, the args supplied to Popen, and maybe a "which docker" to try to see what's going on in the output log and then maybe ask here again if nothing jumps out at you immediately.

Hi, can you please help me with the code I should add to print current env vars and "which docker"?

philge commented 1 year ago

Should I need to install docker on all nodes?

You definitely need Docker installed on all the nodes in the Slurm cluster for this to work.

Hi, we have added Docker installation in custom actions for other compute nodes. But still, we get the same error.