search

4dn-dcic / tibanna

Tibanna helps you run your genomic pipelines on Amazon cloud (AWS). It is used by the 4DN DCIC (4D Nucleome Data Coordination and Integration Center) to process data. Tibanna supports CWL/WDL (w/ docker), Snakemake (w/ conda) and custom Docker/shell command.
MIT License
70 stars 28 forks source link

Step functions fail for snakemake rules #358

Closed nate-d-olson closed 2 years ago

nate-d-olson commented 2 years ago

My step-functions regularly fail 5 minutes after the jobs start but the instances run to completion and the rule output files are successfully transferred to the S3 bucket. The only error I see in the log files is associated with dowloading goofys-latest. 

curl: (35) gnutls_handshake() failed: Error in the pull function.
chmod: cannot access 'goofys-latest': No such file or directory

We have to define the security group and subnet as we don't have a default VPC. Could this error be due to our security group or subnet settings?

nate-d-olson commented 2 years ago

I looked into this issue and it appears that the default DNS server used by this version of tibanna is blocked by my institution's firewall. I was able to develop a workaround but the docker-based tibanna and modifying the /etc/resolv.conf using the snakemake --precommand argument. I will close this issue but can provide additional information if it will help other users or the tibanna development team.