search

snakemake / snakemake-github-action

A Github action for running a Snakemake workflow
MIT License
53 stars 12 forks source link

Flag --use-singularity #9

Open marrip opened 3 years ago

marrip commented 3 years ago

Hey,

I first want to say that I really appreciate that you put these actions together - it makes things so much easier to have workflows continuously linted and tested. I was just wondering if you intend to make it possible to use singularity-containers in the testing process. I know this might not be super trivial but really helpful! Let me know what you think. 😃

marrip commented 3 years ago

I came back to this issue after some time and digged a bit deeper. It seems that in order to get singularity in docker running, docker needs to get started as --privileged. I am not very familiar with github actions yet and if it even allows for these options to be set but I tested it locally on my Mac and it works for a simply "Hello world" example without a snakemake workflow. Not sure if you have time to take a look at it.

nikostr commented 3 years ago

Have you made any additional progress, @marrip ?:)

marrip commented 3 years ago

Hey, actually no, I did some research and it seems like it's not possible to run singularity inside docker as part of a github action. At least not for now. I assume github has to change the action configuration to allow for the --privileged flag. Or if someone finds another solution I would be happy to hear about it. 😊

nikostr commented 3 years ago

My workaround was basing my action on the conda action and working from there:

https://github.com/nikostr/dna-seq-deepvariant-glnexus-variant-calling/blob/main/.github/workflows/python-package-conda.yml

So, instead of relying on the snakemake docker image, I set up an environment in ubuntu-latest myself. That way I'm able to run both conda and singularity.

marrip commented 3 years ago

That looks very promising, I will try to use that as well! Cheers!

nikostr commented 3 years ago

Note that I've set the tests up using pytest, based on the (somewhat buggy) snakemake --generate-unit-tests. I adapted that to the full pipeline as well. If you're just interested in running snakemake, and seeing that the pipeline works, you should be able to replace the pytest stuff with just snakemake followed by whatever args you want.

marrip commented 3 years ago

It works! It's awesome, I simply run my snakemake command with the appropriate flags and it pulls the images and runs the workflow on my test data set. I am really happy right now 😄 Big thank you!