c-scale-community / workflow-coastal-hydrowaq

Porting and deploying the HiSea use case on C-SCALE
Apache License 2.0
3 stars 1 forks source link

Snakemake draft #41

Closed sebastian-luna-valero closed 1 year ago

sebastian-luna-valero commented 1 year ago

I was playing a bit with https://github.com/c-scale-community/c-scale-tutorial-snakemake

Then decided to create this draft for a snakemake workflow for HiSea.

Regarding https://github.com/c-scale-community/use-case-hisea/issues/40, Snakemake can also help with job submission to SLURM: https://snakemake.readthedocs.io/en/stable/executing/cluster.html

sebastian-luna-valero commented 1 year ago

Sorry, forgot to add instructions for testing:

# install snakemake
mkdir working-dir
cd working-dir
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh -b -p conda-install
source conda-install/etc/profile.d/conda.sh
conda install mamba -c conda-forge --yes
mamba create -c bioconda -c conda-forge -n snakemake snakemake-minimal --yes
conda activate snakemake

# run workflow
git clone https://github.com/c-scale-community/use-case-hisea.git
cd use-case-hisea/
git checkout --track origin/snakemake
snakemake -c1 pre_processing_boundary --use-conda --conda-frontend mamba
sebastian-luna-valero commented 1 year ago

Fixed now the snakemake rule for pre_processing_boundary

Note that I had to manually execute:

mkdir -p data/output
mkdir -p data/output/DFMWAQ_input/
sebastian-luna-valero commented 1 year ago

@backeb @lorincmeszaros I fixed the issue with pre_processing_boundary.

I think the best next step is for us three to have a dedicated meeting to move this forward with new steps for the workflow (if you still want to go down the snakemake route)

backeb commented 1 year ago

I've forwarded the meeting for Friday morning, where Lőrinc and I are planning to work on this.