Run RECAST hello world example on REANA

[matthewfeickert]

Following up on Issue #1, now that there is a Snakemake based workflow for the RECAST hello world example, we should additionally add a workflow and instructions to be able to run this workflow on REANA.

This would include:

• [x] Extending the environemnt.yml for to add REANA's dependencies (or better is to create a new reana-envrionemnet.yml that has them)
  • This would be https://github.com/reanahub/reana-client
• [x] Adding a script to run the workflow from inside the environment.
  • This can be done manually for now and doesn't need to be added to CI.
• [x] Adding instructions to a Markdown file on how to run this.

[AndriiPovsten]

@matthewfeickert for running a Snakemake workflow as I understand from this example: [(https://blog.reana.io/posts/2021/support-for-running-snakemake-workflows/)] I adittionaly should create a reana-snakemake.yaml file and after see the output from reana-client validate -f reana-snakemake.yaml --environments --pull:? Something like this:

version: 0.8.0
inputs:
  files:
    - data/input.txt
    - results/helloworld_output.txt
  directories:
    - recast_helloworld/snakemake
  parameters:

    input: recast_helloworld/snakemake/input.txt

workflow:

  type: snakemake

  file: recast_helloworld/Snakefile

outputs:
  files:
    - results/example_output.txt

Because just reana-environment.yml shouldn't be enough for running Snakefile on REANA?:

name: Snakemake-reana-helloworld-example
channels:
  - conda-forge
dependencies:
  - python=3.11
  - bioconda::snakemake=7.25.0
  - pip
  - pytest
  - pandas
  - samtools
  - matplotlib
  - ruff
  - pip:
    - reana-client

[matthewfeickert]

Given our call today the recommendations I have for moving forward are:

• [x] Open up a PR with the work you currently have for this Issue so that we can reference the code directly on GitHub and have an easier time comparing it using the GitHub tools.
  • PR #16
• [x] Add both the exact commands that you're running and the error messages you're getting as a comment on that PR, which is very useful for debugging.
• [x] Follow the 2021-11-30 Support for running Snakemake workflows blog post and try to reproduce the blog post and make sure that everything demoed there (which I think includes code from https://github.com/reanahub/reana-workflow-engine-snakemake and https://github.com/reanahub/reana-demo-root6-roofit) works.
  • If you can reproduce the blog post without errors:
    • [ ] Compare the REANA and Snakemake configuration files from the working blog post to the not working files you have now.
  • If you get errors reproducing the blog post:
    • [ ] Open a new issue on https://github.com/reanahub/reana-workflow-engine-snakemake/issues/ (as recommended in the blog post) that details: what you did, what errors you got, and what you expected to have happened. Tag me in the Issue as well so that I can track it.

[matthewfeickert]

1. So we already have a PR that is most of the way there with PR #16. :+1: So we can focus on having that get cleaned up and merged into main as the top proiroity for this week.
   • We also noted that when running in the logs we get a note of

     conda environments: ignored
     Singularity containers: ignored

which seems to indicate that these aren't getting picked up by Snakemake at runtime. This should get figured out, and see if this is due to options not being passed at the CLI API or if this is due to some configuration issue. Probably easiest to start by googling on these commments.

3. After that is merged in, then we can look at having @AndriiPovsten resolve any merge conflicts that exit in PR #17, which is going to get renamed and rescoped to addd the use instruction so the README.
4. After both of these are merged into main, then I will add an additional example into Issue #15 and we can go from there.

As the project ends 2023-09-22 we should stop "the work" with finishing Issue #15 and we can then spend the rest of the time making sure the documentation is clear as well as working on preparing a very nice presentaiton to show off the work that has been done.

[AndriiPovsten]

@matthewfeickert I suppose that the problem is due to the missing singularity package. I was trying to install it but received such message:

conda install -c conda-forge singularity
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - singularity

Current channels:

  - https://conda.anaconda.org/conda-forge/osx-64
  - https://conda.anaconda.org/conda-forge/noarch
  - https://repo.anaconda.com/pkgs/main/osx-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/osx-64
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

But singularity is installed via the conda-forge channel as said here: singularity. Do you know any other ways to install it? Just in case I'll try to add in the environment.file

[AndriiPovsten]

Close #14 After this issue, we have:

• Reana-environment.yml file
• Read.me instructions for running with reana-client
• reana-snakemake.yaml file
• modified Snakefile and input.yml file