search

usegalaxy-eu / workflow-testing

Automated testing of workflows against Galaxy
10 stars 21 forks source link

[WIP] test script and venv requirements #55

Open cat-bro opened 4 years ago

cat-bro commented 4 years ago

A script to clone training-material and find workflows with tests then run the tests + non-GTN-workflow tests in this repo.

The virtual environment requires planemo and parsec. There is a conflict installing these requirements and it ends up using bioblend version 0.14.0 which is too new for parsec, but this seems to be fine for the parsec calls used in the script.

@bgruening what code is on Jenkins for this currently? I'd like to make the history.html links and the planemo reports available after the test run. The planemo report is useful if the test has failed because of tool versions, history links are useful for just about everything else.

TODO: Makefile, automatic updating of the readme

hexylena commented 4 years ago

There is a conflict installing these requirements and it ends up using bioblend version 0.14.0 which is too new for parsec, but this seems to be fine for the parsec calls used in the script.

yeah, should be fine, the diff between 0.14.0 and what parsec was pinned to, should be very minimal.

what code is on Jenkins for this currently

pip install planemo==0.62.1
pip install galaxy-parsec
export PLANEMO_OPTIONS="--verbose"
export GALAXY_URL="https://usegalaxy.eu"

bash run_galaxy_workflow_tests.sh $WORKFLOW

this is everything that's in jenkins currently.

And the following glob chooses which files to archive:

**/tool_test_output.html, **/history.html
cat-bro commented 4 years ago

@hexylena thanks for reviewing.

I've taken out a few lines. There will be more that needs to change because I hadn't considered that Jenkins was running a build for each test. So the workflow_list and the test_output folder will both need to be outside of the repo.

hexylena commented 4 years ago

Or we can switch it to running a single build! I used the multi-build strategy only because I wanted the nice pass/fail badges for each WF, but, this looks so much more conceptually simple/correct than the old design (where updating the workflow list was super difficult) that it's definitely worth sacrificing the badges for, we can find another, better solution for those.