Run tool tests for all the tools

[afgane]

Instructions from Martin (https://gist.github.com/martenson/1eca8774ef9ddd47e013258eb14663a7):

export GALAXY_TEST_DEFAULT_WAIT=1800
pip install --index-url https://wheels.galaxyproject.org/simple --extra-index-url https://pypi.python.org/simple --upgrade https://github.com/galaxyproject/planemo/archive/master.zip
pip install --upgrade https://github.com/mvdbeek/ephemeris/archive/add_pysam_continue_if_test_fetching_errors.zip
get-tool-list -g https://usegalaxy.org/ -o test.galaxy.yml --skip_changeset_revision
shed-tools test -a $API_KEY -g https://usegalaxy.eu/ --parallel_tests 10 -t test.galaxy.yml
planemo test_reports tool_test_output.json --test_output_xunit tool_test_output.xunit --test_output tool_test_output.html

[nuwang]

History containing results so far: https://3.208.202.35/galaxy/u/nuwan/h/testhistory

[nuwang]

A couple of fixes suggested by @mvdbeek that reduced failures. Need to be done prior to running the tests.

pip install pysam
pip install --upgrade galaxy-tool-util>=20.5.0.dev1

[nuwang]

export GALAXY_TEST_DEFAULT_WAIT=3600 pip install planemo ephemeris pysam get-tool-list -g https://usegalaxy.org/ -o test.galaxy.yml --skip_changeset_revision shed-tools test -a $API_KEY -g https://usegalaxy.eu/ --parallel_tests 10 -t test.galaxy.yml planemo test_reports tool_test_output.json --test_output_xunit tool_test_output.xunit --test_output tool_test_output.html

[almahmoud]

First try failed due to: OSError: [Errno 24] Too many open files I added ulimit -n 4096 and so far has not failed yet. However, ephemeris should probably be optimized to not have thousands of files open at once, I'm guessing that's a bug.

Small snippet of lsof 2>/dev/null | grep ubuntu | sort

shed-tool 11126 11143           ubuntu 1367u      REG                8,1       47      34440 /tmp/tmpzvhs83bdmultiCovBed_result1.bed (deleted)
shed-tool 11126 11143           ubuntu 1368u      REG                8,1      259      34448 /tmp/tmp750h1vdqnucBed_result2.bed (deleted)
shed-tool 11126 11143           ubuntu 1369u      REG                8,1       47      34442 /tmp/tmpsglw1g2xmultiCovBed_result1.bed
shed-tool 11126 11143           ubuntu 1370u      REG                8,1      161      34446 /tmp/tmpfotfnew7nucBed_result1.bed
shed-tool 11126 11143           ubuntu 1371u      REG                8,1      259      34450 /tmp/tmpip5lan0gnucBed_result2.bed
shed-tool 11126 11143           ubuntu 1372u      REG                8,1      280      34452 /tmp/tmp7b58k0nhnucBed_result3.bed (deleted)
shed-tool 11126 11143           ubuntu 1373u      REG                8,1      280      34454 /tmp/tmpn5kbmbx6nucBed_result3.bed
shed-tool 11126 11143           ubuntu 1374u      REG                8,1       26      34460 /tmp/tmp7quq5qiisubtractBed_result2.bed (deleted)
shed-tool 11126 11143           ubuntu 1375u      REG                8,1       13      34464 /tmp/tmpy_xh9zrksubtractBed_result3.bed (deleted)
shed-tool 11126 11143           ubuntu 1376u      REG                8,1       26      34462 /tmp/tmph4t2pdorsubtractBed_result2.bed
shed-tool 11126 11143           ubuntu 1377u      REG                8,1      412      34468 /tmp/tmp9c8u2cipbamToBed_result1.bed (deleted)
shed-tool 11126 11143           ubuntu 1378u      REG                8,1       26      34458 /tmp/tmpxj_5dtzusubtractBed_result1.bed
shed-tool 11126 11143           ubuntu 1379u      REG                8,1      412      34470 /tmp/tmpu4s4z_grbamToBed_result1.bed

Count (so far, ~1 hour in), i.e. lsof 2>/dev/null | grep ubuntu | cut -f 1 -d ' ' | sort | uniq -c:

      4 (sd-pam
     58 bash
     22 cut
     25 grep
     33 lsof
     36 screen
  16185 shed-tool
     23 sort
      4 sshd
     60 systemd
     22 uniq

Seems like it's almost never closing files after opening them.