Closed mvdbeek closed 4 years ago
wm75
commented
4 years ago What about all these errors like:
tool_script.sh: line 9: /tmp/tmpb9_yhhm8/job_working_directory/000/26/COMMAND_VERSION: Read-only file system
Looks like a lot of the above tools are failing with this.
mvdbeek
commented
4 years ago That's something we're working on (the COMMAND_VERSION file doesn't get written in the correct location), but shouldn't be the cause of the error. If you look for instance at the htseq_count error you have:
/tmp/tmpsv44thj7/job_working_directory/000/19/tool_script.sh: line 9: /tmp/tmpsv44thj7/job_working_directory/000/19/COMMAND_VERSION: Read-only file system
[Errno 2] No such file or directory: '/tmp/tmpsv44thj7/tmp/26_tmp.sam'
[Exception type: FileNotFoundError, raised in count.py:52]
[E::hts_open_format] Failed to open file /tmp/tmpsv44thj7/tmp/26_tmp.sam
samtools view: failed to open "/tmp/tmpsv44thj7/tmp/26_tmp.sam" for reading: No such file or directorySo the true error is that something is going wrong with the input sam file.
mvdbeek
commented
4 years ago wm75
commented
4 years ago Good to know you're fixing this already :-) Look at the failing raceid tools for examples where I think the command version is the only reason for failure.
bernt-matthias
commented
4 years ago Hi, will try to take care of stacks(2) samtools and maxbin2.
PR: Should be done against iuc-repo? Will IUC testing then be in the simple container?
mvdbeek
commented
4 years ago Yeah, let's test against iuc repo, but no, we don't test against the container yet, I'll see what I can do about https://github.com/galaxyproject/tools-iuc/pull/2677 (it should be as simple as that, currently working on the Galaxy PR). If you see and understand any of the errors in the test report feel free to starting PR'ing ... it'll probably still take until Monday or Tuesday for getting things merged into 19.09 / planemo / ... given that it's thanksgiving weekend in the US.
mvdbeek
commented
4 years ago @wm75 after fixing COMMAND_VERSION being mounted in ro I see this:
File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/case.py", line 59, in testPartExecutor
yield
File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/case.py", line 628, in run
testMethod()
File "/Users/mvandenb/src/galaxy/test/functional/test_toolbox.py", line 99, in test_tool
self.do_it(tool_version=tool_version, test_index=test_index)
File "/Users/mvandenb/src/galaxy/test/functional/test_toolbox.py", line 36, in do_it
verify_tool(tool_id, self.galaxy_interactor, resource_parameters=resource_parameters, test_index=test_index, tool_version=tool_version, register_job_data=register_job_data)
File "/Users/mvandenb/src/galaxy/lib/galaxy/tool_util/verify/interactor.py", line 789, in verify_tool
raise e
File "/Users/mvandenb/src/galaxy/lib/galaxy/tool_util/verify/interactor.py", line 785, in verify_tool
job_stdio = _verify_outputs(testdef, test_history, jobs, tool_id, data_list, data_collection_list, galaxy_interactor, quiet=quiet)
File "/Users/mvandenb/src/galaxy/lib/galaxy/tool_util/verify/interactor.py", line 1006, in _verify_outputs
raise JobOutputsError(found_exceptions, job_stdio)
Job in error state.
Job in error state.
Job in error state.
Job in error state.
Job in error state.
-------------------- >> begin captured stdout << ---------------------
Problem in history with id 2891970512fa2d5a - summary of history's datasets and jobs below.
--------------------------------------
| 1 - matrix.filter.rdat (HID - NAME)
| Dataset State:
| ok
| Dataset Blurb:
| 7.9 MB
| Dataset Info:
| The uploaded binary file format cannot be determined automatically, please set the file 'Type' manually
| Peek:
| <table cellspacing="0" cellpadding="3"><tr><td>binary data</td></tr></table>
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 2 - Clustering using RaceID on data 1: PDF Report (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 3 - Clustering using RaceID on data 1: RDS (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 4 - Clustering using RaceID on data 1: Cell-to-Cluster assignments (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 5 - Clustering using RaceID on data 1: Cluster - Genes per Cluster (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 6 - Clustering using RaceID on data 1: Log (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| Job 5729865256bc2525
| State:
| error
| Update Time:
| 2019-11-28T14:22:22.804340
| Create Time:
| 2019-11-28T14:22:09.907655
|
--------------------------------------
| Job 2891970512fa2d5a
| State:
| ok
| Update Time:
| 2019-11-28T14:22:08.041991
| Create Time:
| 2019-11-28T14:22:02.955169
|
--------------------------------------
Problem in history with id 2891970512fa2d5a - summary of history's datasets and jobs below.
--------------------------------------
| 1 - matrix.filter.rdat (HID - NAME)
| Dataset State:
| ok
| Dataset Blurb:
| 7.9 MB
| Dataset Info:
| The uploaded binary file format cannot be determined automatically, please set the file 'Type' manually
| Peek:
| <table cellspacing="0" cellpadding="3"><tr><td>binary data</td></tr></table>
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 2 - Clustering using RaceID on data 1: PDF Report (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 3 - Clustering using RaceID on data 1: RDS (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 4 - Clustering using RaceID on data 1: Cell-to-Cluster assignments (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 5 - Clustering using RaceID on data 1: Cluster - Genes per Cluster (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 6 - Clustering using RaceID on data 1: Log (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| Job 5729865256bc2525
| State:
| error
| Update Time:
| 2019-11-28T14:22:22.804340
| Create Time:
| 2019-11-28T14:22:09.907655
|
--------------------------------------
| Job 2891970512fa2d5a
| State:
| ok
| Update Time:
| 2019-11-28T14:22:08.041991
| Create Time:
| 2019-11-28T14:22:02.955169
|
--------------------------------------
Problem in history with id 2891970512fa2d5a - summary of history's datasets and jobs below.
--------------------------------------
| 1 - matrix.filter.rdat (HID - NAME)
| Dataset State:
| ok
| Dataset Blurb:
| 7.9 MB
| Dataset Info:
| The uploaded binary file format cannot be determined automatically, please set the file 'Type' manually
| Peek:
| <table cellspacing="0" cellpadding="3"><tr><td>binary data</td></tr></table>
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 2 - Clustering using RaceID on data 1: PDF Report (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 3 - Clustering using RaceID on data 1: RDS (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 4 - Clustering using RaceID on data 1: Cell-to-Cluster assignments (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 5 - Clustering using RaceID on data 1: Cluster - Genes per Cluster (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 6 - Clustering using RaceID on data 1: Log (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| Job 5729865256bc2525
| State:
| error
| Update Time:
| 2019-11-28T14:22:22.804340
| Create Time:
| 2019-11-28T14:22:09.907655
|
--------------------------------------
| Job 2891970512fa2d5a
| State:
| ok
| Update Time:
| 2019-11-28T14:22:08.041991
| Create Time:
| 2019-11-28T14:22:02.955169
|
--------------------------------------
Problem in history with id 2891970512fa2d5a - summary of history's datasets and jobs below.
--------------------------------------
| 1 - matrix.filter.rdat (HID - NAME)
| Dataset State:
| ok
| Dataset Blurb:
| 7.9 MB
| Dataset Info:
| The uploaded binary file format cannot be determined automatically, please set the file 'Type' manually
| Peek:
| <table cellspacing="0" cellpadding="3"><tr><td>binary data</td></tr></table>
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 2 - Clustering using RaceID on data 1: PDF Report (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 3 - Clustering using RaceID on data 1: RDS (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 4 - Clustering using RaceID on data 1: Cell-to-Cluster assignments (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 5 - Clustering using RaceID on data 1: Cluster - Genes per Cluster (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 6 - Clustering using RaceID on data 1: Log (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| Job 5729865256bc2525
| State:
| error
| Update Time:
| 2019-11-28T14:22:22.804340
| Create Time:
| 2019-11-28T14:22:09.907655
|
--------------------------------------
| Job 2891970512fa2d5a
| State:
| ok
| Update Time:
| 2019-11-28T14:22:08.041991
| Create Time:
| 2019-11-28T14:22:02.955169
|
--------------------------------------
Problem in history with id 2891970512fa2d5a - summary of history's datasets and jobs below.
--------------------------------------
| 1 - matrix.filter.rdat (HID - NAME)
| Dataset State:
| ok
| Dataset Blurb:
| 7.9 MB
| Dataset Info:
| The uploaded binary file format cannot be determined automatically, please set the file 'Type' manually
| Peek:
| <table cellspacing="0" cellpadding="3"><tr><td>binary data</td></tr></table>
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 2 - Clustering using RaceID on data 1: PDF Report (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 3 - Clustering using RaceID on data 1: RDS (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 4 - Clustering using RaceID on data 1: Cell-to-Cluster assignments (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 5 - Clustering using RaceID on data 1: Cluster - Genes per Cluster (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| 6 - Clustering using RaceID on data 1: Log (HID - NAME)
| Dataset State:
| error
| Dataset Blurb:
| error
| Dataset Info:
| *Dataset info is empty.*
| Peek:
| *Peek unavilable.*
| Dataset Job Standard Output:
| *Standard output was empty.*
| Dataset Job Standard Error:
| *Standard error was empty.*
|
--------------------------------------
| Job 5729865256bc2525
| State:
| error
| Update Time:
| 2019-11-28T14:22:22.804340
| Create Time:
| 2019-11-28T14:22:09.907655
|
--------------------------------------
| Job 2891970512fa2d5a
| State:
| ok
| Update Time:
| 2019-11-28T14:22:08.041991
| Create Time:
| 2019-11-28T14:22:02.955169
|
--------------------------------------
--------------------- >> end captured stdout << ----------------------we may need to specify the filetype on upload ?
bgruening
commented
4 years ago One error-class that we should look out for is that all Java based tools that do some plotting might miss some fonts in the container. Keep an eye on this.
mvdbeek
commented
4 years ago Is there some workaround for missing fonts ? A conda package maybe, or a new base container ?
bgruening
commented
4 years ago Yes, but we need to understand the problem at first. For conda-forge we now have: fonts-conda-ecosystem ... but no clue if this fixes our problem. Just something to look out for and ping me if you see it ... goenrichment seems to have this exact problem.
mvdbeek
commented
4 years ago Oh, and I forgot, the tests run on python 3.7, so there may be another class of errors we may hit in the cheetah templates.
bgruening
commented
4 years ago All or nothing! :tada:
mvdbeek
commented
4 years ago For raceid I had to check the actual output file as there is a stderr redirect:
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/usr/local/lib/R/library/rgl/libs/rgl.so':
libGL.so.1: cannot open shared object file: No such file or directory
Warning: namespace 'RaceID' is not available and has been replacedThat's of course a problem for debugging things remotely. The issue itself seems difficult / not possible to fix with conda-only means: https://github.com/conda-forge/pygridgen-feedstock/issues/10#issuecomment-365983589
We may need to provide an alternative docker base image for these cases
bernt-matthias
commented
4 years ago Added dada2 to my list and documented the problems in the original list. Some seem real bugs (samtools and the dada_fetcher). Maxbin 2 just stumbles over tar -x.
For dada2 its probably because it uses Rdata files which might be compressed differently on the minimal system and therefore it results in a diff.
I have no idea about why the old stacks tools do not find any of the binaries. Might this be a (temporary) problem with conda?
For stacks2 there are minor differences in the test data (I will check if we can/should allow them).
bernt-matthias
commented
4 years ago A strange but maybe easy one is qiime
'getpwuid(): uid not found: 1001'
Also odd: seems to use python2.7
mvdbeek
commented
4 years ago
- Should we just allow larger diff (essentially just test for file presence) or
I think that's reasonable, unless there's a way we can assert something about what's inside the rdata file.
mvdbeek
commented
4 years ago A strange but maybe easy one is qiime
'getpwuid(): uid not found: 1001'
This is old matplotlib trying to create it's own tempdir (in a non-standard way) for config files. I think we may be able to fix this by settingMPLCONFIGDIR ... or perhaps this could be fixed by specifying a newer profile version where galaxy will create a new, writable home dir.
mvdbeek
commented
4 years ago or perhaps this could be fixed by specifying a newer profile version where galaxy will create a new, writable home dir.
phew, yep, that seems to work, but has unearthed that Galaxy doesn't mount the job home directory in rw. So adding profile="18.01" is enough on the tool side and https://github.com/galaxyproject/galaxy/pull/9051 should fix it on the Galaxy side
mvdbeek
commented
4 years ago Similar issue with scanpy, which does the following:
Sevag Hanssian @sevagh Jul 14 2018 15:00 UTC
If true, cache enables a file-based cache to shorten compilation times when the function was already compiled in a previous invocation. The cache is maintained in the pycache subdirectory of the directory containing the source file; if the current user is not allowed to write to it, though, it falls back to a platform-specific user-wide cache directory (such as $HOME/.cache/numba on Unix platforms).so $HOME isn't writable
bernt-matthias
commented
4 years ago bax2bam: mv: can't remove '/tmp/tmpzvg40cuz/job_working_directory/000/2/tmpnuxq8_9j': Read-only file system As far as I can see this mv comes from Galaxy.
mvdbeek
commented
4 years ago I reran the tests yesterday night with the fixes merged so far, we're down to 409 failing tests (was 484) -- https://mvdbeek.github.io/iuc-tool-test-results/
bernt-matthias
commented
4 years ago graphlan seems to be caused by a conda conflict:
Package matplotlib conflicts for:
graphlan=1.0.0 -> matplotlib[version='1.4.*|1.5.*']
Package python conflicts for:
graphlan=1.0.0 -> python=2.7
Package biopython conflicts for:
graphlan=1.0.0 -> biopython=1.66ruvseq is caused by a symlinked R script (links to deseq2) which is probably not copied correctly
mvdbeek
commented
4 years ago ruvseq is caused by a symlinked R script (links to deseq2) which is probably not copied
that's unfortunate, but maybe highlights that we should probably make a proper package out of these scripts (or put them in a macro that we can share through symlinks)
mvdbeek
commented
4 years ago With the latest fixes in 19.09 / master I think running planemo test --biocontainers should give us the same results as what is run in the github workflow (except some non-fatal version command issue noted by @wm75 in https://github.com/galaxyproject/tools-iuc/issues/2701#issuecomment-559472441).
bernt-matthias
commented
4 years ago For transtermhp: command line is transtermhp.py' $TRANSTERMHP genomeref.fa gfffile seemingly $TRANSTERMHP is empty, but should be set actually during activation of the conda env. Works locally for me with planemo t, but not with planemo test --biocontainers. So I guess activation scripts in /etc/conda/activate.d/ do not run..?
mvdbeek
commented
4 years ago Yeah, we assume scripts are on PATH
bernt-matthias
commented
4 years ago In this case $TRANSTERMHP is an environment valiable pointing to the location of some test data.
mvdbeek
commented
4 years ago Yeah, conda is stripped from the image, so the activate script can't run. We could possibly be smarter about the image build process and figure out the env variables set by Conda. In the meantime $(dirname $(command -v transterm))/../data/expterm.dat should work as a $TRANSTERMHP fallback
mvdbeek
commented
4 years ago
- this tool requires a chromosome length file as input, so not sure how to test.
@gregvonkuster is that within a data table ? https://github.com/jmchilton/picard/commit/4df8974384081ee1bb0f97e1bb8d7f935ba09d73 is an example PR that adds this
mvdbeek
commented
4 years ago Alright, triggered another test at https://github.com/mvdbeek/tools-iuc/commit/ed6617a3ee508ddf39ce7bc1af334714d4d45e90/checks?check_suite_id=343542706, let's see how far we're getting now.
mvdbeek
commented
4 years ago Down to 335 failing tests, and a lot of them are actually fixed but due to a bug in Galaxy we're often not getting the latest container version, fixing that now.
mvdbeek
commented
4 years ago Down to 239 failing tests!
mvdbeek
commented
4 years ago It seems that some of the ebi APIs have changed and it isn't immediately obvious what those are. If someone uses or knows this API it would be awesome if they could fix the ebi_metagenomics and ebi_search_rest_results tools.
I also can't get qiime to write proper biom format files, they all end up corrupted when containerized, but it seems they do work in the travis environment outside of biocontainers.
The good news is that we're down to 123 failing tests or so!
hepcat72
commented
4 years ago Is this why I get the error ModuleNotFoundError: No module named 'Bio' from ncbi_eutils_esearch when I try to run it via planemo serve --galaxy_python_version 3.7? Or is that a separate issue? I just tried a test via planemo to reproduce the error I was getting before trying to work on the issue I'd just created (#3111).
peterjc
commented
4 years ago @hepcat72 If you're getting No module named 'Bio' from a Python script, it looks like it is missing Biopython as a dependency (and that's a separate problem to #3111 which I've just commented on).
hepcat72
commented
4 years ago Yeah, google was telling me the same thing. But is that missing dependency an issue associated with this issue we're in here (i.e. an undeclared dependency).
mvdbeek
commented
4 years ago All of the tools that can be fixed are fixed now, so I think we can close this. If you want to check if a tool is currently passing you can check https://github.com/galaxyproject/tools-iuc/actions?query=workflow%3A%22Weekly+global+Tool+Linting+and+Tests%22 which is run weekly.
bernt-matthias
commented
4 years ago Fixed currently failing bwameth https://github.com/bioconda/bioconda-recipes/pull/23440
bernt-matthias
commented
4 years ago bernt-matthias
commented
4 years ago 
jaredbernard
commented
3 years ago Every time I use maker, I get a similar error to the kraken error above. Here's my error message:
Possible unintended interpolation of @2 in string at /cvmfs/main.galaxyproject.org/deps/_conda/envs/__maker@2.31.11/bin/…/lib/5.26.2/x86_64-linux-thread-multi/Config_heavy.pl line 260.
Possible precedence issue with control flow operator at /cvmfs/main.Can you advise how to fix this bug?
Hello IUC,
Tl;DR:
Help us fix tool tests!
we would like to make sure that our tools are well tested and working in the widest range of deployment scenarios possible. And as you may have noticed, tool dependency installation from Conda may break (or be fixed!) as time passes due to dependency-of-a-dependency upgrades, newer/older Conda versions etc.
To make sure that our tool wrappers will work on a wide range of deployments it is important that we include all dependencies that a tool has (i.e include coreutils if a wrapper uses
paste, include sed if a wrapper usessed, include python if a wrapper calls a python script). One way to make sure this is the case is to run tool tests in a minimal container that does not contain any software except the one listed in our tool wrappers<requirements/>tag set.We also currently build containers for all IUC tools as well as other tool repositories listed in https://github.com/galaxyproject/planemo-monitor, but so far we are not testing that tool runs actually succeed in these containers. For dependencies (and dependency combinations) that are not available as a downloadable container Galaxy can build containers when needed. This functionality is also available via
planemo test --biocontainers.We have set up a github workflow that tests all IUC tools using
planemo testand the--biocontainersflag. The test run can be seen here: https://github.com/mvdbeek/tools-iuc/commit/529a26943046ca80b18bccae31021db5c317eb7e/checks?check_suite_id=331921644 And a html report of only the failing tests is here: https://mvdbeek.github.io/iuc-tool-test-results/ Note also that the tool tests run against python 3.7, so this could be another class of errors we may need to fix (should concern only the cheetah templating).The list of tools with failing tests is the following:
If you're currently working on one of these tools write your name under the tool, so that we avoid duplicating our work