search

galaxy-iuc / standards

Documentation for standards and best practices from the Galaxy IUC
http://galaxy-iuc-standards.readthedocs.io/en/latest/
6 stars 16 forks source link

Add parallelism to best pratice tool list. #7

Closed jmchilton closed 9 years ago

jmchilton commented 9 years ago

See also https://github.com/galaxyproject/planemo/issues/132#issuecomment-97142030.

bgruening commented 9 years ago

:+1: on this, but the parallelism feature is horribly broken and I'm not sure we should advertise it.

nsoranzo commented 9 years ago

:+1: and I agree with Bjoern.

peterjc commented 9 years ago

Horribly broken? Certainly fragile and not yet general purpose (see recent galaxy-dev discussion on splitting BAM files for example).

It works for us pretty nicely with BLAST and simple sequence analysis tools (split single input FASTA file, merge tabular/text/blastxml output) but this is a narrow ideal use case.

bgruening commented 9 years ago

@peterjc can you restart Galaxy if you have a parallelism-job running?

Also these ones is not fixed yet:

Don't get me wrong, I love this feature. But it has so many corner cases that I don't think it's not ready for public consumption. I constantly loose all my BLAST job if I restart Galaxy for example.

peterjc commented 9 years ago

I try not to restart our production Galaxy if there are any cluster jobs running (BLAST or otherwise), because it has had trouble recovering the (child) jobs in the past.

I would agree with "not ready for public consumption".

jmchilton commented 9 years ago

Happy to just say this is the order in planemo then and not update this document. Should we remove code for from this list for the same reason then? Planemo will be record of the preferred order if they appear but neither should be advertised?

peterjc commented 9 years ago

Sounds like a solution - have them in the planemo standard order, but advertise neither.

(This reminds me, what should we use instead of <code file="example.py"> for advanced validation, e.g. https://github.com/peterjc/pico_galaxy/blob/64aecfa6041b3b7fe2c70a30236a7aa15e650523/tools/mira4_assembler/mira4_de_novo.xml )

jmchilton commented 9 years ago

https://github.com/galaxyproject/planemo/commit/9bf1eab046d4b3a50129d20cab49767245e2d6f9

About code I have no clue what to replace it with - I think the answer is that it is still needed in many cases unless you just want to do the validation after the job is submitted. Your code file is one of the better ones because it is very self contained and not importing random galaxy internals that make evolving the platform difficult.