search

galaxyproject / galaxy

Data intensive science for everyone.
https://galaxyproject.org
Other
1.41k stars 1.01k forks source link

Expose converters in the tool panel ? #9649

Closed mvdbeek closed 2 years ago

mvdbeek commented 4 years ago

Sometimes you want to convert a collection of files, but we don't expose our existing converters in the tool panel. If you know the tool id you can construct a url like https://usegalaxy.org/root?tool_id=CONVERTER_vcf_to_vcf_bgzip_0, but I think we should (optionally?) expose these in the tool panel.

nsoranzo commented 4 years ago

I was considering to suggest that too today when looking at GFF-to-BED issue #9604 . Having converters in the tool panel would help especially for workflows when you want to make an explicit conversion between steps.

guerler commented 4 years ago

Cool idea. Would that mean that we could use them in workflows explicitly? Is that useful? Also, how is this feature related to the "Edit dataset attributes" > "Datatypes" > "Change datatype"?

nsoranzo commented 4 years ago

Cool idea. Would that mean that we could use them in workflows explicitly? Is that useful?

Yes, especially since I don't think it's presently possible to connect an output in format A to an input in format B even if there is an A->B converter available (I've just tried to connect a bam output to a bigwig input in a workflow on usegalaxy.org and the editor won't let me).

Also, how is this feature related to the "Edit dataset attributes" > "Datatypes" > "Change datatype"?

No, that is used only to change the datatype column in the database, but is related to "Edit attribute" > "Convert", which lists all the available conversions (i.e. converter tools specified in lib/galaxy/config/sample/datatypes_conf.xml.sample) for the current dataset format.

mvdbeek commented 4 years ago

(I've just tried to connect a bam output to a bigwig input in a workflow on usegalaxy.org and the editor won't let me).

:( I think that used to work and we should make sure that this will continue to work regardless of having explicit converters.

mvdbeek commented 2 years ago

Came up again in the backend meeting today as a use case for the VGP project

jdavcs commented 2 years ago

There are several ways to implement this, and before I move any further, I'd like to ask for some feedback.

There will be a configuration option (on/off) to expose the built-in converters - that's the simple part. What I'm not sure about is how they should be displayed. There are 148 built-in converters, so, I think, displaying them requires a collapsible menu - same as the other tool sections.

a) We could add a "Built-in Converters" tool section to be displayed with the other tools. However, we already have a built-in section for manually curated converters: "Convert Formats". Having these two sections appear side-by-side (or not side-by-side) in the same list might be confusing.

b) Another option is to list them separately, similar to Workflows - at the bottom. In this case, we would have a tool panel label, "BUILT-IN CONVERTERS". That label is not clickable, but below it would be a tool section title "Converters", which would be clickable (like other tool section titles) and would expand to list all the 148 converters.

If I read the code correctly, with option (a), the list of these converters would be filtered and sorted (by search rank) together with the other items in the toolbox. I'm not sure that's what the user would expect from such a list though? Option (b), on the other hand, would be static - like the workflows section at the bottom. I think option (b) is more intuitive for such a list: with built-in converters I think I'd rather click to expand and then browse all available options, not search like I would do with tools. But that's only my guess, and it's not based on my experience using galaxy for research.

The implementation choices are different for options a and b, so any feedback would be helpful. And there can be other options too, of course.

mvdbeek commented 2 years ago

There are 148 built-in converters, so, I think, displaying them requires a collapsible menu - same as the other tool sections.

Yes, loading them in a tool section seems fine. But where'd you get that number from ? I'm counting 60 that are currently loaded: ack converter config/datatypes_conf.xml.sample|ack 'file="(.*?)"' --output '$1'|sort|uniq|wc -l

jdavcs commented 2 years ago

Oh yes, 60 (I was using the wrong data - unfiltered output from api/datatypes/converters).

So I'll add them as a tool section, so they'll be displayed together with the other tool sections (not separately like Workflows). Thanks!

mvdbeek commented 2 years ago

Done in https://github.com/galaxyproject/galaxy/pull/13447, thanks again @ic4f!