GTN tool panel views

[hexylena]

I was recently reminded of https://github.com/galaxyproject/galaxy/pull/12327 (thanks again @jmchilton) and somehow we've not done anything about it for 2 years, but we really should!

We have IDs of tools used in tutorials & topics, and we could pretty easily generate tool panel views by topic + for the whole GTN, that we export and admins can install easily.

[bernt-matthias]

Accidentally, I also stumbled over this one yesterday and was wondering how it could happen that I forgot about this. Managing the toolbox is a major annoyance and tool panel views seem to be an excellent solution. Also, the exclusion of deprecated tools could be solved by this.

How can we make such tool panel views usable by many admins. On my instance, I could quite easily views with subsets of sections or merged sections, but this would have limited reusability on other Galaxy instances since they probably use different section names (even if mine are mainly identical to the TS categories).

Regarding the idea of generating views from GTN topics I was wondering if the idea is to produce one view where each topic generates one section -- or rather one view per topic (but then how to generate the substructure?).

Would be cool if we could have something like for TPV, i.e. a community-maintained set of yaml files. GTN could automatically create PRs and apart from this communities of practice (eg metabolomics, proteomics, ...) could contribute with manual PRs.

Was wondering if the yaml files could be combined with kind of include directives.

@lldelisle also mentioned some work you did to get tool lists + section lists from the big instances .. but I forgot the details :(

Can we somehow trigger some feedback of the admin community on this issue?

[lldelisle]

Indeed @hexylena published a json mapping tool_id to tool_panel_section_label for big instances, see: https://usegalaxy-eu.github.io/usegalaxy-eu-tools/api/labels.json https://galaxyproject.github.io/usegalaxy-tools/api/labels.json https://usegalaxy-au.github.io/usegalaxy-au-tools/api/labels.json

But could be a good idea to set up yaml for tool panel views that admins could reuse. Still, as this rely on tool_panel_section_label, I don't know if the discrepancies in tool_panel_section_label would be an issue.

[hexylena]

All of those labels get aggregated into a single listing one can use via the GTN. It's exposed here: https://training.galaxyproject.org/training-material/api/toolcats.json (the category listed depends on the value agreed to by multiple servers.)

[hexylena]

I was wondering if the idea is to produce one view where each topic generates one section -- or rather one view per topic (but then how to generate the substructure?).

the original idea from my side was one view per topic in the GTN (which vaguely somewhat maps to subdomains as well)

the substructure is generated via grouping by this GTN aggregates toolcats.json file, we group based on the categories of every listed tool, by their expected section name, even if that section isn't actually present in that server.

[bernt-matthias]

This is pretty cool and will certainly be useful.

All of those labels get aggregated into a single listing one can use via the GTN.

Where is the code living that generates this? Is this a majority decision?

Maybe we can extend this tool to build views by aggregating assignments of tools to labels/sections from multiple possible sources, like:

• other servers
• GTN
• manually curated assignments (maybe curated by a community)
• local preferences
• ontologies
• categories listed in .shed.yml

One thing that came to my mind is that for specialized panels, e.g. metagenomics, one could need a more fine-grained structure. I'm mostly thinking here of large tool collections like QIIME2 or OpenMS where tools might be categorized even more. For qiime2 the tool suites (and names) define such a structure conveniently, for other tool collections one might need other means to get a good sub-categorization.

Currently, tool panels do two things:

1. show only a subset of the tools
2. show those tools in a possibly different hierarchy

Maybe it could be of interest to allow to separate this, e.g. show me all metagenomics tools structured by EDAM topics.

[hexylena]

Where is the code living that generates this?

https://github.com/galaxyproject/training-material/blob/main/bin/fetch-categories.rb

that definitely sounds like a lot of interesting solutions to automating it!

[hexylena]

I'm not sure why they haven't been posted already, but @bgruening linked me these repositories as well:

• https://github.com/galaxyproject/gx-tool-db
• https://github.com/galaxyproject/gx-tool-db-project
• https://github.com/galaxyproject/gx-tool-db-panel-sync

so seems like @jmchilton and @mvdbeek are working in a similar direction potentially? (I would suggest they don't clone the GTN but instead use the API, and we make whatever improvements they need! because we have much nicer tool panel listings in the API that could be easier to use and wouldn't require cloning the GTN. available via the API)

And @nomadscientist wrote me recently about the same possibility of syncing tool panels across instances + syncing with subdomains.

Perhaps there is a very good opportunity here for Galaxy, if accessed via a subdomain, to show the toolpanel with the same name, if it exists, rather than the default. So a annotation.usegalaxy.eu could use the annotation.yml tool panel by default for that subdomain, and we could remove the python based toolbox filters completely then.

[mvdbeek]

(I would suggest they don't clone the GTN but instead use the API, and we make whatever improvements they need

I'm not super sure what you mean by that. This was the easiest way to get the panels from .eu to test.galaxyproject.org, but of course that can't remain the source of truth. Should the GTN be the source of truth for the subdomain data ?

So a annotation.usegalaxy.eu could use the annotation.yml tool panel by default for that subdomain, and we could remove the python based toolbox filters completely then.

that was the idea when this was implemented, you can set default_panel_view by host, see https://github.com/galaxyproject/galaxy/pull/12328

[hexylena]

I'm not super sure what you mean by that. This was the easiest way to get the panels from .eu to test.galaxyproject.org, but of course that can't remain the source of truth.

Ah I wasn't talking about getting panels from eu/test, instead about how the https://github.com/galaxyproject/gx-tool-db-project which includes a copy of the gtn, and adding an alternative code path to walking the workflows and instead just fetch from the api. but that's a separate issue / probably offtopic for this thread.

Should the GTN be the source of truth for the subdomain data ?

We could be? I wouldn't be opposed to it, we're using very similar subdomains as GTN topics, on some level it could make sense?

that was the idea when this was implemented, you can set default_panel_view by host, see https://github.com/galaxyproject/galaxy/pull/12328

fantastic!! I had missed that. Great to find out it's already solved :)

[hexylena]

discussing with @nomadscientist about where these community homes live (gtn, hub CoP pages, subdomain pages), and the role of e.g. the single cell topic page which she's already expanded greatly compared to other topics, including the workflow listing widget. And given that a few of the subdomains are moving towards including multiple GTN 'widgets' on their subdomain homes, we could conceive of just hosting the whole subomain page ourselves.

I can definitely see an argument for the GTN becoming the 'home' of those communities, and then us centralising everything about these communities here:

• subdomain home page
• widgets that are inclued on that page
• community resources / homepage
• tool listing generation

[bernt-matthias]

Btw. I just discovered that there can be tool panel views of type training.

[hexylena]

Yep, all my drafts had that tag and the cute lil hat icon. It's quite nice!

[nomadscientist]

Oh wow this looks amazing

[hexylena]

It seems like @bebatut's project will implement something similar and I guess we don't need to duplicate this work, we don't need toolboxes generated from GTN topics.