Browsable topic hierarchy in the repo

[yhe-wmo]

TT-NWPMD meeting on 2023.06.13:

@alexandreleroux demonstrated browsable topic hierarchy, which has been generated automatically by a short script. Domonstration

TT meeting considers the browsable topic hierarchy more user friendly than raw CSVs; and appreciates Alex and his colleagues on developing the concept and the script. TT meeting decides to adopt the browsable topic hierarchy.

• Action: Secretariat to implement browsable topic hierarchy in the main branch (e.g. github action to automatically rerun the script after a commit), with assistance from Alex's team.
• Action: Secretariat to further discuss with TT-WISMD on using browsable topic hierarchy for other domains.

[alexandreleroux]

Thanks @yhe-wmo ,

• If we want to test the current script on other TH domains, are there existing repos we could test against @amilan17 ? I only found this one (tt-nwpmd) under wmo-im?

Next steps ;

• My colleague @philippeth will test against other TH domains, if any
• Philippe will test running the short script automatically after a commit, as per ticket description
• Philippe will submit an MR with the script

Does that make sense to you? Thanks!

[amilan17]

@tomkralidis I added this to the TT-WISMD project for discussion at our next meeting

[amilan17]

@alexandreleroux I can test this in my forked repo, but I'll need some guidance on how to implement.

[tomkralidis]

TT-WISMD 2023-06-22:

• supportive for working level browsing
• will be valuable to add channel copy/paste to README
• will revisit once initial NWP topic proposal is put forth

[PhilippeTh]

I have completed the code and the workflow on my branch: https://github.com/PhilippeTh/tt-nwpmd/tree/PT_Markdown_Restructure_Enhancement

The github workflow is in .github/workflows/main.yml, the script itself along with the README to explain the workflow and how to run the script can be found in https://github.com/PhilippeTh/tt-nwpmd/tree/PT_Markdown_Restructure_Enhancement/tree/script

If you have any comments, feedback or questions feel free to reply and if we see no problem with the way I've done it I will then make a merge request.

Thanks!

[tomkralidis]

TT-WISMD 2023-07-07:

• depends on https://github.com/wmo-im/wis2-topic-hierarchy/issues/44#issuecomment-1625378084
• @amilan17 will test

[yhonda21]

TT-NWPMD 2023-07-05: The new tool was tested and implemented by an ECCC colleague, Phillippe. He also created a document on how to use it. Currently, our proposed TH is only available. Once other TH is developed, it will be automatically integrated by the tool. Who will take care of this tool will be discussed at the next TT-WISMD meeting.

[yhonda21]

TT-NWPMD meeting on 2023.09.14

• TT-WISMD discussed. One of the TT-WISMD members found that this brousable topic hierarchy would be a heavy process for the complicated TH. So the approach of the directory strucutre of the TH would be sufficient. TT-WISMD hasn't reached the conclusion yet. The ticket #14 is kept open.

[alexandreleroux]

Thanks @yhonda21 for the followup.

Can you ask clarifications to TT-WISMD how this potential improvement would be a heavy process for the complicated TH?

• The creation / update of the new ReadMes can easily be fully automated by automatically running the script after a commit to the TH repos, no manual intervention required. What would be 'heavy'?
• The script adds new content based on the existing TH csv files, doesn't impact existing content
• As we've seen during yesterday's TT-NWPMD meeting to which Enrico participated, we needed such browsable topic hierarchy in order to make sense of potential changes to the current hierarchy and proceed with a proper analysis. Without such a way to visualize the topic hierarchy, teams having to assess and improve TH have a big handicap.

Of course if it's not this simple script, the need for visualizing TH beyond the raw CSV files remain.

Thanks for the additional feedback Yuki and anyone else!

[josusky]

Hi @alexandreleroux, when we tried to use it in the 8 level TH where some levels had hundreds of possible values (especially center ID and country), and the script generated hundreds of thousands of files (thus hundreds of MBs). That is what Tom calls "heavy". It will be a bit less heavy if the center ID and country are joined (https://github.com/wmo-im/wis2-topic-hierarchy/issues/58). So in TT-WISMD, we are not against visual aids, we were just afraid of flooding the git repository with tons of almost identical files. Maybe it helps if you look at the TH (or at least its first 8-ish levels) as as set of key-value pairs that are put together, in a defined order. Each CSV file defines values for one key. Below that it starts to be a hierarchy, a tree, but the beginning is a multidimensional matrix.

[alexandreleroux]

Thanks @josusky for your feedback.

Now that https://codes.wmo.int/wis/topic-hierarchy is alive and is browsable in a similar way that what this ticket proposed last year, I'm closing this ticket.

Thanks and regards!