Change the default concept tree

[hohonuuli]

The current tree is MBARI's we'd love to be able to switch between trees for most concept name functions.

[hohonuuli]

@kakanikatija Suggests that we use the FathomNet tree. She wants to call it MorphoTaxa.

To use FathomNet tree we would need to merge in equipment from MBARI's tree. This would break the tree browser too.

[hohonuuli]

I'd like to keep this solution generic and not tied to any M3 services. One path would be to export the equipment tree from the VARS KB as a CSV. Then modify the WormsLoader to read our new CSV on the monthly worms load and merge it into the WorMS tree. We only need enough info from VARS to build WormsConcepts. Two things to be aware:

1. Each WormsConcept will need a unique id that doesn't clash with any already in WoRMS.
2. The equipment branch will need a parentId to attach to. The catch is that we will probably need to add a new root node (e.g. object), then attach both the worms tree and the equipment to.

[hohonuuli]

I've added a script to generate CSV files that could be merged into the worms tree.

Usage

kb-branch-to-worms-format <kb branch url> <starting synthetic aphia id>

git clone git@github.com:fathomnet/fathomnet-support.git
cd fathomnet-support
sbt stage
cd target/universal/stage/bin
./kb-branch-to-worms-format "http://dsg.mbari.org/kb/v1/phylogeny/down/equipment" 1000 > equipment.csv

Output

id,parentId,names
1000,,equipment
1001,1000,platform
1002,1000,Clathrate Bucket
1003,1000,Benthic Instrument Node;BIN
1004,1000,TPC;temperature, pressure, conductivity sensor
1005,1000,Wax corer
1006,1000,site marker
1007,1000,Dissolution Ball
1008,1000,Odor Pump
1009,1000,Remote Instrument Node;RIN
...

[hohonuuli]

Next steps:

• [x] Set up a cron job to export specific branches of the public VARS KB tree (e.g. equipment, geological features) and store them in the same location that we download the WoRMS data to (I think thats' on Eione, but I'll double check)
• [x] Modify the Main method in the worms server to accept additional files and combine them with the main worms tree.

This will give us a more complete tree with a few caveats:

• The root node will be object and won't match the worms root
• We're adding synthetic aphiaIds for the new nodes. They will be unique in our server, but they may have counter parts in WoRMS that don't match (e.g. Rock might have an id that matches an Extinct species used in worms)
• The added branches will have a blank ("") rank.
• There will be UI adjustments in the FathomNet web interface as we'll be going from using about 5000 concepts to about a million. Probably will need to trigger a search after a few characters are typed instead of live searching an in memory dataset.

[hohonuuli]

Updated cron on eione to grab equipment and geological feature branches from the external VARS knowledgebase. The new crontab is:

58 23 1 * * podman run mbari/fathomnet-support kb-branch-to-worms-format "http://dsg.mbari.org/kb/v1/phylogeny/down/equipment" 1000 > /home/podman_user/worms/kb_equipment.csv
59 23 1 * * podman run mbari/fathomnet-support kb-branch-to-worms-format "http://dsg.mbari.org/kb/v1/phylogeny/down/geological%20feature" 10000 > /home/podman_user/worms/kb_geological_feature.csv
0 0 2 * *  /home/podman_user/fetch_worms.sh
0 2 2 * *  /home/podman_user/fathomnet_start.sh

[hohonuuli]

Released as version worms-server 0.3.0. I've added the geological features and equipment branches of the VARS KB.

• http://fathomnet.org:8888/taxa/descendants/geological%20feature
• http://fathomnet.org:8888/taxa/descendants/equipment

Note that all non-worms nodes will have negative aphiaIds to indicate that we made them up. Also note that there are no guarantees of unique names enforced when the new branches are added to the worms trees.

[hohonuuli]

Also, there's a new root node named object with an aphiaId of -1

[hohonuuli]

@ermbutler Did you update the UI to use the fathomnet taxonomy provider instead of MBARI's?

[hohonuuli]

• [ ] Check that Erin has switched taxa providers
• [ ] Deploy updated UI

[ermbutler]

Hi @hohonuuli I put in a pull request for the change to the explorer terms autocomplete here: https://github.com/fathomnet/fathomnet-web-ui/pull/5

It is using https://fathomnet.org/worms/query/startswith/{inputValue}

Take a look and let me know if you need any changes. I left a small amount of logging in the code that I can remove later on.

[hohonuuli]

• [X] Change the autocomplete for the "What" search box from M3 to fastworms (FathomNet) taxa provider
• [X] Change the taxa tree to use fast worms (instead of m3)
• [ ] Change the autocomplete field when editing/creating a bounding box to use fast worms (FathomNet) taxa provider. This will need @ermbutler debounce/lookup logic from the dev/terms-autocomplete_issue-95 branch

[hohonuuli]

I've made the above change on the main branch as they're relatively minor. Not that there's an issue with duplicate nodes being added to the search tree that still needs to be resolved before deploying these changes.

[ermbutler]

Final task @ermbutler todo:

• [x] Check warnings
• [ ] Add autocomplete solution from PR to the annotation workflow
• [x] Close PR for the concept search

[ermbutler]

Related to this PR - https://github.com/fathomnet/fathomnet-web-ui/pull/5 Still todo.

[ermbutler]

I checked the warnings - they are related to where the key is placed on the iteratable tree nodes. It is a warning and not an error, there may even be a way to suppress the warning if it is too repetitive which I might look into because functionally things look ok.

I started the autocomplete work but the UI there is a bit tricky and I would like to "punt" on it @hohonuuli -- I am creating another github issue so we can prioritize it, and closing this one.