emily-bicks commented 1 year ago

Description

Calculates the average pairwise depth in the taxonomy that known groupings of entities stay together. Modifies four scripts:

config / entity_groups_topics.yaml: defines the entity groupings based on wikipedia academic disciplines
config/entity_groups_subtopics.yaml: defines the entity groupings based on the crowsourced pairs
pipeline / taxonomy_validation / pairwise_entity_depth: calculates the average pairwise depth
getters / validation: adds getters for the config and the output files

Fixes #50

Instructions:

This was rebased off of 12_validate_taxonomy so has alot more commits than were actually modified in this PR. Would suggest waiting to review until 12_validate_taxonomy has been merged into dev.

ampudia19 commented 1 year ago

@emily-bicks is there anything from here not already in #60 ? Because I see some of the same files (yaml files, pairwise depth scripts), and since +70 files change (due to some duplication) it's difficult to review this one.

emily-bicks commented 1 year ago

I think the only thing thats missing in dev from this PR is: https://github.com/nestauk/dap_aria_mapping/blob/50_topic_coherence/dap_aria_mapping/pipeline/taxonomy_validation/pairwise_entity_depth.py

If you want to just glance at that I can just manually copy and paste it out of this branch and into dev

ampudia19 commented 1 year ago

Ok, changes are made, I'll merge.

nestauk / dap_aria_mapping

50 topic coherence #51

Description

Instructions: