ChatGPT queriers & basic transformation utils are included in this PR. These modify a portion of the existing taxonomy, performing the following set of possible transformations:
remove entities from the taxonomy
remove topics from the taxonomy
remove areas from the taxonomy
merge topics
move entities between topics
split topics
adjust areas
move areas
adjust topic names
add country tags
add continent tags
These are done via a combination of manual revision & querying questions to chatGPT. The code for manual revision is ommitted from this PR.
Discard topics: Spanish entities, French entities, etc.
Split topics: If re-assignment is not possible but entity/ies is/are large or relevant, split into its own topic.
Move levels up and down. Create Miscelaneous Domain / Area levels, split at level 3 with splits currently in level 1 and 2 that may be too specific.
These have been achieved for level 1 and 2, whereas it proved too time consuming for level 3 (excepting moving levels up and down, and the deletion of entities). Note that instead of deleting countries, regions, and cities, these are now bundled in a new topic.
Instructions for Reviewer
The workflow is as follows:
Checklist:
[X] I have refactored my code out from notebooks/
[X] I have checked the code runs
[X] I have tested the code
[X] I have run pre-commit and addressed any issues not automatically fixed
[X] I have merged any new changes from dev
[X] I have documented the code
[X] Major functions have docstrings
[X] Appropriate information has been added to READMEs
Description
ChatGPT queriers & basic transformation utils are included in this PR. These modify a portion of the existing taxonomy, performing the following set of possible transformations:
These are done via a combination of manual revision & querying questions to chatGPT. The code for manual revision is ommitted from this PR.
The original goals of issue #76 were to:
These have been achieved for level 1 and 2, whereas it proved too time consuming for level 3 (excepting moving levels up and down, and the deletion of entities). Note that instead of deleting countries, regions, and cities, these are now bundled in a new topic.
Instructions for Reviewer
The workflow is as follows:![image](https://user-images.githubusercontent.com/61916660/228602743-91595806-f32f-4bdd-88a9-c4ac9ad69584.png)
Checklist:
notebooks/
pre-commit
and addressed any issues not automatically fixeddev
README
s