Acquire MARBLE Data

[janetzki]

Goal

As a developer, I want to acquire the data from MARBLE to use their mappings from words to lexical domains. These mappings indicate to which semantic domain a verse (or word or phrase) belongs (for Hebrew and Greek). Motivation: disambiguate words -> reduce FNs -> increase DC recall > 0.30

Tasks

• [x] get lexical domain mappings for the Old Testament
• [x] get lexical domain mappings for the New Testament
• [x] create a list of all data that we already can access
  • --> 2 parquet files
• [x] Parse mappings from words to lexical domains
• [x] think about how we can map lexical domains to SDs
• [x] see A.1.3 in proposal
• [x] try it out
• [ ] Look at what we already have
  • a) mappings from words in verses to MARBLE domains
  • b) mappings from MARBLE domains to SDs
  • c) mappings from words in verses to SDs
• [ ] refine c) with a) and b) (set intersection)
• [ ] (alternative idea: add lexical domains to the graph (#GNN))
• [ ] (disambiguate example verse with lexical domains)
• [ ] (create more labels by trying out the semi-automatic labeling function from [Refinery](https://github.com/code-kern-ai/refinery) (NLP IDE).)

Notes

• Lexical domains, show for example, if a verse belongs to the lexical domain "sky".
• The lexical domains are useful to disambiguate words.
  • because we know to which lexical domain a word in a verse belongs
  • e.g., "fire" can refer in one verse to burning and in another vers to moving quickly
• Idea: Language-agnostic annotations are our starting point.
• Lexical domains do not match semantic domains
• Resources
  • https://github.com/Clear-Bible/macula-greek/issues/29
  • https://github.com/Clear-Bible/macula-hebrew/issues/61
  • 'macula-greek/sources/MARBLE/SDBG/' (general information)

[janetzki]

Closed because fine-tuning ada looks more promising.