Prototype semantic DBpedia taxonomisation approach

[georgerichardson]

Using DBpedia entities from AI and Genomics project, generate a hierarchical* taxonomy.

Method starting point:

• Generate a set of all DBpedia entities associated with the corpora
• Filter entities (e.g. those with very low frequency)
• Embed each entity separately using a transformer (options could include sbert STS stransformers or SPECTER)
• Use hierarchical or iterative clustering methods to generate clusters of entities at different levels of granularity.
  • This could use agglomerative clustering, HDBSCAN (with reassignment of outliers) or a method that requires a specified n_clusters like K means, GMM etc.
  • Probably worth using some clustering optimisation with e.g. silhouette score as a minimum
  • Liz has some experience with this kind of thing. Juan has developed a robust clustering pipeline for AFS and AHL which might be worth looking into - Izzy/Elena or Rachel know about this.

*I'm not sure the taxonomy needs to be strictly nested to be useful.

[ampudia19]

Following today's chat & given our different exposure to network/LLM methods, perhaps I should have this task assigned to me, and Emily can take care of #10. That said, I'm happy with any alternative, including having both work on both issues.

[emily-bicks]

I'm happy with that!