search

HamedBabaei / LLMs4OL

LLMs4OL:‌ Large Language Models for Ontology Learning
MIT License
87 stars 8 forks source link

Vision, Problem Formulation, Tasks, RQs, Writings, .... #10

Closed HamedBabaei closed 1 year ago

HamedBabaei commented 1 year ago

LLMs4OL

Research Questions

RQ1: Can LLMs identify term/entity types? Task A RQ2: Do LLMs comprehend relations?

Tasks

Task A: The goal is to find out which LLMs are capable of finding terms/entities type without giving prior knowledge about types. Because we don't want to give any knowledge to LLMs about types, this task is a Generation task. Design considerations during solving this task are as follows:

  1. We are interested to know the entity types at the lowest level.
  2. Since one of the possible steps is fine-tuning so splitting data into train and test sets are required!
  3. We only will consider entities/terms.
  4. Entities in types hierarchy leaf inherent in their own parent's types as well (this is for the evaluation part since for this task we don't expect models to know the hierarchies).

Task B: The aim of this task and the next task (Task C) is to understand whatever LLMs could find relations without naming those relations. This relationship could be an undirected or directed relationship. These tasks are classification tasks.

For example:

Acquired Abnormality is a location of a Virus.

The location_of is a relation between mentioned two types. Our goal is to find that Acquired Abnormality and Virus have relations. Not find the name of the relation (which in this case is location of). Because naming relations refers to clustering similar relations and asking experts to name them. So in this task, we are interested to know what is the is a relation in terms/entities types.

In Task B, we want to only find types of relationships that form a hierarchy (a structure that struct types tree format from top to down where the top is a root -- it could be multiple roots -- and down is a leaf) and this type of relationships called is a relations. As an example:

C is a subclass of B. B is a subclass of A. D is a subclass of B. E is a subclass of A.

Task C: However, in types, it is possible to find relations outside of the tree structure, and it is similar to relations between types in graph format. For example (considering Task B example):

E somehow has a relationship with C. C somehow has another direct relation with A.

So, in this task, the goal is to analyze LLMs from this perspective.

Conference

Our Target is for ISWC 2023: https://iswc2023.semanticweb.org/call-for-research-track-papers/

Abstract submission due May 2nd, 2023
Full paper submission due May 9th, 2023
Objection and Response June 13th – 16th, 2023
Notifications July 12th, 2023
Camera ready papers due July 31st, 2023
HamedBabaei commented 1 year ago

Links for Information (the list will be updated regularly

Info date link
Diagram of the paper 23 Feb 2023 LLMs4OL.pdf
Link to editable version of overleaf 23 Feb 2023 ISWC 2023 Overleaf Template
HamedBabaei commented 1 year ago

Things that are on my to do list:

  1. LLaMA for Tasks
  2. ORKG research field taxonomy for Task B: https://orkg.org/help-center/article/20/ORKG_Research_fields_taxonomy
  3. Task D: generation of documentation for classes, properties task