Nardien / KALA

Official Code Repository for the paper "KALA: Knowledge-Augmented Language Model Adaptation" (NAACL 2022)
36 stars 3 forks source link

Ask for the case where relational retrieval cannot retrieve entity embeddings from entity memory ? #4

Closed pavaris-pm closed 1 year ago

pavaris-pm commented 1 year ago

@Nardien @JinheonBaek Specifically, I am interested in understanding how KALA handles unseen data that was not included in the training set by utilizing a knowledge graph to retrieve the neighborhood of the unseen entity from entity memory and perform weight aggregation with its neighbors.

Relational Retrieval which leverages a KG G to retrieve entity embeddings from the memory, according to the relations defined in the given KG

However, I am curious to know how KALA handles cases where the unseen entity does not have a neighborhood that can be retrieved from entity memory (has no relations about that unseen entity defined in given KG).

In that case, does KALA treat the unseen entity as a null entity and handle it using fixed gamma and beta values, similar to when using entity memory alone? (point-wise) or is there another mechanism to handle such case ? I would greatly appreciate any information you can provide regarding this aspect of KALA.

Thank you in advance.

Nardien commented 1 year ago

I appreciate your interest in our work and apologize for the late response.

I've included the answer to your question below.

Does KALA treat the unseen entity as a null entity and handle it using fixed gamma and beta values, similar to when using entity memory alone? or is there another mechanism to handle such cases?

Yes. As a consequence, an unseen entity having no relation with other entities is not modulated by KALA, as the tokens do not belong to any entity. There is no additional algorithm to handle this case.

I hope my answer clarifies your question. If you have any other questions, feel free to ask. Thank you.

pavaris-pm commented 1 year ago

Thank you !!!