search

thunlp / PELT

Source code for "A Simple but Effective Pluggable Entity Lookup Table for Pre-trained Language Models"
43 stars 8 forks source link

Is there a direct code for Pluaggable Entity lookup Table? #5

Closed Kunlun-Zhu closed 2 years ago

Kunlun-Zhu commented 2 years ago

Hi, Deming, thanks for the work. After reading some of the source code, is there a direct abstract of the Lookup table code, or we just only use the generate embeddings? If there is one, how may I use it in some other task such as link prediction, thanks.

Kunlun-Zhu commented 2 years ago

I see a 'lookup_table' parameter here, modeling_tf_transfo_xl.py
Is there where the 'pluggable Entity look up table' was represented.

Kunlun-Zhu commented 2 years ago

I also see one here modeling_xlnet.py

YeDeming commented 2 years ago

We should re-generate the entity embeddings according to the required qid for other tasks. The related code is at: https://github.com/thunlp/PELT/blob/eefecaada2094996cea0c1053cca59307fadbd49/transformers/src/transformers/modeling_robertaent.py#L255-L260

Kunlun-Zhu commented 2 years ago

Thanks, I think that I get the idea.