search

Teddy-Li / LLM-NLI-Analysis

14 stars 2 forks source link

IRP dataset? #1

Open i-am-neo opened 1 year ago

i-am-neo commented 1 year ago

@Teddy-Li @nighttime Thanks for your interesting paper! I'm interested in having another language model take a test using your methods.

Can you point me to datasets for Random Premise task (IRP ) and typed identifiers (IRP _TA)?

Thanks for your assistance.

Teddy-Li commented 1 year ago

Hello! Thanks for your interest! Apologizes for the previously slightly outdated repository structure; I have recently restructured the codes and data and results files so they are hopefully clearer for use. The I{RP} and I{RP_TA} files now sit in this folder; particularly, loading the type-argument variant requires both files, handled by the load_typed_general_entries function in utils.py. More detailed instructions on analyzing the results will shortly become available on the README of this repo. Do let me know if you have more questions :)

Cheers

iBibek commented 1 year ago

Thank you @Teddy-Li