flairNLP / fabricator

[EMNLP 2023 Demo] fabricator - annotating and generating datasets with large language models.
Apache License 2.0
98 stars 12 forks source link

Automatically infer TextLabelPrompt for QA and ClassLabelPrompt for Text Classification #25

Closed julian-risch closed 1 year ago

julian-risch commented 1 year ago

This PR adds a infer_prompt_from_dataset function that automatically infers TextLabelPrompt or ClassLabelPrompt with the correct parameters from a dataset's meta data (if available).

Also adding three tests and changed two of the examples so that they now use infer_prompt_from_dataset.

for extractive question answering on SQuAD 2.0

# prompt = TextLabelPrompt(
#     input_variables=arguments.input_variables,
#     target_variable=arguments.target_variable,
#     task_description=arguments.task_description,
# )
prompt = infer_prompt_from_dataset(dataset)

and for text classification on imdb:

# idx2label = dict(enumerate(fewshot_examples.features[arguments.target_variable].names))
#
# prompt = ClassLabelPrompt(
#     input_variables=arguments.input_variables,
#     target_variable=arguments.target_variable,
#     label_options=idx2label,
#     task_description=arguments.task_description,
# )
prompt = infer_prompt_from_dataset(dataset)