search

JoelNiklaus / LawInstruct

This repository is a collection of legal instruction datasets
11 stars 3 forks source link

Multi legal neg #23

Closed VioCamill closed 5 days ago

VioCamill commented 1 week ago

Example multi_legal_cue_detection_swiss:

{"dataset_name": "MultiLegalNegation", "subset_name": "multi_legal_cue_detection_swiss", "source": "https://huggingface.co/datasets/rcds/MultiLegalNeg", "instruction_language": "en", "prompt_language": "de", "answer_language": "de", "jurisdiction": "SWITZERLAND", "task_type": "TEXT_CLASSIFICATION", "downloaded_timestamp": "07-05-2024", "instruction": "Extract the negations from the following sentence.", "prompt": "Text: Die kantonalen Akten wurden eingeholt. Ein Schriftenwechsel wurde nicht durchgef\u00fchrt. ", "answer": "Extracted negations: - nicht \n"}

Example multi_legal_scope_detection_swiss: 

{"dataset_name": "MultiLegalNegation", "subset_name": "multi_legal_scope_detection_swiss", "source": "https://huggingface.co/datasets/rcds/MultiLegalNeg", "instruction_language": "en", "prompt_language": "de", "answer_language": "de", "jurisdiction": "SWITZERLAND", "task_type": "TEXT_CLASSIFICATION", "downloaded_timestamp": "07-05-2024", "instruction": "Extract the scopes for the following negation cues.", "prompt": "Text: Die kantonalen Akten wurden eingeholt. Ein Schriftenwechsel wurde nicht durchgef\u00fchrt. \nNegation cues: - nicht \n", "answer": "Scopes for given cues: - cue: nicht, scope before cue: Ein Schriftenwechsel wurde, scope after cue: durchgef\u00fchrt \n"}
JoelNiklaus commented 1 week ago

Looks good generally. Some small remarks: For the cue detection task, please change the instructions to add "cue": "Extract the negations from the following sentence." => "Extract the negation cues from the following sentence."

For the scope detection task: how often do multiple cues with different scopes occur?

VioCamill commented 6 days ago

Updated version: Example multi_legal_cue_detection: 

{"dataset_name": "MultiLegalNegation", "subset_name": "multi_legal_cue_detection", "source": "https://huggingface.co/datasets/rcds/MultiLegalNeg", "instruction_language": "en", "prompt_language": "fr", "answer_language": "fr", "jurisdiction": "FRANCE", "task_type": "TEXT_CLASSIFICATION", "downloaded_timestamp": "07-09-2024", "instruction": "Spot the negation cues in the following sentence.", "prompt": "Text: Dans la mesure o\u00f9 la recourante fait valoir que la cour cantonale a n\u00e9glig\u00e9 l'effet que l'on pourrait attendre d'une peine assortie du sursis ou encore qu'il \u00e9tait arbitraire de la condamner \u00e0 une peine ferme, sa critique revient \u00e0 se plaindre du refus de lui accorder le sursis, question qui sera examin\u00e9e ci-apr\u00e8s (cf. infra, consid. 3), non pas de la quotit\u00e9 de la peine inflig\u00e9e. ", "answer": "Extracted negation cue: non"}
{"dataset_name": "MultiLegalNegation", "subset_name": "multi_legal_cue_detection", "source": "https://huggingface.co/datasets/rcds/MultiLegalNeg", "instruction_language": "en", "prompt_language": "fr", "answer_language": "fr", "jurisdiction": "FRANCE", "task_type": "TEXT_CLASSIFICATION", "downloaded_timestamp": "07-09-2024", "instruction": "Spot the negation cues in the following sentence.", "prompt": "Text: Dans la mesure o\u00f9 la recourante fait valoir que la cour cantonale a n\u00e9glig\u00e9 l'effet que l'on pourrait attendre d'une peine assortie du sursis ou encore qu'il \u00e9tait arbitraire de la condamner \u00e0 une peine ferme, sa critique revient \u00e0 se plaindre du refus de lui accorder le sursis, question qui sera examin\u00e9e ci-apr\u00e8s (cf. infra, consid. 3), non pas de la quotit\u00e9 de la peine inflig\u00e9e. ", "answer": "Extracted negation cue: pas"}

Example multi_legal_scope_detection:

{"dataset_name": "MultiLegalNegation", "subset_name": "multi_legal_scope_detection", "source": "https://huggingface.co/datasets/rcds/MultiLegalNeg", "instruction_language": "en", "prompt_language": "fr", "answer_language": "fr", "jurisdiction": "FRANCE", "task_type": "TEXT_CLASSIFICATION", "downloaded_timestamp": "07-09-2024", "instruction": "Determine the scopes associated with the negation cues in the following passage.", "prompt": "Text: Dans la mesure o\u00f9 la recourante fait valoir que la cour cantonale a n\u00e9glig\u00e9 l'effet que l'on pourrait attendre d'une peine assortie du sursis ou encore qu'il \u00e9tait arbitraire de la condamner \u00e0 une peine ferme, sa critique revient \u00e0 se plaindre du refus de lui accorder le sursis, question qui sera examin\u00e9e ci-apr\u00e8s (cf. infra, consid. 3), non pas de la quotit\u00e9 de la peine inflig\u00e9e. \nNegation cue: non", "answer": "Scopes for given cue: scope before cue: sa critique revient, scope after cue: de la quotit\u00e9 de la peine inflig\u00e9e"}
{"dataset_name": "MultiLegalNegation", "subset_name": "multi_legal_scope_detection", "source": "https://huggingface.co/datasets/rcds/MultiLegalNeg", "instruction_language": "en", "prompt_language": "fr", "answer_language": "fr", "jurisdiction": "FRANCE", "task_type": "TEXT_CLASSIFICATION", "downloaded_timestamp": "07-09-2024", "instruction": "Determine the scopes associated with the negation cues in the following passage.", "prompt": "Text: Dans la mesure o\u00f9 la recourante fait valoir que la cour cantonale a n\u00e9glig\u00e9 l'effet que l'on pourrait attendre d'une peine assortie du sursis ou encore qu'il \u00e9tait arbitraire de la condamner \u00e0 une peine ferme, sa critique revient \u00e0 se plaindre du refus de lui accorder le sursis, question qui sera examin\u00e9e ci-apr\u00e8s (cf. infra, consid. 3), non pas de la quotit\u00e9 de la peine inflig\u00e9e. \nNegation cue: pas", "answer": "Scopes for given cue: scope before cue: de la quotit\u00e9 de la peine inflig\u00e9e, scope after cue: de la quotit\u00e9 de la peine inflig\u00e9e"}
JoelNiklaus commented 5 days ago

There may have been a misunderstanding. If there are multiple words in a negation cue, those should still be one example.

VioCamill commented 5 days ago

Example multi_legal_cue_detection:

{"dataset_name": "MultiLegalNegation", "subset_name": "multi_legal_cue_detection", "source": "https://huggingface.co/datasets/rcds/MultiLegalNeg", "instruction_language": "en", "prompt_language": "de", "answer_language": "de", "jurisdiction": "GERMANY", "task_type": "TEXT_CLASSIFICATION", "downloaded_timestamp": "07-10-2024", "instruction": "Extract the negation cues from the following sentence.", "prompt": "Text: Dar\u00fcber hinaus sei die Gespr\u00e4chsnotiz weder von einem Gesch\u00e4ftsf\u00fchrer noch von dem Kl\u00e4ger unterzeichnet worden. ", "answer": "Extracted negation cues: cue1: weder \ncue2: noch \n"}

Example multi_legal_scope_detection:

{"dataset_name": "MultiLegalNegation", "subset_name": "multi_legal_scope_detection", "source": "https://huggingface.co/datasets/rcds/MultiLegalNeg", "instruction_language": "en", "prompt_language": "de", "answer_language": "de", "jurisdiction": "GERMANY", "task_type": "TEXT_CLASSIFICATION", "downloaded_timestamp": "07-10-2024", "instruction": "Identify the scopes for the negation cues in the given text.", "prompt": "Text: Dar\u00fcber hinaus sei die Gespr\u00e4chsnotiz weder von einem Gesch\u00e4ftsf\u00fchrer noch von dem Kl\u00e4ger unterzeichnet worden.  \nCues: cue1: weder \ncue2: noch \n", "answer": "Scopes for given cues: cue1: weder \nscope before cue1: sei die Gespr\u00e4chsnotiz \nscope after cue1 :von einem Gesch\u00e4ftsf\u00fchrer \ncue2: noch \nscope before cue2: von einem Gesch\u00e4ftsf\u00fchrer \nscope after cue2 :von dem Kl\u00e4ger unterzeichnet worden \n"}