bigai-nlco / LooGLE

ACL 2024 | LooGLE: Long Context Evaluation for Long-Context Language Models
MIT License
147 stars 6 forks source link

JSON data wrapped in single-quotes appear in 'qa_pairs' #9

Closed vshallc closed 1 month ago

vshallc commented 1 month ago
from datasets import load_dataset

data = load_dataset('bigainlco/LooGLE', shortdep_qa, split='test')
print(shortdep_qa[0]['qa_pairs'])
"""
Outputs:

[{'S': 'Picardo was born in Jer...
...
...make the buildings look old.'}]

"""

All keys ('S', 'Q', 'A') and all values except those who already contain single-quotes appears in single-quotes.

Parsing data with eval() works, but json.loads() fails...

lijiaqijane commented 1 month ago

Thanks for your comments and we'll fix it and update the data for better use.