Support JSONL format - Githubissues

HumanSignal / label-studio

Label Studio is a multi-type data labeling and annotation tool with standardized output format

https://labelstud.io

Apache License 2.0

19.58k stars 2.43k forks source link

Support JSONL format #1960

Open johann-petrak opened 2 years ago

johann-petrak commented 2 years ago

For larger files and generally for easier handling of example sets in json format, it would be very useful if JSONL format was supported:

each example a JSON map, on its own line
no enclosing array

In other words, such files would get processed line by line, with each line getting converted to/from json as needed on export/import.

See https://jsonlines.org/

makseq commented 2 years ago

Are you talking about LS storages with jsonl import? Or about the common LS import files?

johann-petrak commented 2 years ago

Sorry, what I meant is for

uploading examples to annotate (Import button within a project)
downloading annotated examples (Export button within a project)

jsonl has become an often-used format in nlp because it is flexible, yet it is, unlike a json array file, line oriented, so other tools can be used to filter, update. sort, etc examples in jsonl files.