search

qurator-spk / neat

Named entity annotation tool
Apache License 2.0
27 stars 5 forks source link

Support streaming for large TSV files #1

Closed cneud closed 4 years ago

cneud commented 5 years ago

Use papaparse streaming with chunk https://www.papaparse.com/faq#streaming and preferably pagination using sentence boundaries (= newlines)

cneud commented 4 years ago

tested successfully for up to 4MB (~300,000 lines) TSV files