ge-semtk / semtk

Drag and drop SPARQL queries and data ingestion for virtuoso and soon other SPARQL endpoints
http://semtk.research.ge.com
Other
37 stars 6 forks source link

Handle large files in SPARQLgraph import tab #473

Open cuddihyge opened 1 year ago

cuddihyge commented 1 year ago

When a file is drag-and-dropped to SPARQLgraph for uploading, it reads the whole thing into memory instead of streaming it. This precludes large imports. Also precludes using dropping zip files with manifests (which are usually big).

see uploadtab.js function onDropData() which is bound to the dropZones.

It seems like we ought to be able to use FormData something like stackoverflow example to stream a multipart file.