dkmfbk / rdfpro

RDFpro
Creative Commons Zero v1.0 Universal
12 stars 5 forks source link

@download should process SPARQL results in a streaming way #3

Open ziorufus opened 9 years ago

ziorufus commented 9 years ago

When downloading from Virtuoso, the full response is buffered by HttpClient / Sesame leading to OutOfMemory if a large amount of data is fetched. Solving the issue will require revising Sesame HTTPClient class. Considered the number of dependencies imported by that class and its limited use in RDFpro, it is better to reimplement SPARQL IO directly in DownloadProcessor and UploadProcessor (possibly importing SPARQL XML parser from Sesame).