search

woutslabbinck / SolidEventSourcing

A pipeline to transform location data (gpx file format) to RDF data and publish them as an Event Source on Solid
MIT License
2 stars 2 forks source link

Cannot read files with size greater than 512MB. #2

Open argahsuknesib opened 2 years ago

argahsuknesib commented 2 years ago

Hello,

I am using the library, and I have a file greater than 512MB. As the library uses, turtleStringToStore which expects the datatype string as an input, and 'string' can store 512MB max. What would be the best way to handle this? Processing the files in batch? The screenshot details the error.

image

woutslabbinck commented 2 years ago

This can be closed if your source file is split up into smaller files (i.e. kind of batch processing), right?