search

pietercolpaert / hardf

Low level PHP library for RDF1.1 based on N3.js
https://packagist.org/packages/pietercolpaert/hardf
MIT License
36 stars 7 forks source link

results table #8

Closed moissinac closed 7 years ago

moissinac commented 7 years ago

Having the same memory footprint for 2K triples and 4M triples seems strange. How is it possible?

pietercolpaert commented 7 years ago

We read the turtle file streamingly instead of loading the file entirely in memory. We each team read a chunk, parse that, return triples, clear memory, continue parsing, etc. The only thing that is kept in memory are the prefixes and an entire literal, so the memory consumption could raise when large literals are given or when a lot of prefixes are used.