pchampin / sophia_rs

Sophia: a Rust toolkit for RDF and Linked Data
Other
210 stars 23 forks source link

parsing and serializing in a stream #141

Closed cmdoret closed 8 months ago

cmdoret commented 8 months ago

First of all, thanks for your incredible work, this library looks awesome!

I am still learning rust, so I may be overlooking something here.

Context I want to write an equivalent of rdfpipe in rust using the sophia@0.8.0 crate, so I looked at the parse.rs example.

My question is based on the match block used to instantiate a parser based on the format:

https://github.com/pchampin/sophia_rs/blob/9fb2bda69eb0d536d5ebaabc6373a970e418f3dd/sophia/examples/parse.rs#L56-L62

Problem

Is it possible, and if so what is the recommended way to return the output of e.g. NTriplesParser.parse(input) into a lazy iterator, and then stream it to a Serializer without collecting triples. I imagine this may not be trivial e.g. when parsing / serializing prettified ttl.

What I would like to do resembles the Backend trait in the weso/RemoteHDT repo, however they avoided the issue by chaining parser.parse(input).add_to_graph(graph) which if I understand correclty, will load all triples into memory. In my case a streaming approach would be preferable.

damooo commented 8 months ago

@cmdoret You may wish to check rdf_dynsyn crate, which provides sophia parser and serializer implementations, that take syntax as runtime param, and return/take same quad-source/triple-source as result/input. Then piping will be trivial.

cmdoret commented 8 months ago

Thanks @damooo, it looks like this is indeed what I would need :+1:

damooo commented 8 months ago

@cmdoret , I am pushing a new version to integrate json-ld support today, It would have few breaking changes from existing version.