Deserializer works with bufio.Scanner interface for line splitting, allowing to pass io.Reader as input to Parse. This should also reduce GC pressure on large datasets, because deserializer processes lines on the fly without a need to store them in a separate slice. Performance impact is minimal, as newScannerFor switches Scanner interface to custom implementation for []byte and string to bypass copy operations in original bufio.Scanner. There is also a benchmark for that.
Serializers can implement an optional RDFSerializerTo interface advertising it's ability to write serialized RDF directly into io.Writer without a need of temporary string representation. N-Quads serializer implements such behavior now.
kazarena
commented
9 years ago
Changed N-Quads code a bit:
bufio.Scannerinterface for line splitting, allowing to passio.Readeras input toParse. This should also reduce GC pressure on large datasets, because deserializer processes lines on the fly without a need to store them in a separate slice. Performance impact is minimal, asnewScannerForswitches Scanner interface to custom implementation for[]byteandstringto bypass copy operations in originalbufio.Scanner. There is also a benchmark for that.RDFSerializerTointerface advertising it's ability to write serialized RDF directly intoio.Writerwithout a need of temporary string representation. N-Quads serializer implements such behavior now.All changes are backward-compatible.