We were expecting the importer to act like a stream, trying to call importer.on('end' etc. but our importer is actually a pipeline -- it pumps the inputs through a number of streams. Current behavior of pipelines (using pump is that you give it a callback function that it triggers when it's done pumping all the content through the pipeline.
For the long term, we could experiment with making it emit an 'end' event instead, but for now I just made the code work with pump as-is
You can't write objects to hypercore -- only strings and buffers
I was hoping to write our json to hypercore as actual json objects, but hypercore only accepts strings. This means that we will have to parse those JSON strings whenever we read them from hypercore and will have to stringify them when we write to hypercore.
benjaminettori
commented
8 years ago
This PR sorts out a couple of confusing things.
Lessons I learned:
The importer is a pipeline, not a stream.
We were expecting the importer to act like a stream, trying to call
importer.on('end'etc. but our importer is actually a pipeline -- it pumps the inputs through a number of streams. Current behavior of pipelines (using pump is that you give it a callback function that it triggers when it's done pumping all the content through the pipeline.For the long term, we could experiment with making it emit an 'end' event instead, but for now I just made the code work with pump as-is
You can't write objects to hypercore -- only strings and buffers
I was hoping to write our json to hypercore as actual json objects, but hypercore only accepts strings. This means that we will have to parse those JSON strings whenever we read them from hypercore and will have to stringify them when we write to hypercore.
So @benjaminettori this makes #32 obsolete!