search

pouchdb-community / pouchdb-replication-stream

Replicate PouchDB/CouchDB databases with Node.js-style streams
Apache License 2.0
188 stars 59 forks source link

Broken when replicating attachments #51

Open nolanlawson opened 8 years ago

nolanlawson commented 8 years ago

See https://github.com/nolanlawson/pouchdb-dump-cli/issues/14

nolanlawson commented 8 years ago

OK, I can reproduce by pouchdb-dumping a database that contains an attachment. It only occurs when there are attachments.

nolanlawson commented 8 years ago

The problem is that the writeable-stream adapter needs to define a _get now. Unfortunately due to https://github.com/pouchdb/pouchdb/pull/5018 the _get method is now required for adapters of type http.

WorldMaker commented 7 years ago

Is this still an issue? I was contemplating trying something wild like WebTorrent as the backend for PouchDB replication in a somewhat peer-to-peer fashion, but for my projects there are enough attachments (photos) that it doesn't make sense to try to build something like that if it won't support attachments.

nolanlawson commented 7 years ago

Yes I haven't had time to fix it. PRs welcome. :)

WorldMaker commented 7 years ago

I'm hitting some issues where I may need a replicable export dump in the near future, and as mentioned attachment support is important to the projects I'm working on. I'm trying to gauge if it would be worth trying to integrate that into this project or start "fresh" with a file format that more easily supports large binary attachments as "first-class" than newline-delimited JSON (such as using one of the JS-based zip file libraries).

nolanlawson commented 7 years ago

The problem isn't JSON vs non-JSON, it's that pouchdb-replication-stream needs to implement the _get method.

WorldMaker commented 7 years ago

In my use-case of many, potentially large image files, base64 overhead is possibly a concern, that's what lead to me considering maybe a zip-based format. Especially too because that makes it easier for users to extract the zip to explore things like embedded photos as opposed to needing a base64 decoder.

rileyai-dev commented 1 year ago

@nolanlawson I'd be happy to help if you give me a little guidance... Was it solved in the cli?