pouchdb-community / pouchdb-load

Load documents into CouchDB/PouchDB from a dumpfile
Apache License 2.0
120 stars 33 forks source link

Optimize checkpointing when using for not read-only replications #14

Open jacargentina opened 9 years ago

jacargentina commented 9 years ago

I'm using this as a "fast start" for my cordova app. The thing i would love to make faster is the "replicate.to(...)" step.

As pointed on the using instructions, after doing load(..), it must be passed to normal "replicate.from(...)". As i'm doing a two way replication (not read-only), i need to do "replicate.to(...) AND replicate.from(...)"...

I've found that in that scenario, i must wait a LOT for the upload part. See this:

Dump files: 10 txt files, taking 6.1 MB in size Time spent on the db.load(...) for those 10 files: 15 secs. Time spent on the db.replicate.to(...): 1.48 mins. Time spent on the db.replicate.from(...): 38.41 secs.

So my intent is to make that 1.48 mins. MUCH lower.

@nolanlawson Hope this can be possible?

nolanlawson commented 9 years ago

Are you using the proxy option? The times for both db.replicate.to() and db.replicate.from() seem much, much too high.

Also, when you say to, I assume you mean local.to(remote)?

A live example would be very helpful to illustrate what's going on here.

jacargentina commented 9 years ago

Yes to all. I'll try to put a live sample shortly.

jacargentina commented 9 years ago

@nolanlawson There you have the full example to test locally

https://github.com/jacargentina/pouchdb-load-testing

CharlesGrimont commented 5 years ago

Hi ! We are trying to use the proxy function and a lot of network request such as _changes are made after the bulk load and first pull replication !

I thought that once imported we would be completly synchronised. Is this normal ?

Thanks

bilalswiftsolutions commented 8 months ago

any update on this?