search

pouchdb-community / pouchdb-load

Load documents into CouchDB/PouchDB from a dumpfile
Apache License 2.0
120 stars 33 forks source link

Loading from dump is become very slow with pouchdb v6.0.7 as compared to v5.4.5 #53

Open sagarkm opened 7 years ago

sagarkm commented 7 years ago

We have dump files created with -s 5000 that creates individual dump files of 2-3 MB on average. There are ~ 30 files, totaling to ~ 60MB of dump files.

We created an array of all those files and load them sequentially to create the single pouchDB. When we used PouchDb v5.4.5 it completed the entire process of downloading and loading all files within 7 mins. Now we upgraded to PouchDB v6.0.7 and the same code now takes around ~ 35 mins (almost 5X the time). The network bandwidth is almost the same.

Loading individual files benchmarks to approx 55 sec (~ 1 min) per files with v6.0.7, which used to take only few seconds with v5.4.5.

We are using the latest version of Pouchdb-load (v1.4.6).

Is this a known issue? This makes me feel to rollback to older version of PouchDb. But I send a lot of time to upgrade to v6.

nolanlawson commented 7 years ago

This is not a known issue, thanks for reporting. A reproducible test case would be very helpful here.

On Oct 19, 2016, at 1:40 AM, Sagar Mody notifications@github.com wrote:

We have dump files created with -s 5000 that creates individual dump files of 2-3 MB on average. There are ~ 30 files, totaling to ~ 60MB of dump files.

We created an array of all those files and load them sequentially to create the single pouchDB. When we used PouchDb v5.4.5 it completed the entire process of downloading and loading all files within 7 mins. Now we upgraded to PouchDB v6.0.7 and the same code now takes around ~ 35 mins (almost 5X the time). The network bandwidth is almost the same.

Loading individual files benchmarks to approx 55 sec (~ 1 min) per files with v6.0.7, which used to take only few seconds with v5.4.5.

We are using the latest version of Pouchdb-load (v1.4.6).

Is this a known issue? This makes me feel to rollback to older version of PouchDb. But I send a lot of time to upgrade to v6.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nolanlawson/pouchdb-load/issues/53, or mute the thread https://github.com/notifications/unsubscribe-auth/AARUwiRKkcL0FvLHJuunHrCNEyDXnR7_ks5q1dePgaJpZM4KatDN.