search

pouchdb-community / pouchdb-dump-cli

Command-line tool for dumping a CouchDB/PouchDB database to a file
Apache License 2.0
84 stars 20 forks source link

RangeError: Invalid string length #7

Open davidje opened 9 years ago

davidje commented 9 years ago

Doing a dump of the NPM isaacs.iriscouch.com mirror dataset from my local couchdb replication 

# pouchdb-dump http://localhost:5984/registry > dump.txt

eventually around 6GB

[info] [<0.8699.1>] 127.0.0.1 - - GET /registry/wheels-pubsub/wheels-pubsub-0.0.1.tgz?rev=3-ceb4b05dfdc0c8f934233a8b08bfca8a 200
[info] [<0.8697.1>] 127.0.0.1 - - GET /registry/grunt-saucelabs-qunit/grunt-saucelabs-qunit-1.0.2.tgz?rev=25-9ab135f8b93da08d763ed30fbd53950b 200
[info] [<0.8688.1>] 127.0.0.1 - - GET /registry/punch-sass-compiler/punch-sass-compiler-0.0.3.tgz?rev=7-662bbe261eafdbf1a89cebc88fe87133 200
[info] [<0.8694.1>] 127.0.0.1 - - GET /registry/grunt-saucelabs-qunit/grunt-saucelabs-qunit-1.0.4.tgz?rev=25-9ab135f8b93da08d763ed30fbd53950b 200
[info] [<0.8698.1>] 127.0.0.1 - - GET /registry/grunt-saucelabs-qunit/grunt-saucelabs-qunit-1.0.1.tgz?rev=25-9ab135f8b93da08d763ed30fbd53950b 200
[info] [<0.8700.1>] 127.0.0.1 - - GET /registry/grunt-saucelabs-qunit/grunt-saucelabs-qunit-1.0.5.tgz?rev=25-9ab135f8b93da08d763ed30fbd53950b 200
[info] [<0.8712.1>] 127.0.0.1 - - GET /registry/grunt-saucelabs-qunit/grunt-saucelabs-qunit-1.1.0.tgz?rev=25-9ab135f8b93da08d763ed30fbd53950b 200
[info] [<0.8701.1>] 127.0.0.1 - - GET /registry/grunt-saucelabs-qunit/grunt-saucelabs-qunit-1.1.1.tgz?rev=25-9ab135f8b93da08d763ed30fbd53950b 200
unexpected error
{ [RangeError: Invalid string length]
  result:
   { ok: false,
     start_time: Sat Jun 27 2015 05:42:19 GMT-0700 (PDT),
     docs_read: 9250,
     docs_written: 9200,
     doc_write_failures: 50,
     errors: [ [Circular] ],
     last_seq: 9200,
     status: 'aborting',
     end_time: Sat Jun 27 2015 05:52:00 GMT-0700 (PDT) } }

Apache CouchDB 1.6.1

barbalex commented 8 years ago

I have what seems to be the exact same error:

unexpected error
{ [RangeError: Invalid string length]
  result:
   { ok: false,
     start_time: Tue Nov 10 2015 23:30:08 GMT+0100 (CET),
     docs_read: 3724,
     docs_written: 3657,
     doc_write_failures: 67,
     errors: [ [Circular] ],
     last_seq: 80116,
     status: 'aborting',
     end_time: Tue Nov 10 2015 23:31:29 GMT+0100 (CET) } }
RangeError: Invalid string length
    at Object.stringify (native)
    at DestroyableTransform._transform (/usr/local/lib/node_modules/pouchdb-dump-cli/node_modules/pouchdb-replication-stream/node_modules/ldjson-stream/index.js:30:19)
    at DestroyableTransform.Transform._read (/usr/local/lib/node_modules/pouchdb-dump-cli/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:184:10)
    at DestroyableTransform.Transform._write (/usr/local/lib/node_modules/pouchdb-dump-cli/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:172:12)
    at doWrite (/usr/local/lib/node_modules/pouchdb-dump-cli/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:237:10)
    at writeOrBuffer (/usr/local/lib/node_modules/pouchdb-dump-cli/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:227:5)
    at DestroyableTransform.Writable.write (/usr/local/lib/node_modules/pouchdb-dump-cli/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:194:11)
    at PouchDB.WritableStreamPouch.api._bulkDocs (/usr/local/lib/node_modules/pouchdb-dump-cli/node_modules/pouchdb-replication-stream/lib/writable-stream.js:45:16)
    at PouchDB.<anonymous> (/usr/local/lib/node_modules/pouchdb-dump-cli/node_modules/pouchdb/lib/adapter.js:809:15)
    at PouchDB.<anonymous> (/usr/local/lib/node_modules/pouchdb-dump-cli/node_modules/pouchdb/lib/deps/adapterFun.js:49:21)
2015-11-10_23-30-07_artendb_dump.txt
 > Uploading "/home/alex/2015-11-10_23-30-07_artendb_dump.txt.tar.gz" to "/2015-11-10_23-30-07_artendb_dump.txt.tar.gz"... DONE

this is in the executed bash file:

FILENAME=$(date +"%Y-%m-%d_%H-%M-%S_artendb_dump.txt")
FILENAME_GZ=$FILENAME.tar.gz
pouchdb-dump http://localhost:5984/artendb > $FILENAME

couchdb 1.6.1 on Ubuntu 14.04.3 LTS

nolanlawson commented 8 years ago

I welcome anybody to try to fix this 200-line project, but I currently don't have time to look into this; sorry.