dump(): TypeError: Cannot use 'in' operator to search for '_id'

Artanis commented 9 years ago

I'm trying to use this in the browser, having found createWriteStream() attached to PouchDB instances (I assume instructions in Readme are for node, as they place it at fs.createWriteStream()).

When I call dump, I get TypeError: Cannot use 'in' operator to search for '_id' in {"version":"1.1.1","db_type":"idb","start_time":"2015-03-21T07:25:34.570Z", ...} at pouchdb.replication-stream.js:10883

A little poking through the code revealed that that line is using the in operator on a string, and that several functions up the stack, JSON.stringify() had been used.

I have replicated this in the following code, though I may be doing something wrong:

<!DOCTYPE html>
<html>
  <head>
    <!-- Using latest release for both -->
    <script src="pouchdb.js"></script>
    <script src="pouchdb.replication-stream.js"></script>
  </head>
  <body>
    <script>
      var bug = function () {
        var db = new PouchDB('test');

        db.put({
          "_id": "testing/broken-write",
          "type": "testing",
          "name": "broken-write"
        });

        var ws = db.createWriteStream();

        db.dump(ws, { batch_size: 1})
          .then(function (res) {
            console.log(ws);
            console.log(res);
        });
      };

      bug();
    </script>
  </body>
</html>

nolanlawson commented 9 years ago

Can you point to the line in the source code that causes this?

Artanis commented 9 years ago

Sorry, should have done that to begin with. Everything I'm doing is in the browser, though, so that's all I can inspect.

The JSON.stringify() call I found is at pouchdb.replication-stream.js:50, in a .then() callback function. It looks like index.js:49.

Chrome stack trace: WriteStream._write (pouchdb.replication-stream.js:10883) doWrite (pouchdb.replication-stream.js:10291) writeOrBuffer (pouchdb.replication-stream.js:10281) Writable.write (pouchdb.replication-stream.js:10248) (anonymous function) (pouchdb.replication-stream.js:50)

There's another at pouchdb.replication-stream.js:7415, which causes a similar error when attempting to write out the actual documents (I removed the stringify call at line 50 to get to this). As far as I can tell, this line does not appear elsewhere in the repository, and may be from a dependency [edit ldjson-stream/index.js:27].

Removing both stringify calls removes the error ... but I haven't figured out how to get at the data that's been written to the stream yet (the code I posted in the issue body is almost exactly what I wrote to start learning how to use pouchdb-replication-stream), so I can't tell you if this 'solves' the problem, or just replaces it with a different, silent error.

nolanlawson commented 9 years ago

That's really weird. What's the header object when it occurs? Can you provide a reproducible test case? (Sorry, but I don't have time to debug this at the moment.)

nolanlawson commented 9 years ago

Also yeah, it's very likely that this is a bug in how browserify is browserifying Node streams for the browser. If you cannot reproduce the bug in Node, then you may want to look at stream-browserify.

calvinmetcalf commented 9 years ago

_write is often the base of an error because that is often the top of the stack, this case we are writing a string here but then here we are assuming it has an '_id' property. Guess what happens in node if you look for a string inside another string

'bar' in 'baz'
TypeError: Cannot use 'in' operator to search for 'bar' in baz
    at repl:1:7
    at REPLServer.defaultEval (repl.js:124:27)
    at bound (domain.js:254:14)
    at REPLServer.runBound [as eval] (domain.js:267:12)
    at REPLServer.<anonymous> (repl.js:277:12)
    at emitOne (events.js:77:13)
    at REPLServer.emit (events.js:166:7)
    at REPLServer.Interface._onLine (readline.js:195:10)
    at REPLServer.Interface._line (readline.js:534:8)
    at REPLServer.Interface._ttyWrite (readline.js:811:14)

it doesn't print the quotes around the string you looked in, meaning I probably need a check for a string here

Artanis commented 9 years ago

@nolanlawson, the header object is {"version":"1.1.1","db_type":"idb","start_time":"2015-03-24T23:55:40.925Z","db_info":{"doc_count":18468,"update_seq":18468,"idb_attachment_format":"base64","db_name":"test","auto_compaction":false}}, just before it becomes a string for writing. As earlier, it is still a string when it hits PouchStream and has the in operator used on it.

Regarding stream-browserify, the code in that project is nearly identical to the node stream module. It seems unlikely that browserify is inserting JSON.stringify() calls, but I am not familiar with the inner-workings. I managed to I track down the second instance I noted in ldjson-stream/index.js:27.

@calvinmetcalf, The usage example for PouchStream suggests that objects should be written, rather than strings. Are strings valid arguments for write() and this is instead a bug in PouchStream?

As for the error message, I'm seeing that same lack-of-quoting behavior in Chrome DevTools, too. It's confusing until you realize what is happening. A quick search pulls up a stackoverflow question, and a post to the Backbonejs Google Group, though neither of those touched on the elephant in the room: errors should be printing unambiguous values. Any idea where something like this gets reported?

calvinmetcalf commented 9 years ago

should be fixed by version 0.4.1 of pouch-stream

nolanlawson commented 9 years ago

@Artanis please reopen if still not fixed

Artanis commented 9 years ago

Sorry for the delay. Was unexpectedly busy, myself.

This looks to be fixed.

pouchdb-community / pouchdb-replication-stream

dump(): TypeError: Cannot use 'in' operator to search for '_id' #28