incoming transform :: document updates are not live replicated

[nickcmaynard]

I've noticed that the act of attaching transform-pouch to a DB breaks live replication, specifically in the case where an existing document is updated. Creates are fine.

I don't know how to fix this, but I have managed to recreate within the transform-pouch test harness. I've included the test case in my local branch (see linked PR).

Test case may be over-complicated; apologies.

[nickcmaynard]

@nolanlawson Do you have any thoughts on this? Are we mis-using transform/replication somehow?

[nolanlawson]

I admit I'm having trouble understanding your example. The incoming/outgoing functions are supposed to be purely synchronous; they aren't supposed to have side effects. What kinds of real-world cases would cause this error? And what is the error exactly?

[nickcmaynard]

In the real world, we have a local DB (in browser) live replicating from a remote DB (https). Without transform-pouch, we see updates arriving as we'd expect.

With transform-pouch, however, it's a different story. Our db.transform call is as follows:

  // Monitor changes to localDB
  localDB.transform({
    incoming: function(doc) {
      console.log("INCOMING document: ", doc);
      return doc;
    },
    outgoing: function(doc) {
      console.log("OUTGOING document: ", doc);
      return doc;
    }
  });

This function is just a precursor to our eventual function, but we're testing transform-pouch has no side-effects, first!

If db.transform() is called, we see strange behaviour where the local DB doesn't receive updated documents. We still receive new documents, but no changes to those documents are persisted.

This test case shows the issue.

[nickcmaynard]

Linked to PR (above) to show issue. No longer a single commit.

[nickcmaynard]

@nolanlawson Does my explanation make sense, or are we doing something fundamentally wrong here?

[nolanlawson]

To be honest I just don't understand your test. If you provide a fix, though, and if Travis is green, then I will accept it gladly. :)

[nickcmaynard]

Sadly, conversely I don't think I understand the code enough to fix that :-). Will see if I can find a way to get our respective understandings to meet in the middle!

Perhaps the test case could do with simplification. Or more comments. I will take a look.

[numerized]

Hi guys,

I have the same problem with crypto-pouch which I've just found out it relies on transform-pouch. I have to deactivate crypto-pouch and all live replications works. If I activate crytpo-pouch I have this strange behaviour described by @nickcmaynard

Scenario as follow:

A list of documents in cloudant encrypted with crypto-pouch and live replicated with pouchdb

Device1 One document is created. rev1

Device2 Document is appearing in the list rev1

Device1 This new document is updated rev2

Device2 Update is visible rev2

Device1 Makes as many as update as he wants it'll still update device2 revX

Device2 Try to update revX Update ok but not replicated to Device1

Device1 is still with revX Any update from now is separated and not replicated back to the device2.

If I disable the crypto-pouch it comes back to normal all replications are working.

Sample

contacts-providers.ts

var PouchDB = require("pouchdb");
PouchDB.plugin(require('crypto-pouch'));

[...]

init(details){

    this.db = new PouchDB(details.user_id+'_Contacts');
    this.db.crypto('WOWOWOW');

    this.remote = details.userDBs.main;

    let options = {
      live: true,
      retry: true,
      continuous: true
    };

    this.db.sync(this.remote, options);
}

Good luck with this one.

[nickcmaynard]

@numerized Not quite what I see, because my app doesn't make local updates to existing documents. IIRC it does however, create others. Do you see the same pattern with local updates/creates to a docB document?

[nickcmaynard]

(Full disclosure - had to abandon transform-pouch for my usecase)

[nolanlawson]

If you could provide a live test case I could still take a look at this, but based on the description here it's unclear to me what's going on.

[numerized]

HI @nolanlawson sorry got really busy on many things... I'll definitely give crypto-pouch a try in the next few weeks.

I was thinking about it and in my cas I might have found that wkwebview plugin was the reason I had the live-sync to fail from two different devices after updating the document in both sides one after the other.

I'll try it better and report

[jcoglan]

Per commentary elsewhere (e.g. #56) it's possible this is related to incoming() not being called correctly on put() calls to remote databases. We just merged a substantial overhaul of the code that resolves that problem and it would be good to know if the newly released version 2.0.0 fixes this issue.