Replicator replicating old design docs

[nickkitto]

Hello, Up until recently the replicator was replicating the most current version of the design doc. However after a recent push with couchapp it is now replicating an old version of the design document. The console during the push seems to indicates the document was pulled

TDPuller[redacted] inserting 1 revisions...
09-03 12:03:54.185: V/TDDatabase(3915): TDPuller[redacted]: Setting lastSequence to 619 from( 618)
09-03 12:03:54.185: W/TDDatabase(3915): TDPuller[redacted] finished inserting 1 revisions
09-03 12:03:54.300: V/TDDatabase(3915): TDPuller[redacted] STOPPED
09-03 12:03:54.300: V/TDDatabase(3915): TDPuller[redacted] checkpointing sequence=619

This issue persists over a restart and an uninstall/reinstall. I am being shown revision 53 on the devices database while the current revision is 106. If i push the couchapp to the database again the revision on the server increases, when the device replicates it prints to the console similar to above, but the design doc is not updated. Viewing the url on the server shows the correct revision. I am replicating the database with a non continuous replication using ektorps StdCouchDbInstance.replicate method. Using ektorps replicateFrom method.

Another thing I notice is that when I do a push replication to the server it seems to try to send the document back, even though I don't believe it exists

09-03 12:21:05.000: V/TDDatabase(4867): com.couchbase.touchdb.replicator.TDPusher$2@40f846d0: Sending [{_design/jobs #106-cb42d10fef9d20a4a02a67269ea3b8f1}, {account_info #5-d1e67460-2303-41b3-aa93-b91077d10c83}]
09-03 12:21:05.620: V/TDDatabase(4867): Buffer size is 128
09-03 12:21:05.850: V/TDDatabase(4867): Buffer size is 128
09-03 12:21:14.970: V/TDDatabase(4867): com.couchbase.touchdb.replicator.TDPusher$2$1@40ef5238: Sent [{_design/jobs #106-cb42d10fef9d20a4a02a67269ea3b8f1}, {account_info #5-d1e67460-2303-41b3-aa93-b91077d10c83}]

If I try to view the design doc with this url couchapp/_design/jobs/?rev=106-cb42d10fef9d20a4a02a67269ea3b8f1 on the device I do appear to get the latest design doc. So it appears that the revision for this design document is being saved correctly, but not being retrieved for some reason. Let me know if you need any logs/results from queries.

[mschoch]

It would be useful to see what revision is returned by the device, then perform a replication where you expect the revision to change, and then run the first query again showing the wrong version. With logs for the whole sequence of operations as well.

[nickkitto]

I copied the sqlite database off the device and inspected it a bit. The revs tables for the design document have all the revisions from the server (106 and 53 and everything in between). I did notice that both the 53 and the 106 rev have the current flag ticked, which is probably the issue. I am trying to do a fresh replicate now and see if the same revision is marked.

[nickkitto]

Ok I know what the issue is now and why it gets marked at 53. But I do not know why it does not resolve itself.

The application itself has a canned couchapp that is replicated into several databases. The revision of this is currently 53. I replicate it to each of the customers databases when I perform significant updates.

I had difficulty copying from a canned touchdb database and exposed the canned couchapp to imitate this feature. When the application is run and the database doesn't exist it replicates from this canned database. This is usually up to date with all the databases except one. I also have another 'testing' database that I push to seperately. The android application replicates with the canned database and then replicates with the users database.

The testing database has an updated version of the couchapp (the 106 revision). The control flow during a first run of the application is it replicates with the canned database and marks the revision as the current (53). It will then replicate with the testing database and then marks the revision as the current (106). Here I would expect the old revision to be marked as not current, however this does not seem to occur.

Using a similar method with couchdb android didn't cause any problems. I used to use a canned database to be an initial seed. However I never used to replicate with the url. The reason I am not copying from a file is the issue of creating a .touchdb file/attachments folder with updates instead of using a .couch file, and issues of copying the database from within the apk (due to issues with context).

[nickkitto]

Hey Marty, I seem to have a similar issue still occuring with conflicts. The issue I'm facing is that if multiple users edit the same document and cause a conflict, then one user updates agian increasing the revision number the conflict is not resolved. In regular couchdb the highest revision number should win. However this does not seem to be the case (at least in view results, the resulting document gets emitted twice). Two documents should never have the current flag set to be 1 in the sqlite database as far as the docs seem to indicate (http://guide.couchdb.org/draft/conflicts.html).

[mschoch]

So there are 2 issues here.

1. "current" in the TouchDB database is not the same thing as "winning". The design of TouchDB is such that it is expected for multiple rows to be "current" when there is a conflict. I was confused about this as well, but that is how Jens built it. It's really just a terminology thing.
2. You are correct, that even if there is a conflict, we should only ever index one version of the document in the view. Just looking at the code I think there may be a bug in detecting conflicts when indexing.

[snej]

"current" in the TouchDB database is not the same thing as "winning".

Right. The column named 'current' should more accurately be called 'leaf' because it just denotes revisions that have no children. These are the candidates for the current 'winning' revision.

Just looking at the code I think there may be a bug in detecting conflicts when indexing.

I remember some relatively-recent bug fixes in that area, which may not have made it into Android yet.

[nickkitto]

Ah right, I was taking 'current' to mean the revision that was returned. Makes more sense now. Further testing shows me that the document gets emitted twice when it is a part of a view (once with each of these revisions), but you get presented with the expected doc when you retrieve it (highest rev #).

[tleyden]

https://github.com/couchbase/couchbase-lite-android/issues/33