_local document has been replicated

[acutetech]

In my PouchDB days I used a _local document to store soem settings. My understanding was that documnts with an id starting with "_local" would stay on the device and not be sent to the server. This does not seem to be the case with sync-android.

In my Cloudant dashboard I see the following document, which I think should not have arrived there:

{
 "id": "_local/role",
 "key": "_local/role",
 "value": {
  "rev": "28-207d4e0441b94e8ca130d1ec338a9d43"
 },
 "doc": {
  "_id": "_local/role",
  "_rev": "28-207d4e0441b94e8ca130d1ec338a9d43",
  "role": "receiver"
 }
}

I have checked: if I change this document on my local device then do a push-sync, it changes at the server. Unlike the other documents, I do not seem to be able to edit this document at the server, so I have not (yet) been able to conduct the reverse experiment - to see if it is replicated from the server to the device.

In the database created by PouchDB, there is no sign of the _local/role document.

Am I missing something here, or is this a bug?

[ricellis]

sync-android doesn't expose API methods for writing _local documents on the Database interface. Prefixing an id with _local is insufficient to get the behaviour of a CouchDB local document hence the reason it slips through into the replication.

There are internal methods used to write local documents for replication checkpoints only, but they rely on a different document class (LocalDocument vs DocumentRevision) and get written and read via different code paths to a separate table in the underlying SQL database.

So there isn't really a replication bug, the real local documents in sync-android don't get replicated, but it ought to be more clear both in documentation and probably via exception that writing a DocumentRevision with a _local prefix won't work.

[acutetech]

Hmmm.... So is this LocalDocument functionality available to users, or only for your internal use? I started along these lines:

            String id = "_local/role";
            Map<String, Object> map  = new HashMap<String, Object>();
            map.put("role", payload);
            map.put("androidId", mAndroidId);

            DocumentBody body = DocumentBodyFactory.create(map);
            LocalDocument localDoc = new LocalDocument(id, body);

            // Now place it in the local database
            LocalDocument created = mDocumentStore.database().create(localDoc);

which starts OK, but fails at the last line.

Are there user-facing CRUD methods for LocalDocument? If not I guess I make use of other Android storage options.

(BTW, I now seem to have a _local/role document on the server which I can't delete from the Cloudant dashboard. Is it safe there? Is there any chance it will be pushed down to devices?)

A final thought: is there any danger if I continue to use a document with a "_local" id? It does seem to work (locally), but gives conflicts at the server....

[ricellis]

Are there user-facing CRUD methods for LocalDocument?

Sorry if I wasn't clear, there are not at this time. I suspect it was an oversight that the LocalDocument class itself wasn't moved into the internal package.

I now seem to have a _local/role document on the server which I can't delete from the Cloudant dashboard. Is it safe there? Is there any chance it will be pushed down to devices?

Interesting, I've managed to reproduce that situation locally and I think it is likely a bug in CouchDB. It seems that if a _local prefixed document with a _rev is written via _bulk_docs with "new_edits":false (which is how a replicator inserts documents) then it gets written into the main document tree instead of the local document tree. So it isn't really being treated as a local document by CouchDB, but at the same time any attempt to DELETE or GET that document has the _local prefix which sends it to the local document tree (where it does not exist). So it appears impossible to delete (at least via the HTTP API).

I believe it will be pushed down to devices because it appears in the _changes feed and is being treated by CouchDB as a normal (i.e. not local) document.

I think you could possibly get rid of it by doing:

1. Create a new Cloudant DB (let's call it b)
2. Do a filtered replication from your original Cloudant DB (a) to b excluding the _local docs
3. Delete the DB a
4. Create a new DB a
5. Replicate from b to the new a
6. Delete DB b

Alternatively, contact Cloudant support, I believe they would be able to delete it via other (internal) means.

is there any danger if I continue to use a document with a "_local" id?

Yes, for the above reason this is a bad idea. Although it looks like a local doc it isn't and obviously it exposes some rough edges!

Also re-opening this as we, at the least, need to do some doc updates.

[ricellis]

FYI I have raised https://github.com/apache/couchdb/issues/1628 related to this.

[tomblench]

@acutetech #591 has merged which now allows you to use the existing public API methods on Database, namely read, contains, create, delete, to manipulate local documents in the manner you originally described.

You can use a snapshot build to try this out, or wait until the next official release.

[acutetech]

When the problem surfaced I stopped using _local documents and switched to using Android's SharedPreferences mechanism, which seems a satisfactory alternative.

Good that you have dealt with the issue, though. I trust there will be appropriate user-facing documentation for the next release. Thanks.

[tomblench]

OK no problem.