Synch issue with large no of documents

[shettyabhishek]

We are facing an issue where larger number(~500) of documents not getting synched properly. As per our requirement we need to manually upload all the documents to the couchbase server via Synch Gateway using REST API(http://localhost:4984/default/_bulk_docs) . This documents are getting uploaded successfully and we can see all the documents in Synch Gateway(http://localhost:4984/default/_all_docs?include_docs=true) and in the couchbase server admin console with proper id and revision and should sync to the Couchbase Lite. To get the synched data from Couchbase lite we are using the same javascript code from Phonegap Android TodoLite(index.js). Only changes we have done in the js file is removed the dependency on the moustache js and replaced it with jQuery, also added view named session. We observe that the number of documents that are fetched are varying each time, so the number of documents shown up on the screen varies each time. The limit of documents fetched as seen in the sync logs is 50. (Attached a snap shot of the sync logs) . Is there a way this limit be changed ? We are also not being able to make out in what frequency or time interval are the sync calls triggered. We also observed that the sync time is very high, it takes a lot of time to fetch the documents.

Sample JSON pushed to Couchbase server. Have pushed around 300+ such documents to the server.

{"ID":"2907","channels":["public"],"abbreviation":"BB2907","title":"From “A” to “BSM” in 12 weeks: How one insurance company improved service visibility and proactivity with HP Business Service Management","abstract":"Delivering a fully integrated solution of 10 business service management (BSM) applications in 12 weeks is no small feat. Such a task requires careful planning, rigorous organization, and dependable software. The HP BSM suite—a tried and tested solution with effective multi-level integration—provided an insurance company the means to deliver within these tight time limitations. Attend this session to discover how HP BSM helped the company realize important business benefits such as reduction in downtime due to better visibility of the entire IT infrastructure; more efficient operations bridge, with a lower mean time to repair; improved visibility across the IT infrastructure, enabling better proactivity; and better impact analysis of infrastructure outages on services.","timesOffered":"1","type":"Business Breakout","status":"Approved","length":"60","created":"2014-01-29 07:18:43","modified":"2014-06-30 17:06:36","eventConference":"no","published":"true","sessionTime":{"ID":"615","room":"Lando 4202","capacity":"260","registered":"87","full":"false","date":"2014-06-12","time":"9:00","length":"60","dateMilliSec":1402563600000},"Content_Combined_Subtrack":"Business Service Management","Content_Tracks":["HP IT: Best practices and lessons learned","Software"],"Reg_Job_Level":["IT Manager","VP/Director (IT or LOB)"],"Reg_Job_Function":["Infrastructure","Operations"],"Content_Presented_By":"Partner","Session_Allocation_Owner":"SW: Business Service Management","Session_Allocation_Group":"Software","speaker":[{"personID":"38723","fullName":"Monica Benjamin","jobTitle":"ITOM Track Manager","company":"HP","roles":"HP Speaker","bio":"","photo":[]},{"personID":"47544","fullName":"Jim Marston","jobTitle":"Project Manager / Principal Consultant","company":"Innovise ESM","roles":"Partner Speaker","bio":"","photo":[]}]}

Have attached the snap shot of all the changes that we have

1. The view created by us :
   
2. Once the syncing completes we call this below function "dbChanged", the DOM manipulation is implemented using jQuery. Basically as seen above we have created a view called "sessions", so any data that is picked up is from the "sessions" view as seen below:
3. From the index.html perspective we have the below div to which we are appending the records / documents
4. Our syncJson.json looks like this :
5. Sync logs

[ajres]

@shettyabhishek

There were some issue in sync_gateway 1.0.2 that were fixed in the latest release 1.0.3.

If you are not already using 1.0.3 releases are you able to upgrade sync_gateway and couchbase lite to 1.0.3 and retest.

Andy

[ajres]

@shettyabhishek

I also noticed that you seem to be doing a one shot replication rather than a continuous replication, a continuous replication would keep requesting the subsequent batch of 50 docs until all docs had been sync'd and then it would wait for new updates.

Andy

[shettyabhishek]

Hi Andy , Thanks a lot for your reply. I will cross check about the version. But with respect to replication I am doing continuous replication. I cannot put the complete code so just putting the snippet . I am creating a function where I create push and pull objects with continuous property set to true. These objects are used to do the replication .

Code snippet : var remote = { url : config.site.syncUrl }, push = { source : appDbName, target : remote, continuous : true }, pull = { target : appDbName, source : remote, continuous : true },

pushSync = syncManager(config.server, push),
pullSync = syncManager(config.server, pull)

[jchris]

It would help to see logs from your Sync Gateway installation. Any HTTP errors could give us hints about what's going on.

Also it may be that you need to enable Anonymous access: http://developer.couchbase.com/mobile/develop/guides/sync-gateway/administering-sync-gateway/authorizing-users/index.html#anonymous-access

Or maybe you are using the admin API to create users and sessions?

[jchris]

Are all your documents tagged with "channels":["public"] or do some have another set of channels?

[shettyabhishek]

Thanks a lot for your reply and sorry for a delayed response from my end :) .

@jchris , Yes we are using the channel as public. I will verify by enabling the anonymous access to check http errors if any .

@ajres
Andy , I tried using the latest Couchbase Lite android jars 1.0.3-3 ,1.0.3-7 , 1.0.3-9 . But the problem with that is, it does not have "com.couchbase.lite.android.AndroidContext;" class and throws errors.

[tleyden]

@shettyabhishek that sounds like a possible packaging issue w/ couchbase lite android.

Can you file an issue here?

https://github.com/couchbase/couchbase-lite-android

[jessliu]

@shettyabhishek can you confirm what sync_gateway you are using, as well as file another issue in Android? Also, can you send us the complete logs for this to see if this really is a sync_gateway issue?

[ajres]

Closing this out as we can't reproduce without further info from the OP.