search

couchbase / sync_gateway

Manages access and synchronization between Couchbase Lite and Couchbase Server
https://www.couchbase.com/products/sync-gateway
Other
447 stars 138 forks source link

sg-replicate is creating new local document for any change in sg-replicate configuration #2116

Closed ArihantRk closed 4 years ago

ArihantRk commented 8 years ago

I am using below sg-replicate configuration "replications":[ { "replication_id":"continuous-147-144", "source": "ptxdata", "target": "http://10.0.29.144:4985/ptxdata/", "changes_feed_limit":50, "continuous":true }]

any change in configurations(we have changed "changes_feed_limit" value), sg-replicate is creating new _local documents on remote site and replication is starting from sequence number =0

Is this normal behavior? if so , it will results delay in replication if we have more number of documents. though replication ignores the documents replication with rev_diff but results unnecessary reads to local cluster. any tuning in parameters results in more delay for latest documents replications.

tleyden commented 8 years ago

Yes, it's expected behavior due to this code: https://github.com/couchbaselabs/sg-replicate/blob/master/synctube.go#L141-L175

However, looking at the upstream code, it's actually a bug, since it should not be taking the changes_feed_limit into account.

ArihantRk commented 8 years ago

Thanks for quick reply @tleyden , when sync gateway 1.4 is expected to release.

tleyden commented 8 years ago

We have a conference coming up on nov 7-9 (can you make it?). I know the 1.4 release will be after that, but don't have any more info

ArihantRk commented 8 years ago

yup thanks@tleyden , if i am not wrong, i think its nov 8-10. will try to join on live stream.

ddash1 commented 7 years ago

@djpongh Is this bug descoped from 1.4?

ddash1 commented 7 years ago

@djpongh @tleyden Any update when this bug is going to be fixed?

ddash1 commented 7 years ago

@tleyden Can you please update on this issue fix plan? this causes unnecessary full replication just on a change of hostname and or any of the replication {} block parameters. Practically these parameters can change quite often.

adamcfraser commented 7 years ago

@ddash1 It's expected that changes to some of the replication{} block parameters - including hostname in source/target - gets identified as a different replication. This matches the CBL replication behaviour, and is needed to differentiate between replications targeting different hosts. If the goal is to be able to switch replication between multiple hosts in a single SG cluster, you'll currently need to target a load balancer fronting that cluster. An enhancement to support defining multiple source/target hosts in the replication{} block might be feasible, but would be a separate enhancement request.

The fix described above is just to remove changes_feed_limit from the set of properties used to identify the replication checkpoint. This isn't currently targeted for a release.

ddash1 commented 7 years ago

@adamcfraser We are good with the changes_feed_limit exclusion only. That's the one causing issue on our setup currently.

A unique scenario if the host is switched for the source then it is re-replicating full - let me explain. I have a host that has a file system mapped and cannot be extended. I move the filesystem to another host, All data intact, and extend it. I do not want the replication to start again from the beginning as all data is already there.

adamcfraser commented 4 years ago

Closing based on age.