search

couchbase / sync_gateway

Manages access and synchronization between Couchbase Lite and Couchbase Server
https://www.couchbase.com/products/sync-gateway
Other
447 stars 138 forks source link

SG-Replicate causes index out of range #2587

Closed b-smets closed 7 years ago

b-smets commented 7 years ago

Sync Gateway version

We are running Sync Gateway v1.4.0 (CE)

Description

In an attempt to deal with a massive pile-up of obsolete attachments (see https://github.com/couchbase/sync_gateway/issues/1648) we decided to try and replicate the bucket into a new bucket, hoping it would take online live documents and attachments resulting in a far smaller bucketsize. However, when running the one-shot replication as per your documentation page (using the curl generated by your swagger docs) we end up with the following:

Curl command: curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{ "source": "sourcebucket", "target": "http://targetip:4985/targetbucket" }' 'http://localhost:4985/_replicate'

Expected:

The replication completes fine and the new bucket contains all documents of the old bucket.

Observed:

After running for about 20 seconds we end up with the following error which terminates the Sync Gateway service:

panic: runtime error: index out of range
goroutine 8059 [running]:
panic(0xa84cc0, 0xc4200100f0)
#011/usr/local/go/1.7.4/go/src/runtime/panic.go:500 +0x1a1
github.com/couchbaselabs/sg-replicate.ReadBulkGetResponse(0xc420769680, 0xc4207a5de0, 0xc42af10260, 0xc42071a828, 0x8, 0x0, 0x0)
#011/home/couchbase/jenkins/workspace/sgw-unix-build@2/1.4.0/community/godeps/src/github.com/couchbaselabs/sg-replicate/synctube.go:768 +0x150a
github.com/couchbaselabs/sg-replicate.Replication.fetchBulkGet(0xc4207f2ae0, 0x20, 0xc420c6c880, 0xc42071a828, 0x8, 0x0, 0x0, 0x0, 0xc420c6c800, 0xc42097c197, ...)
#011/home/couchbase/jenkins/workspace/sgw-unix-build@2/1.4.0/community/godeps/src/github.com/couchbaselabs/sg-replicate/synctube.go:408 +0xb0e
created by github.com/couchbaselabs/sg-replicate.stateFnActiveFetchRevDiffs
#011/home/couchbase/jenkins/workspace/sgw-unix-build@2/1.4.0/community/godeps/src/github.com/couchbaselabs/sg-replicate/replication_state.go:159 +0x432

Is there any way to work around this issue other than writing a custom application and doing this manually?

adamcfraser commented 7 years ago

This has been fixed in the latest Sync Gateway, but didn't make it to 1.4.1. If you want to pick the change, you can either build from master, or wait for upcoming 1.5.0 DP.

https://github.com/couchbase/sync_gateway/issues/2381

tleyden commented 7 years ago

@b-smets you can grab the latest master branch build from here:

http://latestbuilds.hq.couchbase.com/couchbase-sync-gateway/1.5.0/1.5.0-380/

tleyden commented 7 years ago

@b-smets sorry, correction, I don't believe the 1.4.1-3 build has that fix. Updating the comment to remove that link

nbrys commented 7 years ago

Do you have a timeframe when 1.5 will be released?

adamcfraser commented 7 years ago

1.5 developer preview is scheduled to be released next week.