search

couchbase / go-couchbase

Couchbase client in Go
https://godoc.org/github.com/couchbase/go-couchbase
MIT License
321 stars 92 forks source link

cbdatasource with options.SeqEnd, same mutations over and over #70

Open mschoch opened 8 years ago

mschoch commented 8 years ago

When I set the options.SeqEnd to a set of vbucket high-water marks, I end up getting the same mutation delivered to me over and over again.

Also, is there any callback/notification that the configure end has been reached and things can be shut down?

cc @steveyen

mschoch commented 8 years ago

@steveyen I've uploaded the program which can reproduce the problem:

https://github.com/couchbaselabs/cb2bleve

Uncomment these lines:

https://github.com/couchbaselabs/cb2bleve/blob/master/cb2bleve.go#L40-L48

Then run with a bucket containing a single document. I see DataUpdate() method get called repeatedly for the single key.

steveyen commented 8 years ago

thanks @mschoch

I made some changes to cb2bleve so that it implements the SetMetaData/GetMetaData callbacks in a "barely good enough" way, which should be helpful.

But, the cbdatasource is still at this point handling a stream-end DCP message incorrectly/inefficiently. That is, the DCP mutations will stop, so from the app's perspective the functionality looks right. But, underneath the hood cbdatasource will next inefficiently go into a loop of trying to refresh cluster information again and again, and unfortunately consume lots of CPU.