search

cloudwan / gohan

Gohan is an API Gateway Server written by Go that makes it easy for developers to create and maintain REST-style API
http://www.slideshare.net/natiueno/gohan-61170476
Other
109 stars 44 forks source link

CW-992: SyncWriter watches on ETCD instead of a timer #715

Closed p-kozlowski closed 5 years ago

p-kozlowski commented 5 years ago

In case of HA environments, we may wait up to 2 x eventPollingTime if a transaction is committed outside a cluster leader. Here we remove the recentlySynced "optimization" at the cost of 3x times for DB queries on events table.

Polling parameters are now exposed through the config file to make fine-tuning possible.

p-kozlowski commented 5 years ago

Reviewers, I'm going to give this change a nice round of end-to-end testing, I'll let you know if everything went fine. In the meantime, I don't expect any significant code changes, so you're reviews are welcome.

p-kozlowski commented 5 years ago

Maybe instead of that we could use ETCD for synchronization.

As discussed with @zimnx and @jkmar this will add some load on ETCD: 1 Update() per each transaction generating at least 1 event. This translates to CUD requests + cron handlers, reads would not be affected. Let me confirm with @nvtkaszpir if there's enough performance space for it.

p-kozlowski commented 5 years ago

Let me confirm with @nvtkaszpir if there's enough performance space for it.

Confirmed, this change should not be noticeable in terms of ETCD performance.

p-kozlowski commented 5 years ago

Reviewers, I implemented changes requested by @jkmar. For now I marked this PR as WIP, I'll remove the tag once I'm done with end-to-end testing.

p-kozlowski commented 5 years ago

Reviewers, I'm done with functional testing, everything works fine. I'll leave a loaded test environment overnight to (once again) verify no resource leaks. I think we should be good to merge this PR tomorrow.