proposal: remove consul support / leadership elections

moov-io / achgateway

Payment gateway enabling automated ACH operations in a distributed and fault tolerant way.

https://moov-io.github.io/achgateway/

Apache License 2.0

54 stars 19 forks source link

proposal: remove consul support / leadership elections #174

Closed adamdecaf closed 1 year ago

adamdecaf commented 1 year ago

At Moov we've stopped running achgateway with consul support and others have deployed without consul as well. The consul implementation has several known issues and does not work as expected.

File consistency across instances is not guaranteed with consul. Instances can isolate/upload different sets of ACH files.
Files are not stored in consul which is often assumed to be the case.
Users can be confused to share one Persistent Volume or have unique volumes per instance.

jrnt30 commented 1 year ago

We are not currently using it either. We had initially but ran into a few of those issues outlined above and ended up removing it from our infrastructure. In the current state, I do feel like it could potentially lead to data loss/unexpected file processing semantics if folks are moving quickly.

Personally supportive of deprecating/removing Consul for the time being and either leaning into it and using it as the distributed file store as well or looking to a different HA solution all together.