search

Admiral-Piett / goaws

AWS (SQS/SNS) Clone for Development testing
MIT License
767 stars 144 forks source link

Optionally persisting topology (queues, topics and subscriptions) #266

Open glenjamin opened 1 year ago

glenjamin commented 1 year ago

Hello!

We current have a setup where we run a copy of our stack in a VM for dev purposes, where a number of our apps communicate via SNS and/or SQS.

In production, we configure our queue and subscription topology via terraform, and configure the IAM so that our applications are only able to send or receive messages.

In dev we run the same terraform against the fake endpoints to sort out queues and subscriptions - but as there's no persitence this gets lost every time we reboot.

Would you be open to a PR which enables a config option for saving SQS and SNS topology into a file whenever it changes, and reloading them on app startup?

I've had a look at the structure of the app, and I think this is quite doable, but would likely require moving the []Messages collection into a separate map - that would give me an easy way to only save when topology changes and to exclude the messages themselves

Admiral-Piett commented 9 months ago

@glenjamin Hm, interesting idea. Have you had any luck with just making your changes in a config file and pointing your goaws instance at that? If not, what's missing from that, that you need to see other than you having to make it yourself? That's really the only persistance we have immediately handy.

Though, I could see value in a sort of - "export my state" command into a config file, I'd want to chew on that a bit.

Let me know about the config option. Thanks for thinking it through though!

glenjamin commented 9 months ago

The main thing I'm hoping to avoid is having to repeat our topology config in multiple places, as that runs the risk of it going out of sync

Admiral-Piett commented 9 months ago

Hm, interesting. The only way we currently have to persist a particular setup is through a config file - which I what I would recommend to you in the meantime, event though, you would have to maintain both things you're right. I can see that exporting a current config as being beneficial, though I'll have to think through a design for it. We can leave it here for now.