search

flashmob / go-guerrilla

Mini SMTP server written in golang
MIT License
2.76k stars 361 forks source link

Kinesis Firehose Processor #138

Closed danrubenstein closed 5 years ago

danrubenstein commented 5 years ago

Hi @flashmob!

This is a great project, really looking forward to working with it.

I'm wondering if you would be willing to consider a PR that would add a processor for Kinesis Firehose (or perhaps Kinesis Streams). One thing that I'd like to be able to do is put messages into a log to examine them at a later time (and perhaps by arbitrary consumers), and Firehose / Streams seems like a good way to help build that bridge.

Let me know if this is something you'd consider, and I'll start kicking around some draft code.

Thanks!

flashmob commented 5 years ago

Hi, There is a PR that will add the ability for streaming using the io.Writer interface. https://github.com/flashmob/go-guerrilla/pull/135

Would be great to have someone try it out and see how it works in practice, get some feedback and review. Willing to assist answer any questions in return.

Once you have your own processor, there is no need to merge it upstream. You can develop the processor in your own project, the use the go-guerrilla package as an API, it includes method for adding the processor and enabling it via configuration.

On Thu., 7 Mar. 2019, 10:39 Dan Rubenstein, notifications@github.com wrote:

Hi @flashmob https://github.com/flashmob!

This is a great project, really looking forward to working with it.

I'm wondering if you would be willing to consider a PR that would add a processor for Kinesis Firehose (or perhaps Kinesis Streams). One thing that I'd like to be able to do is put messages into a log to examine them at a later time (and perhaps by arbitrary consumers), and Firehose / Streams seems like a good way to help build that bridge.

Let me know if this is something you'd consider, and I'll start kicking around some draft code.

Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/flashmob/go-guerrilla/issues/138, or mute the thread https://github.com/notifications/unsubscribe-auth/AAnmP54tWJ1XJQ6-sGJqFlAcYe9Z4mrBks5vUFG4gaJpZM4biC26 .

danrubenstein commented 5 years ago

Sounds good - I'm sure it'll take me a bit to grok this, but thanks for pointing me in the right direction. I'm going to go ahead and close this for now, and pick up any specific questions I have in new issues.

Thanks!

flashmob commented 5 years ago

All right. We can communicate on the comments of the PR.

You can check out the branch with the PR and work from there. There's also an example added to the bottom of api_test.go

From the sounds of it, all you need to do is have a Firehose io.Writer and connect that up in similar way as the examples.

On Thu., 7 Mar. 2019, 13:33 Dan Rubenstein, notifications@github.com wrote:

Closed #138 https://github.com/flashmob/go-guerrilla/issues/138.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/flashmob/go-guerrilla/issues/138#event-2185964831, or mute the thread https://github.com/notifications/unsubscribe-auth/AAnmPzQlYZVwgjuFqxmPG90-T4aBhs4Qks5vUHp6gaJpZM4biC26 .

danrubenstein commented 5 years ago

Upon further reflection here - the processor model is sufficient for what I'm looking for - I may come back to the writer later on, but it's not super pressing for me.

Thanks!