search

elliotekj / off_broadway_memory

A Broadway producer for an in-memory buffer
Apache License 2.0
6 stars 4 forks source link

Non-blocking push #4

Open utopos opened 2 months ago

utopos commented 2 months ago

Hello Team,

first of all: great job on this library!

I wanted to ask (not sure whether this is an "issue") whether non-blocking "push" to the Buffer Genserver would make sense?

For example, I am writing an analytics for a url shortening server: every time a url gets resolved from hash (ex my.ly/xyc -> www.onet.pl), I would like to push to my Broadway pipeline some metadata (time of the resolution, browser headers). I would not like to make my analytics queue genserver (Buffer) to block the resolution action. Would "cast" version of the "push" make sense in this situation?

Best, Kuba

elliotekj commented 2 months ago

Hey @utopos,

Yes, there's definitely room for this. I would add a push_async/2 type function to Buffer rather than replacing the functionality of the current push/2.

Let me know if you want to pick this up; if not, I'll get it in this week.

Cheers, Elliot

utopos commented 2 months ago

Hi Elliot,

Thanks for a quick reply. Yes I can definitely do it. How shall we proceed?

Best, Jakub

On Mon, Aug 12, 2024, 15:57 Elliot Jackson @.***> wrote:

Hey @utopos https://github.com/utopos,

Yes, there's definitely room for this. I would add a push_async/2 type function to Buffer rather than replacing the functionality of the current push/2.

Let me know if you want to pick this up; if not, I'll get it in this week.

Cheers, Elliot

— Reply to this email directly, view it on GitHub https://github.com/elliotekj/off_broadway_memory/issues/4#issuecomment-2283435100, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFP6643LUBAHZ674ORJR5WLZRB2IVAVCNFSM6AAAAABMK4YSNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBTGQZTKMJQGA . You are receiving this because you were mentioned.Message ID: @.***>

elliotekj commented 2 months ago

Perfect!

You might want to move the existing bulk push logic https://github.com/elliotekj/off_broadway_memory/blob/main/lib/off_broadway_memory/buffer.ex#L53-L60 to a new function as it’ll be reusable between the call and cast handlers.

As with the existing push function, make sure that both single messages and lists of messages are supported.

If you could then just make sure the your new function is spec’d, documented and tested that would be great.

Once you’re happy, open an PR and we’ll run through it, get it merged and publish a release.

Let me know if you have any questions :)

On 12 Aug 2024, at 11:20, Kuba @.***> wrote:

Hi Elliot,

Thanks for a quick reply. Yes I can definitely do it. How shall we proceed?

Best, Jakub

On Mon, Aug 12, 2024, 15:57 Elliot Jackson @.***> wrote:

Hey @utopos https://github.com/utopos,

Yes, there's definitely room for this. I would add a push_async/2 type function to Buffer rather than replacing the functionality of the current push/2.

Let me know if you want to pick this up; if not, I'll get it in this week.

Cheers, Elliot

— Reply to this email directly, view it on GitHub https://github.com/elliotekj/off_broadway_memory/issues/4#issuecomment-2283435100, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFP6643LUBAHZ674ORJR5WLZRB2IVAVCNFSM6AAAAABMK4YSNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBTGQZTKMJQGA . You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/elliotekj/off_broadway_memory/issues/4#issuecomment-2283592811, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALZW5AXCLLA3UILTU7VZOTZRCD7RAVCNFSM6AAAAABMK4YSNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBTGU4TEOBRGE. You are receiving this because you commented.