Listening for events based on extras

[barrelltech]

Hey! Super cool library. Would be awesome if I could subscribe beyond just an id, but specifically to the extra information. I.e. if you are viewing a post, you could:

# setup 
{EctoWatch, watchers: [{MyApp.Blog.Comment, :interted, extra_columns: [:post_id]}]}

# client
EctoWatch.subscribe(MyApp.Blog.Comment, :inserted, nil/true/:all, post_id)

For my use cases it would cut down on a ton of messages sent.

PS: completely aside, would be super neat if there was a debounce parameter... I saw mentions about config for triggers so just thought I'd drop that one in while I'm here :)

[cheerfulstoic]

Hey, thanks for the issue!

Ok, so making sure I understand: you want the watcher to still send out all inserts, but you'd like the client to be able to subscribe only to comments of a specific post_id value, right?

My first thought is that you could create a GenServer which subscribes to the EctoWatch notifications and then distributes them via a separate PubSub.broadcast.

That said, it would be nice if this "just worked". I'll take a look and see what I can come up with 👍 The API would be the main thing I'd want to figure out. Like you might do:

EctoWatch.subscribe(MyApp.Blog.Comment, :inserted, :post_id, the_id)

But right now, under the covers, there's ore just two PubSub topics being published:

• All updates for all records
• All updates for a specific record ID

So I could imagine a couple of ways to deal with this:

• Require application devs to setup watchers so that they specify which columns is the "identifier" when messages are sent out
• Have watchers send out on the two topics above, but also with one topic for every column specified in extra_columns

I think I'd lean toward the second because:

• I don't think there is a cost to broadcasting to a topic when there aren't any subscribers (so it's ok if most of the time people aren't subscribed to the extra columns)
• If you had multiple watchers for the same schema then all of those watchers would be sending to the "all updates for all records" topic
• I think it would take the least amount of change to the library 😅

But still, I'm going to think on it a bit. It's a great point, though, thanks!

[barrelltech]

If I subscribe to comments:

EctoWatch.subscribe(MyApp.Blog.Comment, :inserted, extra_columns: [:post_id])

Then add a listener

def handle_info({:inserted, MyApp.Blog.Comment, _, %{post_id: "post_id"}}, socket) 

~Would you still make the argument that there's no cost to broadcasting? I'm not being snarky, I'm just new to elixir and genuinely am not sure myself. I would think no subscribers would lead to no cost, but pattern matching would still encur a noticeable cost - especially when you're looking to respond to <0.01% of comments.~

EDIT: I read your message in the wrong order! Sorry, I misunderstood your point. What you described is exactly how I would expect it to work.

Given your documented aversion to extra_columns (or repeatedly stating that they should be used very sparingly) I think it's fair enough to have watchers send out on each of the extra columns. You're already making the case to limit them, if anything this would just bolster your cause :D but I'm bias because that would increase the performance for me in particular.

Also, if there's a way to generate them on subscription, like:

EctoWatch.subscribe(MyApp.Blog.Comment, :inserted, where: [post_id: "1234"])

That would be the most ideal. I don't really need the extra column back in the select in that instance. I just want all comments that are inserted for a specific post.

If you can come up with a plan/spec that you like, I'm happy to try implementing it myself!

[barrelltech]

PS: Yes, you understood me perfectly 🙌

[cheerfulstoic]

I've started a discussion over here: https://github.com/cheerfulstoic/ecto_watch/discussions/7

... and I have lots of thoughts. Would love your opinion over there!