EctoWatch fails when the extra columns aren't explicit associations

[barrelltech]

Many applications these days use external auth - supabase, clerk, auth0, etc - or have their auth in a separate table or service. This means EctoWatch can't subscribe to user's data by their id.

I commented out the validation requiring it to be an association and it works on my machine. I'm not sure the reason requiring a column to be an association.

I removed these two lines in watcher_server:

#       column && not MapSet.member?(state.identifier_columns, column) ->
#         {:error, "Column #{column} is not an association column"}

[cheerfulstoic]

Yeah 😅

So, the reason is because when you subscribe to a value that value becomes part of the PubSub topic. So if you subscribe to the role_id column for value 1234 (e.g. EctoWatch.subscribe({User, :updated}, {:role_id, 1234})) then the PubSub topic would be something like:

ew_updated_for_User|role_id|1234

Allowing subscribing to columns which had potentially large values (like a string/binary column) seemed like a bad idea. Since I couldn't think of a case (and I asked somebody else as well) where you would subscribe to anything but an association column, I put an artificial limit in for now with the thought that it's easier to loosen up the restrictions later 🤷‍♂️

To open it up a bit more while still being careful, I'm thinking it might make sense to allow subscribing for association columns + integer columns... maybe? But that might not work for external services if you've got a UUID string 🤔 Maybe there should be some sort of validation on the size (255 characters if it's a string?) when subscribing?

[cheerfulstoic]

Oh, I guess the other consideration was that whenever an insert/update/delete happens, EctoWatch sends out messages on every topic that might be relevant. If we limit that to associations then we don't send as many messages. If we didn't limit to associations, though, we'd need a topic for every single column/field.

I don't know if there's a huge cost to trying to send message to topics that aren't subscribed to, but I think that there is some cost and making that the default would be a problem.

Perhaps another option is to have an additional option for watchers like subscribable_columns which defines columns that could be subscribed to 🤷

[cheerfulstoic]

👋 I'd love to address this issue. Do you have any thoughts on the above?