search

estuary / flow

🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, with Estuary Flow. 🌊
https://estuary.dev
Other
599 stars 48 forks source link

auto-discovers should always add new bindings #1674

Open psFried opened 11 hours ago

psFried commented 11 hours ago

Currently auto-discovers will only add bindings if addNewBindings is enabled. We'd like to instead have them always add the new bindings, and to have them be disabled if addNewBindings is false. This allows users to selectively enable bindings later.

dyaffe commented 11 hours ago

Something even better here could be having a second discovery mode which doesn't add new bindings by default. The UI could then use that when addNewBindings is set to false

psFried commented 11 hours ago

Something even better here could be having a second discovery mode which doesn't add new bindings by default. The UI could then use that when addNewBindings is set to false

We literally already have this. It's the discovers.update_only column.

dyaffe commented 11 hours ago

Amazing. So could a simple solution just be a UI change to call that when someone has the addNewBindings set to false? That would accomplish the goal.

If so, I would suggest we update this to a high priority frontend issue.

psFried commented 8 hours ago

@dyaffe That's what I'd been thinking, too. But @jonwihl made what I think is a pretty good point. Say a user is capturing from a database with a ton of tables, and they're only capturing a couple of them. Now they want to add a table. If our UI just presents two different options for how to refresh bindings, then neither would provide an easy solution for just adding a small number of possible tables.

But we could serve that use case well if we instead change discovers to always generate models that represent all bindings, and just keep them disabled if addNewBindings is false. And we already have enable|disable-all buttons in case the user does actually want to enable all the new bindings.

I'm not sure that we shouldn't also have the UI set update_only based on the value of addNewBindings, since that's been a point of confusion for users. And I don't think it'd hurt anything. But I'm thinking that the original idea seems like it would result in a better UX. WDYT?