estuary / flow

🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, with Estuary Flow. 🌊
https://estuary.dev
Other
530 stars 45 forks source link

agent: handle recusive spec expansion when publishing tests covering multiple derivations #1494

Open psFried opened 1 week ago

psFried commented 1 week ago

Spec expansion currently just pulls in authorized tasks that read from or write to one of the drafted collections. This works fine for most cases, but there's one case that it doesn't cover:

Say there's a series of derivations that source from one another, like derive/a -> derive/b -> derive/c and a test that inserts into a and verifies the output of c without inserting or verifying b. Then spec expansion would only pull in a and c if you draft just the test.

Of course the chain of derivations can be arbitrarily long, and so our spec expansion logic would need to be recursive in order to handle that.