Add ControlFlow tests to Prefect CI/CD Lifecycle

[aaazzam]

Describe the current behavior

A handful of changes from 3.0 release candidates to GA caused downstream errors in ControlFlow, built on Prefect.

I'd like to propose adding its test suite to somewhere in Prefect's CI/CD lifecycle. Selfishly, so ControlFlow users don't experience errors. More importantly, having a derivative framework in Prefect's CI/CD lifecycle could help Prefect contributors more quickly identify implicit behavior changes that are harder to catch in it's existing test suite.

Describe the proposed behavior

Perhaps on release, ControlFlow's test suite is run.

Example Use

No response

Additional context

No response

[cicdw]

two thoughts on this:

• I think it would be more extensible for ControlFlow to create a job that is triggered by Prefect 3.0 releases and runs its test suite in the controlflow repo; having Prefect action failures in this repo is a signal for the maintainers here to act, which test failures won't always represent
• I think it could be interesting to post issues in any repositories that rely on Prefect (especially if unbounded) about breaking changes / important changes that we have reason to believe will affect their project (I'm not sure how to accomplish this efficiently though tbh, but I feel like it's possible)

[aaazzam]

Yeah, dependabot-style stuff feels more extensible but would require us to pin Prefect which is doable but not ideal.

There's a precedent for OSS libraries to offer ecosystem tests for downstream libraries that enjoy a nontrivial relationship. See, e.g.

• Pydantic includes FastAPI in its test suite
• uv includes Prefect in its test suite

I think it comes down to whether you'd ever hold off a release of Prefect to investigate if you knew ex-ante that it would break ControlFlow? Totally get the argument of "we won't know if we were the ones that caused the breaking".

[cicdw]

The uv example there makes sense because they aren't running our test suite but just confirming that dev installations are healthy; the pydantic/fastapi one though is pretty interesting. And at the end of the day, these repos are both prefecthq properties so we can obviously share some responsibility, just trying to use this as a playground to explore options anyone else could use.

[aaazzam]

💯