snowplow-incubator / common-streams

Other
1 stars 0 forks source link

Add Coldswap class to manage concurrent resource access #53

Closed istreeter closed 7 months ago

istreeter commented 8 months ago

In some of our applications, it is helpful to close and re-open a resource during event processing. For example, in a warehouse loader we might need to close the event-loading channel while we alter the warehouse table to add a new column.

Cats Effect offers a Hotswap which manages a sequence of resources, and runs the resource finalizers in case of cancellation or error. However, it's not quite what we need, because a Hotswap.get returns a Resource[F, Option[R]] whereas we typically need it to return a Resource[F, R]. And because sometimes we need to guarantee that no Resource is opened for the duration of an IO -- A Hotswap has nothing to help with this.

We need our own resource manager... let's call it a Coldswap. We call it "cold" because (unlike a Hotswap) it always closes any previous Resources before acquiring the next one.