stripe / pg-schema-diff

Go library for diffing Postgres schemas and generating SQL migrations
MIT License
347 stars 25 forks source link

Concurrent schema fetching #91

Closed bplunkett-stripe closed 8 months ago

bplunkett-stripe commented 8 months ago

Description

Make schema fetching concurrent. This should accelerate it a fair bit, especially if individual queries have high latency.

Motivation

Accelerate schema fetching

Testing

Tested via unit tests

bplunkett-stripe commented 8 months ago

Well hmm GoRoutineLimter errors on Go on canceled context, so maybe the sync & async runners should too... Let me know what you think

Per semaphore docs:

// Acquire acquires the semaphore with a weight of n, blocking until resources
// are available or ctx is done. On success, returns nil. On failure, returns
// ctx.Err() and leaves the semaphore unchanged.
//
// If ctx is already done, Acquire may still succeed without blocking.

They only check ctx cancellation if the thread is blocked waiting on acquiring a unit.

Either way, we are left with a race condition without implementing some chan behavior, so I think it's probably okay as is, albeit a little weird.

bplunkett-stripe commented 8 months ago

r? @alexaub-stripe