search

timescale / timescaledb-toolkit

Extension for more hyperfunctions, fully compatible with TimescaleDB and PostgreSQL 📈
https://www.timescale.com
Other
383 stars 47 forks source link

Speed up tools/testbin #563

Open epgts opened 2 years ago

epgts commented 2 years ago

Three update tester runs:

  1. 01:40 old-style (pg14 only, no testbin)
  2. 02:45 testbin (pg14 only)
  3. 21:52 testbin (12, 13, 14)

Why is testing all three so crazy slow? (Granted I took only one sample of each, but even if that's way out at the end of the long tail, it's nuts).

Why is even testing just pg14 almost twice as slow?

Paying the extra minute cost at CI time doesn't hurt because it runs in parallel to the even slower test-extensions, but adding 20 minutes to release packaging is no fun.

Likely sources of slowness:

  1. our current approach benefits from having the old toolkit versions AND ready-to-go databases in the github actions cache

  2. new approach downloads binary packages from packagecloud every time

  3. new approach uses apt-get and dpkg which are much more complex than pgx simply copying files into place

And possible fixes:

  1. use cargo pgx start/stop so we can benefit from the cached databases as well as reduce complexity. I blame lack of sleep for pulling out my old local postgres tricks instead of just using that

  2. apt-get install --download-only the old toolkit versions into the container image

  3. dpkg-deb -x each package (bypassing apt, dpkg database maintenance, and pre-/post- installation scripts)

epgts commented 1 year ago

1 up there is done, actually ("use cargo pgx start/stop").