search

beethovenxfi / beethovenx-backend

MIT License
13 stars 15 forks source link

Investigate DB optimization #613

Open franzns opened 10 months ago

franzns commented 10 months ago

We'll soon need to run the balancer jobs on a higher interval which will lead to more load on the DB. Would be good to investigate and maybe optimize some queries/writes so we dont need to scale the DB.

franzns commented 10 months ago

The update-lifetime-values-for-all-pools job seems to be a heavy, I refactored it a bit and DB usage went down https://github.com/beethovenxfi/beethovenx-backend/commit/a567a42d81eb597560f918dbade398386b6f84b1

franzns commented 10 months ago

Need to investigate the COMMIT load, it probably comes from when prismaBulkExecuteOperations() transaction is set to TRUE. Need to check if this is really needed everywhere it's used.

franzns commented 10 months ago

for poolsnapshots and token prices, should we use plain s3 or dynamoDB?

franzns commented 10 months ago

The update-lifetime-values-for-all-pools job seems to be a heavy, I refactored it a bit and DB usage went down https://github.com/beethovenxfi/beethovenx-backend/commit/a567a42d81eb597560f918dbade398386b6f84b1

usage spikes are gone:

Image