database node resource sizes available

percona / everest

Percona Everest is a cloud-native database platform to deploy and manage enterprise-grade PostgreSQL, MongoDB and MySQL database clusters.

https://docs.percona.com/everest/index.html

Apache License 2.0

238 stars 11 forks source link

database node resource sizes available #491

Open fideloper opened 4 months ago

fideloper commented 4 months ago

Hi! It looks like there are some minimum database sizes we can choose: https://github.com/percona/everest/blob/385a439a30b3f2384161f91a13df21efccbab503/api/validation.go#L61-L63

minStorageQuantity = resource.MustParse("1G")   //nolint:gochecknoglobals
 minCPUQuantity     = resource.MustParse("600m") //nolint:gochecknoglobals
 minMemQuantity     = resource.MustParse("512M") //nolint:gochecknoglobals

We're hoping to create many, tiny databases (single node, some temporary and some just very small) - our cost planning would prefer to use smaller CPU sizes than 600m.

Our tests showing this working fine for our minimal use case but these minimums are too high. I was curious if these are set for a reason other than stopping us from shooting ourselves in our foot? Is that possible to be overwritten/changed? (Happy to send a PR).

Thanks!

recharte commented 4 months ago

Hi @fideloper, thank you for bringing this up.

I'll be totally transparent and say that I don't know why we have these particular limits. These were added as part of a bigger scope of validations in this PR.

To be honest, they seem arbitrary and definitely don't make sense to be shared across the different engine types. However, we have a ticket (EVEREST-696) that mentions that backups were failing with these minimum resource requirements so there's a chance that unlocking the minimum requirements would lead to more of these issues.

I think this requires more investigation and testing before we can just blindly remove the limits. We'll look into this and report back.

Nonetheless, can you share more details about your use case? Which DB engine(s) are you interested in running and what are the resource limits that fit your use cases?

fideloper commented 2 months ago

I'm sorry I missed your reply from over a month ago!

Ideally we'd be able to set sizes for both mysql and pgsql, altho we're looking towards MySQL (I have not tested pgsql in smaller sizes like I have with mysql).

We have a use case where we'll have potentially many small databases for non-critical workloads (dev/staging environments). For environments that were more important (and where backups were more important), we'd have larger instance sizes.

calvinps commented 2 months ago

@recharte ideally we'd like to try out a minimum CPU size of 125m to 250m.