ethyca / fides

The Privacy Engineering & Compliance Framework
https://ethyca.com/docs
Apache License 2.0
357 stars 72 forks source link

provide config properties to set db engine `pool_size` and `max_overflow` #2507

Closed adamsachs closed 1 year ago

adamsachs commented 1 year ago

Is your feature request related to a specific problem?

based on testing and investigation in https://github.com/ethyca/fides/pull/2489 and related issues, we've realized it would be useful to allow our sql alchemy db Engine pool_size and max_overflow settings to be configurable as application config properties.

this would be useful for both testing purposes and also for fine-tuning application deployment in the wild - the optimal values for these settings are likely dependent on both expected application workload as well as constraints on the database side - i.e. environmental factors that we can't easily predict!

Describe the solution you'd like

Describe alternatives you've considered, if any

Additional context

See comments on https://github.com/ethyca/fides/pull/2489 and https://github.com/ethyca/fidesops/pull/944 for some additional context

pattisdr commented 1 year ago

@adamsachs my two cents, I'd add values for both and just set reasonable defaults. My feeling is that if folks start to hit these errors, then we should give them all the options to fine-tune for their particular system.

Roger-Ethyca commented 1 year ago

tested with dev help