Closed adamsachs closed 1 year ago
@adamsachs my two cents, I'd add values for both and just set reasonable defaults. My feeling is that if folks start to hit these errors, then we should give them all the options to fine-tune for their particular system.
tested with dev help
Is your feature request related to a specific problem?
based on testing and investigation in https://github.com/ethyca/fides/pull/2489 and related issues, we've realized it would be useful to allow our sql alchemy db
Engine
pool_size
andmax_overflow
settings to be configurable as application config properties.this would be useful for both testing purposes and also for fine-tuning application deployment in the wild - the optimal values for these settings are likely dependent on both expected application workload as well as constraints on the database side - i.e. environmental factors that we can't easily predict!
Describe the solution you'd like
DatabaseSettings
module?Describe alternatives you've considered, if any
Engine
s we create, namely:Engine
used for API operationsEngine
per celery task (for privacy request execution and for message dispatch). if using workers, this will be a separate fides instance; if not, though this will be using the same fides instance, and would therefore read from the same configAdditional context
See comments on https://github.com/ethyca/fides/pull/2489 and https://github.com/ethyca/fidesops/pull/944 for some additional context