We can throttle the container itself through Docker resource limits, but that's not going to solve the core issue - the fact that the ES requests cause lots of impact on the cluster. I don't see any clean way to solve this besides changing the cron schedule for now.
@michielmeersmans do you have suggestions for this one?
Scanning "slower" when processing use cases is the only way I see for now (throttling the scan generator & the speed of which we request documents) but that sounds like a hack too.
We can throttle the container itself through Docker resource limits, but that's not going to solve the core issue - the fact that the ES requests cause lots of impact on the cluster. I don't see any clean way to solve this besides changing the cron schedule for now.
@michielmeersmans do you have suggestions for this one? Scanning "slower" when processing use cases is the only way I see for now (throttling the scan generator & the speed of which we request documents) but that sounds like a hack too.