search

Tinkoff / Horarium

Horarium is the .Net library to manage background jobs
Apache License 2.0
56 stars 12 forks source link

Add ability to throttle job #39

Open PDmatrix opened 4 years ago

PDmatrix commented 4 years ago

Currently, the only way to decrease the load on the database is to increase IntervalStartJob, but there might be problems with it I suggest that we add another setting JobThrottleSettings that will dynamically increase IntervalStartJob if there are no jobs available for a certain period

Algorithm: 1) Try to get jobs with IntervalStartJob 2) If there is no jobs after JobThrottleSettings.JobRetrievalAttempts we increase IntervalStartJobs by JobThrottleSettings.IntervalMultiplier 3) To prevent massive IntervalStartJob, there will be JobThrottleSettings.MaxJobThrottleInterval so we won't wait too long 4) After we successfully retrieve a job, IntervalStartJob will reset to its original value

Bobreshovr commented 4 years ago

it's a good idea, add one comment to PR. What about the load test of this throttle?