Closed dcherry88 closed 4 years ago
adamdriscoll
commented
4 years ago Thanks for writing this up. I agree that we need this. @leeberg actually ran into this very problem trying to run as many scripts as possible. It's currently not throttled at all.
We should probably also consider at what point the job queue is full and we need to start "failing" newly queued jobs.
dcherry88
commented
4 years ago Also a valid thought, would hate to see a scenario of thousands queued up jobs when it should have just stopped taking jobs at 100 or so and flagged out some sort of alert to server admins.
It didn't quite outline it in the original post, mostly because I wasn't sure how complex one scenario would be vs the other, but ideally I think a per-script setting would be nice. Maybe a site-wide limit depending on how many different scripts could be running at any given point. I think a given with the queue idea would also be the ability to pull that status out of the API.
Windos
commented
4 years ago My two cents is it'd be preferable to have this configurable at a script, tag or folder, and server level.
Server level would allow you to manage resource congestion of the server itself.
Script and Tag/Folder level would help mitigate remote resource congestion.
e.g. let's say I have a fragile N-able instance and I only want a max of two scripts running against the SOAP API at a given time. I could tag all relevant scripts as 'N-able' and set the tag to max concurrent = 2.
My overall API limit is met, while being able to have multiple scripts acting against it on various schedules.
dcherry88
commented
4 years ago Has there been any further planning on this feature? This is about the only feature I'm waiting on for our use case to test and get UA licensed.
adamdriscoll
commented
4 years ago We are looking to implement a basic version of this in the next release. I think we'll start with server wide configuration, then script and then maybe more fine grained controls later.
adamdriscoll
commented
4 years ago Per-server concurrent job limits are available. I'm going to open different issues for per-script\per-tag limits
It would be nice to see the ability to control the jobs started up on the UA server by setting total jobs running and having it queue anything past that number to be run as soon as a spot clears up.
If, for instance, someone queued up a particular script/job via API request in bulk of 20, but the script had a specified limit of only running 5 concurrent jobs, it would put the remaining 15 into a queue to be completed as the 5 jobs finish their work.
In situations where we are looking to utilize UA, we could see batch submissions in larger numbers, which could cause resource constraints on the UA server running said jobs. Being able to manage these limits would allow us to scale concurrent job usage based on available resource constraints.
I've not yet done alternative ideas yet, but I imagine something could be set at the start of a script to check with the UA server how many jobs for the script are currently running, before pausing and waiting to recheck again.