netbox-community / netbox

The premier source of truth powering network automation. Open source under Apache 2. Public demo: https://demo.netbox.dev
http://netboxlabs.com/oss/netbox/
Apache License 2.0
15.37k stars 2.49k forks source link

Per Script Worker Queue Setting #16516

Open jchambers2012 opened 3 weeks ago

jchambers2012 commented 3 weeks ago

NetBox version

v3.7.8 and v4.0.5

Feature type

New functionality

Proposed functionality

Looking to enhance the script subsystem to add support to specify what worker queue each script should be run in. This will allow critical scripts to run before background or scheduled scripts. This feature would add a new field in the scripts’ Meta possibly called “rq_queue_name” that can bypass the logic in get_queue_for_model and just use the user defined queue. If “rq_queue_name” is not set then the current function can be called to determine what queue the script should be map too.

Use case

We have a few CI/DI pipeline that hit the API to generate control files in a centralized location versus the logic be distributed across different project code. This simplifies management of the project files but these critical scripts must wait in the queue with other background and user run scripts delaying the running of them.

Some of our non-critical script that update data can take 5 minutes to run and they are not time critical like the CI/DI pipeline.

Database changes

I don’t think any models will need to be updated to support this functionality.

External dependencies

n/a

jchambers2012 commented 3 weeks ago

I mocked up what i think needs to be changed but have not tested - let me know if you want me to work on the feature

https://github.com/netbox-community/netbox/compare/develop...jchambers2012:netbox:script_rq_queue_name?expand=1

arthanson commented 3 weeks ago

@jchambers2012 I've assigned it to you as you said you have some code for it.

jchambers2012 commented 3 weeks ago

Stimulation are working will try and button up the PR this afternoon if not tomorrow image

image