Starting job execution along with inventory sync

[gabhishek627]

ISSUE TYPE

• Feature Idea

COMPONENT NAME

• UI
• API

SUMMARY

• current behavior: A Tower dynamic inventory - synchronizing from a script/program ( "Sourced from a Project" type ) - will not allow any Job Templates associated to it to execute during a sync operation. The inventory sync operation appears to be enforcing a global lock to the aforementioned group of Job-Templates and locking the inventory for the duration of the inventory refresh, from launch up to job completion.

• Expected Behavior: To run the starting job execution along with inventory sync. The Job-Templates/inventory global lock requirements should be relaxed and limited to enforcing the JT global lock only during the actual inventory parsing and PostgreSQL update operation, just after the script/program has emitted the newly generated JSON-formatted inventory, as per dynamic inventory specification, at its standard output . Such change would be justified by the fact that, for a program/script-generated dynamic inventory, the actual inventory structure is created at program/script termination and perceived by Tower as a single monolithic JSON dump operation.

[AlanCoding]

@fosterseth we have some history with loosening up these blocking rules in the past. Could you read through this and give any thoughts?

[fosterseth]

@gabhishek627

In your scenario, removing the block would mean those jobs running alongside the inventory update might not get the latest changes pulled in from the inventory update. Of course, if you need the job to wait on an update, you can make sure Update On Launch is set to true on the inventory source.

@AlanCoding

here is the dependency_graph.py code to determine if the Job can run,

def job_blocked_by(self, job):
    project_block = self.get_item(self.PROJECT_UPDATES, job.project_id)
    inventory_block = self.get_item(self.INVENTORY_UPDATES, job.inventory_id)
    if job.allow_simultaneous is False:
        job_block = self.get_item(self.JOB_TEMPLATE_JOBS, job.job_template_id)
    else:
        job_block = None
    return project_block or inventory_block or job_block

we may wish to remove the project_block and inventory_block conditions, and just have the job_block condition.

If a job really needs to wait on an inventory source update, it would have added it to the dependent_jobs field, and be blocked that way.

[AlanCoding]

We may need to investigate what will happen if we generate inventory while an inventory update is happening. The update is inside of a transaction. If the inventory were to simply get the inventory pre-update, I'd be fine with that. If it gets content half-way through the update, we'd have to add some guard rails to prevent that.

I believe we also need to increase the rigor with which we manage the project local lock.