Refactor Tendrl Jobs system for more performance - Githubissues

Tendrl / specifications

Tendrl specs go here

GNU Lesser General Public License v3.0

6 stars 16 forks source link

Refactor Tendrl Jobs system for more performance #162

Open r0h4n opened 7 years ago

r0h4n commented 7 years ago

Current:

Every job (internal or user created) goes to /queue
Every "new" job in /queue is processed by all tendrl-node-agent and integration (ceph, gluster) agents and tendrl-performance-monitoring and tendrl-node-monitoring agents
Every job is matched against the above agent's tags (read capabilities) and the
Child jobs save a pointer to them in Parent_job.children

Proposed change:

Each job is posted at a location in central store calculated on the basis of "tag" defined for the Tendrl Flow on which the job is based or the node_id of the tendrl-node-agent (if targeted towards specific node)
Parent jobs should contain child jobs entirely inside :job_id/children/
Create an index with job_id: job_location (location as calculated in pt1 above)
Differentiate between internal jobs and user created jobs in above index

This impacts the current internal tendrl jobs, user created tendrl jobs (from api), tendrl job processing

spec branch: "spec/refactor_jobs"

r0h4n commented 7 years ago

cc: @Tendrl/tendrl-core , @Tendrl/tendrl_api