jobs-management / job manager should be improved to permit plugin developers to queue jobs more easily

[carolinebridge]

The job manager / jobs-management widget / etc. should allow plugins to queue jobs more easily.

Currently, the jobs-management widget has some tooling for adding jobs to the widget, but to queue a custom job to this widget would require implementing a lot of repeated infrastructure.

The job manager is currently tightly coupled with the text indexing job that is run on desktop. indexJobsModel.ts is named JobsManager but it only performs text indexing. A number of operations in this model could be generalized and done for a user attempting to make a queue-able job e.g. updating the jobs manager widget automatically.

An example of a plugin using the job manager may be a plugin that procedurally loads in large files one by one into the available tracks. A user could select a folder for example, and let their job/script/etc. read and process each file into a JBrowse track. While this job is running they could continue to use JBrowse as normal / browse their existing tracks. This would be useful for a researcher actively visualizing data while capturing/ curating it.

Ideally, for a plugin developer this might look something like:

// say we click a button to execute a job
onClick( () => {
  const { jobsManager } = rootModel // jobs manager is easily accessible through normal means, currently this is how it is accessed, but again this jobsManager is tightly coupled with text indexing

  jobsManager.queueJob(
    job: MyJob,
    doneCallback: () => {} // similar to how queue dialogue works
  )
} )

where queueJob automatically adds the job to the queue of jobs, sets it to running when it is its turn to run, sets its name, updates its message, and sends the done callback when done.

where MyJob can be some model that extends a generic Job model where a user can add a custom method for "runJob" that actually runs what they need to run (parallel to runIndexingJob in indexJobsModel), maybe the doneCallback here instead of in whats passed to queueJob, status messages, job name, etc.

You can see some of the clunky usage of the widget in indexJobsModel currently, where the model needs to grab the jobStatusWidget from the session widgets every time it wants to update the ui -- this kind of thing should happen on its own and not require the user to write a method for retrieving the job status widget, update the ui on their end of things, etc.

[carolinebridge]

From discuss in meeting:

Could potentially do simply a callback instead of an object for the Job, e.g.

job: (statusCallback)=> { /* do expensive stuff, update statusCallback if it wants to */ }

• might have to review original reasoning for coupling textindexing w the jobs manager
• not necessarily a pluggable element, just allow the jobs manager to accept "jobs" to queue and visualize
• add some extra functions to the job manager that allow users to run jobs immediately e.g. 'runJob' runs the job provided to it immediately versus queueJob runs the job provided to it in a queue