Re-architect machete to be more object oriented and follow SRP

[ericfranz]

A lot of code in SimpleJob is based on inheritance - your model becomes a Workflow or a Statusable or a Submittable and gets a bunch of methods that handle these concerns:

• staging
• submitting
• persisting
• presenting status

Both of these will help a little:

• https://github.com/AweSim-OSC/osc-machete/issues/32
• https://github.com/AweSim-OSC/osc-machete/issues/34#issuecomment-72220500

But generally we should move a lot of this logic out into separate objects that handle their own concerns, such as a Submitter object and a StatusPresenter object etc. SimpleJob could make use of these objects to offer its inheritance approach, but these objects could be used separately if we wanted to have data schemas in our app that SimpleJob did not support.

SRP == Single Responsibility Principle

[nickjer]

I see:

Workflow = a template directory with mustache files Job = a pbs batch script

So what can you do with a pbs batch script?

• you can submit it
• you can view the status
• you can view the output and error files generated by it
• it has a path location
• it can depend on another Job object (pbs batch script)

Basically the state of any Job will match the current jobs table we have in ContainerFillSim

create_table "jobs", force: true do |t|
  t.string   "status"
  t.string   "pbsid"
  t.string   "job_path"
  t.integer  "container_id"
  t.datetime "created_at"
  t.datetime "updated_at"
  t.string   "script_name"
end

Except without the dates.

[ericfranz]

A WorkflowBuilder object could work like this:

• have chainable methods stage, render, build_jobs, submit
• have attributes that include the staged directory and the list of jobs

Staging could happen in the WorkflowBuilder (then you would instantiate it with a template path and a staged directory path). Or that could happen elsewhere, and then WorkflowBuilder would just get the staged directory path).

Going through the process you could do in a ActiveRecord model that has a jobs relation something like:

builder = WorkflowBuilder(template_path, staged_target_dir)
builder.stage.render(self).submit
builder.jobs { |job| jobs.create job: job }

The WorkflowBuilder could be created with a factory method that creates it for an ActiveRecord model so the conventions matching the model name etc. are handled by the factory method. SimpleJob could of course use these objects under the hood.

Isolating staging separate from the WorkflowBuilder means the conventions for what to stage where would be isolated

Chainable methods could even let you do tap if you wanted to get fancy:

builder = WorkflowBuilder(template_path, staged_target_dir)
builder.stage.tap{ |b| copy_input_files_to(b.staged_dir) }.render(self).submit

And since a workflow builder would be a jobs list, and a staged directory, we could call whichever methods we wanted in whatever order, or add our own - meaning we can extend functionality, change the order, do whatever we want with custom code without having to subclass any object or mixin any thing before hand.

Most of the methods that the current SimpleJob mixins add are not methods that we want to add to the public interface of a SimpleJob::Workflow instance. submit may be appropriate. But the other methods are only added in order to facilitate customization of the process used to build the workflow. We can provide customization of this in a better way - if submit is just instantiating one object and calling a bunch of functions on it, then it will be less intimidating for a developer to write their own if they need to. And then it would support other models of data organization that are not easy with SimpleJob::Workflow

[nickjer]

Why couldn't we pass in the name of the 'jobs' ActiveRecord?

builder.stage.render(self, :jobs).submit

That way it would know where to save the state of the jobs to?