Make host a required argument for OSC::Machete::Job

[ericfranz]

1. rename OSC::Machete::TorqueHelper to OSC::Machete::TorqueAdapter
2. add OSC::Machete::TorqueAdapter#default_host which can return nil
3. subclass OSC::Machete::TorqueAdapter with OSC::Machete::OSCTorqueAdapter
4. add class methods to get the hosts and override the default host which would be OSC::Machete::OSCTorqueAdapter.oakley. Other hosts could be OSC::Machete::OSCTorqueAdapter.ruby, OSC::Machete::OSCTorqueAdapter.glenn, OSC::Machete::OSCTorqueAdapter.quickbatch.
5. Add a class OSC::Machete::Adapter with a single class attribute default
6. Change OSC::Machete::Job to use this default adapter
7. Rename torque_helper to torque_adapter in OSC::Machete::Job

Then users can override the default adapter by setting it i.e.:

OSC::Machete::Adapter.default = OSC::Machete::OSCTorqueAdapter

Then any OSC::Machete::Job class created without the "torque_adapter" argument passed in will use this adapter.

[ericfranz]

Starting out we could make the OSC::Machete::Adapter.default be OSC::Machete::OSCTorqueAdapter but eventually we might want to change that and require that the apps specify which adapter to use.

[ericfranz]

OSCTorqueAdapter should not have a default host. If we make host a required argument in OSC::Machete::Job, there is no need to have a default_host. There is a default adapter class.

The web app has to be updated so when it creates an OSC::Machete::Job object the host is passed in. Also, all of the tests that create OSC::Machete::Job objects need updated.

And we would be doing this in two locations: the script and in the code. I feel like we had a discussion about this before... what is the right solution here?

• Old apps will implement host getter that returns the right host for the job
• New apps will store the host in the job_cache column
• When submitting a job, what happens? We could still analyze the script for the headers like we do now. But would we do that prior to instantiating a job object, instead of after?

Or maybe we let you instantiate a job object without a host argument, but if you try to submit, or check status, etc., the OSC::Machete::Job object can complain? Or try to use the default adapter to get the host from the script.

[ericfranz]

This needs a clearer design of what we are going to build.

[nickjer]

Another option, we have osc_machete_rails configured in an initializer with a subclass of OSC::Machete::Adapter.

# Adapter public interface
OSC::Machete::Adapter::submit
OSC::Machete::Adapter::status
OSC::Machete::Adapter::delete

# Possible adapters
class OSC::Machete::Adapters::TorqueAdapter < OSC::Machete::Adapter
  # in here have a factory that produces a TorqueAdapter from a configuration (i.e., OSC specific)
end

# or just make more specific adapters inherited from above

An example of an initializer...

# config/initializers/osc_machete_rails.rb

OscMacheteRails.configure do |config|
  ...
  config.adapter = OSC::Machete::Adapters::OSCTorqueAdapter
end

Then to get the status we feed the adapter a OSC::Machete::Job object. That way you can utilize duck-typing for the adapters. As we can't predict what other adapters may need.

OscMacheteRails.adapter.submit(OSC::Machete::Job.new)
# => job object with pbsid

job = OSC::Machete::Job.new(pbsid: '3829832', host: 'ruby')
OscMacheteRails.adapter.status(job)
# => StatusValue object

The Job object can just be an OpenStruct object with methods that the adapter will ask for.

job = OpenStruct.new
job.pbsid = '9837324'
job.host = 'ruby'

OSC::Machete::Adapters::OSCTorqueAdapter.status(job)
# => StatusValue.queued

Obviously if there are dependencies a slightly smarter object may need to be employed. But as you see this mimics our Store in the ActiveRecord. In fact you could get rid of the OSC::Machete::Job object and use an ActiveRecord object with the methods that the adapter need. So developers can adapt their Store to the chosen system adapter.

[nickjer]

Even crazier idea:

Feeding into the previous comment about getting rid of the OSC::Machete::Job object. We could use a hash to describe a workflow or single job:

# Simple workflow with post-processing
workflow = {
  main: {
    script: '/path/to/main.sh',
    host: 'oakley',
  },
  post: {
    script: '/path/to/post.sh',
    host: 'oakley',
    after_ok: [ :main ]
  }
}

# Submit workflow and get back workflow hash w/ pbsid information
# (or any other adapter specific information used for getting status or deleting jobs)
new_workflow = OSC::Machete::Adapters::OSCTorqueAdapter.submit(workflow)
# =>
# {
#   main: {
#     pbsid: '20983408',
#     script: '/path/to/main.sh',
#     host: 'oakley'
#   },
#   post: {
#     pbsid: '20983409',
#     script: '/path/to/post.sh',
#     host: 'oakley',
#     after_ok: [ :main ]
#   }
# }

# Get status of main job
OSC::Machete::Adapters::OSCTorqueAdapter.status(new_workflow[:main])
# => StatusValue.queued

# Get status of all jobs
OSC::Machete::Adapters::OSCTorqueAdapter.status(new_workflow)
# =>
# {
#   main: StatusValue.queued,
#   post: StatusValue.held
# }

This would make it very simple to store jobs in ActiveRecord and have apps work across different platforms depending on the adapter they chose in their initializer.

Instead of hashes, we could use OpenStruct or simple classes that coincide with what the specific adapter expects for the job interface.