Revisit the balance between JobAttributes and ResourceSpecification

[hategan]

Is your request related to a problem? Please describe. The problem is that some properties (such as job duration) are both properties of the job and possibly desired dimensions for an allocation. Furthermore, some properties currently in ResourceSpec, such as process_count should be in JobAttributes instead. For a bit of a discussion, see #174.

Describe the solution you'd like Devise a proper separation of job properties into attributes that are separate from the resource request (or be clear how and when they overlap)

[frobnitzem]

For context, slurm's API history also shows continuing evolution on how to specify resources and attributes as hardware and use cases change over time. I think the lesson there is that nothing short of a DSL (e.g. a computation yielding true if resources "fit the bill", or, better, a function from resources to predicted run-time) can offer a general solution.

• generic resource name-value pairs

  • gres - name[[:type]:count] list of generic consumable resources used to specify things like type of GPU available
  • tres - type:[name:]count list of resources tracked by the site

• flexible resource specs

  • --nodes=<minnodes>[-maxnodes], --use-min-nodes

• specific resource sub-languges

  • gpus, network topology, etc.

• process binding flags

  • --mem-bind, --gpu-bind, --hint

• hardware settings

  • --power, --gpu-freq, --cpu-freq

[hategan]

I think the lesson there is that nothing short of a DSL (e.g. a computation yielding true if resources "fit the bill", or, better, a function from resources to predicted run-time) can offer a general solution.

:)

We talked about this towards the beginning of the project and speculated that, likely, the "right solution" involves some turing-complete thing. That's probably ResourceSpecV3.

Although, if we're to be pedantic and as I think @andre-merzky was saying, there's job shape, resource requested, and the mapping between the two and we're somewhat conflating the last two in PSI/J (in addition to the ambiguities discussed in #174).