search

radical-cybertools / radical.saga

A Light-Weight Access Layer for Distributed Computing Infrastructure and Reference Implementation of the SAGA Python Language Bindings.
http://radical-cybertools.github.io/saga-python/
Other
83 stars 34 forks source link

SMT - new attribute for SAGA job (used for LSFJob) #799

Closed mtitov closed 4 years ago

mtitov commented 4 years ago

SMT can be set with the following values [1, 2, 4] (according to Hardware Threads at the Summit User Guide)

Test run was made at Lassen (since Lassen also support SMT) with the following additional parameters:

# extend resource config with the following parameter
        "saga_jd_supplement"          : {"smt": 2}

and for the RP layer set the following environment variable

export RADICAL_SAGA_SMT=2

Later this will be unified in a way that only one parameter would be needed (either one or another). These updates will be provided at the RP ticket #2167

mtitov commented 4 years ago

as a follow up comment/question - since there is no summitdev at RP, should it be removed from RS as well?

andre-merzky commented 4 years ago

Hi @mtitov - I'm hesitant to touch the SAGA API. It's the one API we have very tightly defined, and so far we managed to stick to that spec. I would suggest to move the SMT setting to the CPUArchitecture attribute, maybe as SMT:n?

mtitov commented 4 years ago

@andre-merzky I see your point and I agree not to have it as a separate attribute, but have more changes to propose then :)

  1. Should attribute CPUArchitecture be renamed into SystemArchitecture? (considering that it might contain information about GPUs as well)
  2. Have CPU[System]Architecture as a dictionary (currently its value is a single string: architecture name) for a better extension of it with description parameters. And to have the following possible keys: ['cpu', 'gpu', 'smt'] 
    
# radical.saga.job.description

self._attributes_register (CPU_ARCHITECTURE, None, sa.STRING, sa.DICT, sa.WRITEABLE)

andre-merzky commented 4 years ago

All fair points - but I'd still rather stick to the specification for the time being which also defines the attribute names and types. We may at some point want to touch the spec itself and update it, but I don't think that is a can of worms we should open right now...

mtitov commented 4 years ago

Closed in favor PR #806