search

DUNE / dist-comp

Action items for DUNE distributed computing, and common scripts that are used.
2 stars 0 forks source link

DUNE global pool--need coordinated transition to GLIDEIN_DUNESite in matching #94

Open StevenCTimm opened 10 months ago

StevenCTimm commented 10 months ago

have to coordinate changing jobsub, Justin, and the frontend.

StevenCTimm commented 4 months ago

Andrew changed JustIN back to use GLIDEIN_Site for now

Andrew-McNab-UK commented 4 months ago

I'm not sure this change is needed. During DC24 when we got tens of thousands of jobs running, the Requirements in the submit files looked like this:

(((isUndefined(target.GLIDEIN_Entry_Name) == false) &&
        (stringListIMember(target.GLIDEIN_Entry_Name,my.Desired_Entries)))) &&
    (TARGET.Arch == "X86_64") && (TARGET.OpSys == "LINUX") &&
    (TARGET.Disk >= RequestDisk) && (TARGET.Memory >= RequestMemory) &&
    (TARGET.HasFileTransfer)

where Desired_Entries is a list of the CE names that are suitable for that workflow/stage, written in the factories' format. Desired_Sites is also defined correctly, using the GLIDEIN_Site names. I don't think there's a particular need to change over to using the DUNESite names at this level, since it's all hidden from users and is written using the names the GlideInWMS and pilot factory people are used to.