dmwm / WMCore

Core workflow management components for CMS.
Apache License 2.0
46 stars 107 forks source link

Support campaign type via job class ads #11707

Open khurtado opened 1 year ago

khurtado commented 1 year ago

Impact of the new feature This feature would create a new class ad for the campaign type for monitoring. As recently discussed over mattermost, this script: https://github.com/dmwm/cms-htcondor-es/blob/vm-legacy/src/htcondor_es/convert_to_json.py

has some logic embedded in order to figure out the campaign type out of the request name. Instead of parsing the request name, we should actually define new classads and provide the right information in a job-basis.

Is your feature request related to a problem? Please describe. This is a child of #10914 The campaign types supported are:

 - "MC Ultralegacy"
 - "Data Ultralegacy"
 - "Phase2 requests"
 - "Run3 requests"
 -  "RelVal"
 - "Run2 requests"
 - "SnowmassWinter21"

and are defined in the following jira ticket comment: https://its.cern.ch/jira/browse/CMSMONIT-174#comment-3050384 and the logic is implement at the monit level, based on the request names

More types could exist/be added in the future, so we need to take this into account while addressing this ticket. This is information that should not be guessed at monit level, but rather just reported (hence the need to move this feature to WMCore).

Describe the solution you'd like We need to come up with a solid plan to define the campaign type and propagate it all the way to the job level through class ads. This might require changes at the Task/Step level and JobCreator/Submitter level (plus the simplecondor plugin and we need to properly support ReReco, Task Chain and StepChain workflows.

Here is the list of issues to be completed:

Describe alternatives you've considered Keep using the current mechanism in monit for this.

Additional context

10914

https://github.com/dmwm/cms-htcondor-es/blob/vm-legacy/src/htcondor_es/convert_to_json.py#L1087-L1112

amaltaro commented 1 year ago

@khurtado Kenyi, can you please provide the current definition (pointers?) to what campaign type is supposed to be? What values are currently supported and so on. Please update the original description with that information.

khurtado commented 1 year ago

@amaltaro Done! Please let me know if I missed anything.