search

dmwm / WMCore

Core workflow management components for CMS.
Apache License 2.0
46 stars 107 forks source link

Remake input data placement upon site list changes #12040

Open amaltaro opened 1 month ago

amaltaro commented 1 month ago

Impact of the new feature MSTransferor

Is your feature request related to a problem? Please describe. This is a sub-task of this meta-issue: https://github.com/dmwm/WMCore/issues/8323 Towards providing a feature that allows workflow site lists to be changed while workflows are active in the system.

Describe the solution you'd like Whenever the site lists of an active workflow changes, we need to revisit the input data placement rules - if any - and update their RSE expressions accordingly. That will involve at least the following: 1) accessing the workflow transfer document to find out the rule ids 2) if a site was added to the SiteBlacklist, that site needs to be removed from the rule ids RSE expression 3) if a site was added to the SiteWhitelist, that site needs to be added to the rule ids RSE expression 4) the new rule ids need to be persisted in the transfer document, replacing those rules that are no longer valid.

This can be implemented either: a) synchronous with the status transition, but it might make the client HTTP request just too expensive (ReqMgr2 + Globalqueue + Rucio change all in a single client call) b) asynchronous, but then we need a mechanism to flag/identify workflows that need data re-placement.

Describe alternatives you've considered There are a few ways to implement this feature, like: a) trigger a new data placement with the new site lists b) once the new rule creation is sucessful, we could delete the previous rule superseded c) similar to a), but instead of making a new rule, we could consume the rule ids already persisted in the database (via MSTransferor/MSMonitor) and update their RSE expression accordingly.

Hasan made the following observation for case c): "You cannot update (update-rule) the rse expression of a rule and keep the same rule id. You can change the rse expression by "moving" (move-rule) a rule which creates a new rule."

Additional context NOTE that the best would be to let MSTransferor take care of this data re-placement, such that it considers everything (campaign configuration, pileup configuration, rse quota, etc).

amaltaro commented 1 month ago

PS: based on implementation details, we might have to break this down into 2 or 3 issues.