Adopt a different account for RelVal input data placement in MSTransferor

[amaltaro]

Impact of the new feature MSTransferor

Is your feature request related to a problem? Please describe. Yes, we would like to avoid blocking Release Validation workflows in a scenario where central production has exhausted all the wmcore_transferor quota at the usual target RelVal sites.

Describe the solution you'd like We would like to separate central production from RelVal input data placement in MSTransferor. This is the first out of 3 issues to fully support this new feature. With this ticket, we should setup the following pre-requeriments:

• contact the CMS Rucio team and create a new account for RelVal input data placement (named as wmcore_transferor_relval
• contact the DM team and define proper quotas for this account at 4 RSEs: T2_CH_CERN, T1_US_FNAL_Disk and T2_DE_DESY, T2_US_Nebraska
• add a new parameter to the MSTransferor configuration file which will have the rucio relval account

and implement this new feature in WMCore, thus:

• in MSTransferor, fetch the relval account limits and the current usage
• RelVal workflow need to have input data location performed against the new rucio account
  • if input data is already available and locked by the relval account, then there is no need for any input data placement
  • otherwise, create a rule against the RSEs where the workflow is meant to read data from (PSN x PNN), using the new RelVal rucio account. Of course, if the RSE does not have enough quota, we should make a record in the log and skip the input data placement until space is freed up.
• relval RSE nodes out of quota should also be reported via the status REST API

Describe alternatives you've considered Bypass any quota limits and always make RelVal input data placement, regardless. However, people apparently didn't like this approach :)

Additional context none

[nsmith-]

I believe we would only need to make quotas for the select sites which run RelVal. @haozturk that would just be T2_CH_CERN and T1_US_FNAL_Disk right?

[amaltaro]

It shouldn't have been closed. Reopening.

[amaltaro]

After further discussion and developments, I updated the description and required changes in this ticket.