Rely on PhEDEx (or something else) for the storage management within MicroServices

[amaltaro]

Impact of the new feature ReqMgr2 MSTransferor and likely MSMonitor

Is your feature request related to a problem? Please describe. As part of the Unified logic for input data placement, we need to have a mechanism to reliably monitor:

• how much storage is available at each PNN for the DataOps group
• how much data is commited to each PNN

such that it can drive the location selection logic for input data placement (also part of the Unified black box, as requested by issue #9138 )

Describe the solution you'd like Create the client API under the Services package and integrate it to the MicroServices where needed. Certainly in the MSTransferor algorithm to see which PNNs are eligible to receive data or not.

Describe alternatives you've considered It could be handled through reviewing the Transferor logic in Unified and integrating whatever hasn't yet...

Additional context See latest changes in this PR: https://github.com/dmwm/WMCore/pull/9330 and also some hints as discussed in this ticket: https://github.com/CMSCompOps/WmAgentScripts/issues/439#issuecomment-522857388

[amaltaro]

After chatting with Eric about this use case using Rucio, it works in a similar manner as PhEDEx does; however, in Rucio quotas are defined in an account and RSE basis. Thus, the PhEDEx group will be the Rucio account we use for production activity.

Here is the API and an example of its output:

>>> pprint(list(client.get_account_usage("transfer_ops")))
[{u'bytes': 94216814068955,
  u'bytes_limit': 1000000000000000,
  u'bytes_remaining': 905783185931045,
  u'files': 42322,
  u'rse': u'T3_US_NERSC',
  u'rse_id': u'1cb4cc86c6674c17b20463fa36c4aa9e'},
 {u'bytes': 7080039001511,
  u'bytes_limit': 10000000000000,
  u'bytes_remaining': 2919960998489,
  u'files': 10857,
  u'rse': u'T1_DE_KIT_Disk_Test',
  u'rse_id': u'dc967804cd6a407a8603b54ddc73adf7'},
 {u'bytes': 6582466425893,

and if we want to get the account usage against a specific RSE, then we can provide it as a 2nd parameter, e.g.:

>>> pprint(list(client.get_account_usage("transfer_ops", "T2_US_MIT_Test")))
[{u'bytes': 5987593178954,
  u'bytes_limit': 10000000000000,
  u'bytes_remaining': 4012406821046,
  u'files': 9071,
  u'rse': u'T2_US_MIT_Test',
  u'rse_id': u'd479089e01434deead1ee377e949276b'}]