Test and apply the BB8 Daemons

cmoore24-24 commented 1 year ago

Looking into how BB8 rebalancing works, and how it can be applied system-wide. Will create transfer and rebalancing benchmarks, and identify proper use cases.

dynamic-entropy commented 1 year ago

Hi @cmoore24-24 Maybe you can evaluate against US T2 being highly locked scenarios, https://its.cern.ch/jira/browse/CMSTRANSF-709

cmoore24-24 commented 11 months ago

As it stands, BB8 is a little limited. If we could add some way to rebalance RSEs by account (so as to avoid workflow related rules that shouldn't be touched) I think it would have much more potential.

cmoore24-24 commented 4 months ago

Adding a task list to more finely track this issue; progress has been slow but I will be committing more time until it is complete.

Tasks

[x] BB8 Core Function Testing
- [x] Rebalance Rule Complete: Used the rebalance rule to successfully move a rule to a new RSE with the BB8 signature
- [x] List Rebalance Rule Candidates Complete: Returns a list of rules that match all of the parameters in the rule_clause list. This can be updated manually, but the majority of the additions are made automatically using options given directly to BB8.
- [x] Select Target RSE In Progress: Running into problems with RSESelector; unable to produce a list of target RSEs because of the error rucio.common.exception.InsufficientTargetRSEs: There are not enough target RSEs to fulfil the request at this time. Details: Target RSE set not sufficient for number of copies. (1 copies requested, RSE set size 0)
- [x] Rebalance RSE Waiting
[ ] Implement a small scale test of BB8

ericvaandering commented 2 days ago

Can we update this? I opened a new issue, but that may not be quite right with the info that's here.

dmwm / CMSRucio

Test and apply the BB8 Daemons #608

Tasks