Providing a group when injecting new block in phedex

[vlimant]

https://cmsweb.cern.ch/phedex/datasvc/doc/inject

does not allow to pass the group which injects the block. Is there a hard limitation to providing a group ?

[nikmagini]

Hi Jean-Roch,

by design, the "group" of a block replica is set via subscription, not at injection. If you want to assign a newly injected block replica to a group, you need to subscribe it to the same node after injection.

Closing since it's not a PhEDEx issue

Cheers N

[vlimant]

Could be more economical to be able to provide the group upfront.

[TonyWildish]

perhaps, but would not be correct. There may be several replicas with different group owners.

Besides, the semantics are different. Injection tells PhEDEx 'these files exist'. Subscriptions tell PhEDEx that 'this group wants this data'. Hence the choice, which isn't going to be changed anytime soon.

[nikmagini]

I would also disagree on the 'economical' side: disentangling groups from subscriptions would require changes deep down in PhEDEx central agents. On the other hand, WMAgent already knows how to subscribe blocks after injection - you just need to enable it for one additional site (i.e. the injection site)

[vlimant]

The wma subscriptions are made, as far as I can tell by watching it, by dataset. The block subscription after block injection will require a good large number of auto-approved block subscription, with no_mail (economical for me is phedexid, but it does not seem to be an issue to get the counter rolling). Would be fine by me, as long as dataops data gets own by dataops and is accounted for.

[TonyWildish]

you can make a dataset subscription that will automatically include new blocks as they are created (this is the default). So there's no need for wma to make more than the one suscription per dataset.

[nikmagini]

Hi Tony,

subscribing by dataset wouldn't work for this usecase - blocks are produced across multiple production sites, and we don't want to subscribe the whole dataset to all of them.

N

[ticoann]

I was told that making block level subscription is not recommended (when Ricky was here). Isn't that a problem anymore?

[nikmagini]

Not a problem since PhEDEx 4.0.0 deployed in 2011

[TonyWildish]

Ops have their own bias against making block-level subscriptions, but technically there's no reason not to use them. I personally think they should be used much more, using only dataset-level subscriptions has its own problems (not all datasets fit at all sites etc).

So I suggest you make sure the Ops management know what's going on, but yes, use block-level subscriptions.