dCache / dcache

dCache - a system for storing and retrieving huge amounts of data, distributed among a large number of heterogenous server nodes, under a single virtual filesystem tree with a variety of standard access methods
https://dcache.org
276 stars 133 forks source link

Request: constraining APEL Star accounting in storage-/pool-/linkgroups #4900

Open thdesy opened 5 years ago

thdesy commented 5 years ago

Moin all,

the Star accounting script selects/aggregates its accounting info by groups as granularity. But as we have internal group space that we do not intend to be published in the aggregated accounting infos, an ability on a more finer restriction on storage-/pool- or linkgroups would be useful as to exclude groups not to be published.

Cheers, Thomas

paulmillar commented 5 years ago

Could you elaborate on the use-case here?

Are there pools that a VO is able to write into, but should not be included in the accounting? Or are the VOs for which the script should not generate star records?

thdesy commented 5 years ago

Hi Paul,

we have next to pledged resources a large local user storage in the dCache instance. As we would like to publish only the pledged resources, we would like to exclude the local user resources. Probably on storage group level as we can select on it.

paulmillar commented 5 years ago

Just to check I understand correct, there are several pools that members of a VO may write into. There is a subset of these pools (let's call them WLCG pools) that should be reported and another subset of pools (let's call them LOCAL pools) that should be excluded from the report.

The WLCG and LOCAL sets of pools are non-overlapping and, combined, these two sets of pools cover all pools that this VO can write into. In other words, all pools that a member of this VO can write into is either a WLCG pool or is a LOCAL pool.

The StAR record should include all files that use at least one WLCG pool, and should exclude any file with data stored only on LOCAL pools.

thdesy commented 5 years ago

Hi Paul - yes, exactly. We would like to exclude files, that are stored only on LOCAL pools.