dCache / dcache

dCache - a system for storing and retrieving huge amounts of data, distributed among a large number of heterogenous server nodes, under a single virtual filesystem tree with a variety of standard access methods
https://dcache.org
284 stars 135 forks source link

Pinning via Rest-API/dcache-view and check pinnig status #5004

Open christianvoss opened 5 years ago

christianvoss commented 5 years ago

Dear dCache development team,

while discussing the possibilities of pinning files for a certain time frame we wondered whether support for this could be integrated either in the Rest-API, which we have not seen in case it is already in place, or directly via dCache-view.

Pinning whole directories and getting the remaining pinning time would be appreciated.

Thanks a lot, Christian

onnozweers commented 4 years ago

Hi Christian,

I asked this recently to the dCache devs, and the answer was that there is no pinning lifetime when talking through the API. So users will have to change QoS to "tape-and-disk" to stage a dataset, then when they're done, they would need to change the QoS back to "tape".

Cheers, Onno

alrossi commented 4 years ago

dCache team: Since this has been marked as an open enhancement request, can we perhaps reopen a discussion about it? This is something, as Dmitry has mentioned, which our users here at FNAL wish to have as well (that is, a lifetime specification). I know there was some resistance to doing this via an additional parameter on the API in preference for a JSON policy object. While that object may need to be thought out, perhaps we could just start with it containing only the lifetime specification. That way, when it gets fully fleshed out, the API need not change.

alrossi commented 4 years ago

Also, pinning directories is something FNAL also wants, optionally recursively. I am ready to take over development of this feature.

alrossi commented 3 years ago

Can we close this issue? We have prototype Bulk PIN activity now, along with Bulk UPDATE-QOS, and also the ongoing WLCG discussion.