search

xrootd / xrootd-python

Python bindings for XRootD are now part of the main repository.
https://github.com/xrootd/xrootd
6 stars 8 forks source link

Is file migrated on tape? #7

Closed chaen closed 10 years ago

chaen commented 10 years ago

Hi,

I wanted to use the python API against the castor storage at CERN. With the nsls command, you can know whether a file has been replicated with the "m" flag. Is there a way to get the same information from pyxrootd? The Offline flag from the StatInfo tells you whether the file is not on disk, but is irrelevant if it is on disk and tape. And I am not quite sure, but I think that PoscPending means that we are waiting for the replication to happen. So how can I know whether the file is on disk AND on tape?

Thanks a lot!

esindril commented 10 years ago

Hi,

At this point (2.1.14) this information is not provided by the CASTOR XRootD plugin. We are looking into this functionality to be added maybe in the next major release.

Cheers, Elvin

chaen commented 10 years ago

Thanks for the quick answer. Actually I was wondering if xrootd was directly supported by castor or if it was a plugin? Do you know if there is a way to get the info via the standard xrootd cli for the time being? Thanks!

ljanyst commented 10 years ago

Technically CASTOR-XRootD is a server-side plugin to the XRootD framework, exporting the CASTOR data to XRootD users. So, as far as the clients are concerned, CASTOR can be seen as any other XRootD service. XRootD cli has various ways of sending "custom" queries to the server, but the server needs to know how to answer them, and this functionality is not provided for the time being.

ljanyst commented 10 years ago

Furthermore, the cli uses the same underlying API as xrootd-python.

chaen commented 10 years ago

Okay, thanks for the answers

chaen commented 10 years ago

In the same topic, I did not find any way to specify the service class in the python API. Did I miss it in the doc, or is this functionality not present yet? I can open another ticket for that if needed.

Cheers, Chris

ljanyst commented 10 years ago

Hi Chris,

when using CopyProcess, you can add the service class as a part of the URL CGI. Ie.

process.add_job( 'root://localhost//tmp/spam?svcClass=<svcClass>', '/tmp/spam3' )

Cheers, Lukasz

chaen commented 10 years ago

Excellent, thank you! May I suggest you to specify this in the doc? :-)

Cheers, Chris

ljanyst commented 10 years ago

Passing parameters in the URL is an XRootD functionality, but this particular parameter is specific to Castor an meaningless in general context, so this should end up in the Castor docs. I will ask the relevant people to do this.

esindril commented 10 years ago

It is already documented here: https://twiki.cern.ch/twiki/bin/view/DataManagement/X2CASTOR

chaen commented 10 years ago

Yes, they document the -OS and -OD options for xrdcp cli, but I did not know that we could pass it that way in the python API. Thanks

ljanyst commented 10 years ago

OK, I see where the confusion comes from. -OS and -OD do nothing more than appending the CGI string to the URL. @esindril perhaps the other syntax should be used throughout the docs in order to be consistent with other apps (including ROOT).

ljanyst commented 10 years ago

XRootD as of version 4.0.0 has a kXR_bkpexists flag. This will be reflected in XRootD.client.responses.StatInfo.flags as XRootD.client.flags.StatInfoFlags.BACKUP_EXISTS in XRootD-Python 0.2.1.

@esindril can you please make sure that the appropriate modifications are made in Castor 2.1.15 to set this flag?

esindril commented 10 years ago

Done. Will be enabled once there is a XRootD release supporting these flags. https://git.cern.ch/web/CASTOR.git/commit/32a050b0c94a0b0b707c2d4ce0d4d9a2ad7f7cc8