ssl-hep / ServiceX

ServiceX - a data delivery service pilot for IRIS-HEP DOMA
BSD 3-Clause "New" or "Revised" License
19 stars 21 forks source link

EOS Directory DID Finder #557

Closed BenGalewsky closed 1 month ago

BenGalewsky commented 1 year ago

There are some resources that are on EOS, but not registered with Rucio. This DID Finder will report all of the files in an EOS directory. A DID Path argument will specify whether to descend into subdirectories.

gordonwatts commented 1 year ago

Let's see... I'm riffing here...

Here is a directory:

eos://eos/user/g/gwatts/public/data/mc16_13TeV.311423.MGPy8EG_A14NNPDF23_NNPDF31ME_HSS_LLP_mH600_mS150_lthigh.deriv.DAOD_EXOT15.e7357_e5984_s3234_r10201_r10210_p4696

That directory should be visible publically, and has a single file in it.

And what if they want to run on one file?

eos:///eos/user/g/gwatts/public/data/mc16_13TeV.311423.MGPy8EG_A14NNPDF23_NNPDF31ME_HSS_LLP_mH600_mS150_lthigh.deriv.DAOD_EXOT15.e7357_e5984_s3234_r10201_r10210_p4696/DAOD_EXOT15.26710706._000001.pool.root.1

Things to consider:

bbockelm commented 1 year ago

Note "EOS" is the name of software while, conceptually, here you're only using the fact it's a filesystem remotely accessible over a protocol like WebDAV or XRootD. There's several instance of this at CERN and maybe a dozen or two more outside CERN.

Might be useful to define scope here -- is it the concept of a remote filesystem? Any XRootD endpoint? Any EOS instance? CERN EOS? CERN ATLAS EOS? Etc.