search

CMSDAS / b2g-long-exercise

B2G long exercise for the CMSDAS
https://cmsdas.github.io/b2g-long-exercise/
Other
1 stars 14 forks source link

Files not accessible on lxplus #4

Closed bregnery closed 4 years ago

bregnery commented 4 years ago

@lcorcodilos, the necessary bstar .root files are not accessible on lxplus. They are accessible on the lpc though.

AFJohan92 commented 4 years ago

For added context, this works on LPC: eosls -l /store/user/lcorcodi/bstar_nano/rootfiles/BprimeLH1200_bstar16.root On lxplus I cannot get to this file or directory. The path is also different than what is written on the instructions

lcorcodilos commented 4 years ago

Just to confirm, you tried this command correct? (with the lxplus eos path)

python bs_select.py -i /eos/home-l/lcorcodi/Storage/rootfiles/BprimeLH1200_bstar16.root -y 16

AFJohan92 commented 4 years ago

Yes: (timber-env) [jbonilla@lxplus763 BstarToTW_CMSDAS2020]$ python bs_select.py -i /eos/home-l/lcorcodi/Storage/rootfiles/BprimeLH1200_bstar16.root -y 16 Error in : no read permission, could not open file /eos/home-l/lcorcodi/Storage/rootfiles/BprimeLH1200_bstar16.root Error in : no read permission, could not open file /eos/home-l/lcorcodi/Storage/rootfiles/BprimeLH1200_bstar16.root Error in : no read permission, could not open file /eos/home-l/lcorcodi/Storage/rootfiles/BprimeLH1200_bstar16.root Error in : no read permission, could not open file /eos/home-l/lcorcodi/Storage/rootfiles/BprimeLH1200_bstar16.root Error in : no read permission, could not open file /eos/home-l/lcorcodi/Storage/rootfiles/BprimeLH1200_bstar16.root Error in : no read permission, could not open file /eos/home-l/lcorcodi/Storage/rootfiles/BprimeLH1200_bstar16.root Error in : no read permission, could not open file /eos/home-l/lcorcodi/Storage/rootfiles/BprimeLH1200_bstar16.root Error in : no read permission, could not open file /eos/home-l/lcorcodi/Storage/rootfiles/BprimeLH1200_bstar16.root Error in : no read permission, could not open file /eos/home-l/lcorcodi/Storage/rootfiles/BprimeLH1200_bstar16.root Traceback (most recent call last): File "bs_select.py", line 160, in run(options) File "bs_select.py", line 97, in run a.Cut('filters',a.GetFlagString(flags)) File "build/bdist.linux-x86_64/egg/TIMBER/Analyzer.py", line 241, in GetFlagString File "build/bdist.linux-x86_64/egg/TIMBER/Analyzer.py", line 188, in FilterColumnNames Exception: vector ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager,void>::GetColumnNames() => GetBranchNames: error in opening the tree Events (C++ exception of type runtime_error)

lcorcodilos commented 4 years ago

Hmmm... are you able to ls the file? I don't think it's permissions for the file but maybe for a parent folder?

[lcorcodi@lxplus768 BstarToTW_CMSDAS2020]$ ll /eos/home-l/lcorcodi/Storage/rootfiles
total 420172850
-rw-r--r--. 1 lcorcodi zh   314130699 Sep 22 02:49 BprimeLH1200_bstar16.root
lcorcodilos commented 4 years ago

While I think read permission is all that is needed, I gave execute permissions to the file just in case if you'd like to try again.

AFJohan92 commented 4 years ago

Still doesn't seem to work correctly.

On lpc, I can do this: (timber-env) [bonillaj@cmslpc178 BstarToTW_CMSDAS2020]$ eosls /store/user/lcorcodi/bstar_nano/rootfiles/BprimeLH1200_bstar16.root BprimeLH1200_bstar16.root and I was able to get the script to run with the following path (if I use the above it doesn't work, that is I need to add /eos/uscms/) (timber-env) [bonillaj@cmslpc178 BstarToTW_CMSDAS2020]$ python bs_select.py -i /eos/uscms/store/user/lcorcodi/bstar_nano/rootfiles/BprimeLH1200_bstar16.root -y 16

On lxplus I cannot access the file or folder at all, even if I play around with the path (below is a copy-paste of your command above) [jbonilla@lxplus771 ~]$ ll /eos/home-l/lcorcodi/Storage/rootfiles ls: cannot open directory /eos/home-l/lcorcodi/Storage/rootfiles: Operation not permitted Worst yet, I actually don't find your eos space on lxplus: [jbonilla@lxplus771 ~]$ eos ls /eos/cms/store/user

lcorcodilos commented 4 years ago

Okay so this is an issue with my EOS setup then (this is my first time using EOS space at lxplus). Is there a specific request I need to make to get space in /eos/cms/store/user? I noticed I wasn't there either but I was talking with another senior graduate student about it and he led me to believe that the mounted /eos/home-l/lcorcodi/ would be sufficient

lcorcodilos commented 4 years ago

Alternatively, we could put these files on /eos/cms/store/group/phys_b2g but I imagine I need permission to write there. @clelange is that correct?

AFJohan92 commented 4 years ago

Good question. I don't have one myself. I found these instructions: https://twiki.cern.ch/twiki/bin/view/CMS/T2CHCERN#Right_of_access_to_CERN_CMS_T2 It says you need to be at least 50% time at CERN but I would try anyway. Follow the instructions there to how to configure the ticket. I think we may want to figure out a temporary solution for this. Can you access this? ls /eos/user/j/jbonilla/samples/test.txt

lcorcodilos commented 4 years ago

I can access that, yes. What's the magic? :-)

AFJohan92 commented 4 years ago

That is my cernbox space. I somehow found how to access that from lxplus, but idk how to access it from the lpc...

lcorcodilos commented 4 years ago

Just to confirm, all of the students will be on lxplus, correct?

AFJohan92 commented 4 years ago

I think so, because LPC is DOE and not all participants are cleared to use it.

AFJohan92 commented 4 years ago

Got it! Just had to massage the path a bit. The following command works from lxplus: python bs_select.py -i root://cmsxrootd.fnal.gov//store/user/lcorcodi/bstar_nano/rootfiles/BprimeLH1200_bstar16.root -y 16

lcorcodilos commented 4 years ago

Ah nice. Will that work for students who don't have LPC access? I thought not but I could be wrong.

clelange commented 4 years ago

You can/should put them in a new directory in /eos/user/c/cmdas/long-exercises - the students have read access and you have write access

lcorcodilos commented 4 years ago

@clelange thanks! Just to check, how much available space is there? The files are about 450 GB all together

clelange commented 4 years ago

Should not be a problem, we allocated 500 GB per long exercise. You can use eos quota

AFJohan92 commented 4 years ago

@lcorcodilos let me know when you copy over the files so I can change the paths in the tutorial

AFJohan92 commented 4 years ago

Ah nice. Will that work for students who don't have LPC access? I thought not but I could be wrong.

Good point, maybe the redirector is available to everyone. Else one can use xrootd-cms.infn.it or cms-xrd-global.cern.ch

lcorcodilos commented 4 years ago

Okay so everything is at /eos/user/c/cmsdas/long-exercises/bstarToTW except for JetHT D for 2018. It's > 50 GB so scp was complaining and I couldn't get xrdcp to work (it would hang and then say "Operation expired"). This file was made from smaller pieces but after I did the hadd, the smaller pieces were deleted. I'll remake the pieces overnight though and then move them over individually via scp in the morning. There's certainly enough MC to get going though.

AFJohan92 commented 4 years ago

Btw I updated the tutorial to have the new paths and it worked for people :)