search

cms-analysis / HiggsAnalysis-KITHiggsToTauTau

http://cms-analysis.github.io/HiggsAnalysis-KITHiggsToTauTau/
6 stars 18 forks source link

cleaning of filelists incompatible with recent Kappa version #31

Closed rfriese closed 8 years ago

rfriese commented 8 years ago

Dear all, I cleaned the filelists a lot more and would like to hear your feedback first before I merge this into the master.

I removed those sample lists that do not work with the current master of the Kappa/Artus/KITHiggs Repository. One would have to jump back in git anyway to be able to run on them or could re-skim them. In Case of 8TeV this should be possible still since we keep Kappa Backward-compatible to CMSSW_5_3_X. For Spring15 we will never run on them ever again.

Additionaly, I removed the NAF and XROOTD filelists. It seemed to me that they have not been maintained in the last months and I do not see a need for them since DCAP is reliable and works site-independent.

Do you agree with these changes? There's no urgent need for cleaning them, but atm I can't realy see a reason for keeping them also.

Cheers

thomas-mueller commented 8 years ago

One quick comment: The creation of NAF* filelists was disabled in this commit. I am not so sure, if the NAF* (in contrast to the less reliable XROOT*) filelists are really "deprecated". I think there are cases where they allow a much faster access to the files. The also ease copying single files for testing purposes somewhere else, because one can simply look up the full paths there.

Another point: cleaning up is always good! I agree, that we can delete all filelists and the files on dCache, that we anyhow cannot process with the current code. But I wont't be able to check this PR in detail. (Again here: constant nicks would make life easier, as only the files of superceeded dates would have to be deleted.)

pistone87 commented 8 years ago

Hi Raphael,

I would like to keep the 8TeV samples, since I might need them again. For running on them I still have a very old version of Kappa&Co. locally, so the filelists in the new version of Kappa&Co. can be deleted (I have the filelists that I need in my old working directory).

The important point is that the files are not deleted. ;)

Thanks.

Cheers, Claudia

On 07/13/2016 05:50 PM, Raphael Friese wrote:

Dear all, I cleaned the filelists a lot more and would like to hear your feedback first before I merge this into the master.

I removed those sample lists that do not work with the current master of the Kappa/Artus/KITHiggs Repository. One would have to jump back in git anyway to be able to run on them or could re-skim them. In Case of 8TeV this should be possible still since we keep Kappa Backward-compatible to CMSSW_5_3_X. For Spring15 we will never run on them ever again.

Additionaly, I removed the NAF and XROOTD filelists. It seemed to me that they have not been maintained in the last months and I do not see a need for them since DCAP is reliable and works site-independent.

Do you agree with these changes? There's no urgent need for cleaning them, but atm I can't realy see a reason for keeping them also.

Cheers

    You can view, comment on, or merge this pull request online at:

https://github.com/cms-analysis/HiggsAnalysis-KITHiggsToTauTau/pull/31

    Commit Summary

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cms-analysis/HiggsAnalysis-KITHiggsToTauTau/pull/31, or mute the thread https://github.com/notifications/unsubscribe/AFBE3kq6z0kA1sAm_gAnOdz6eSGHenGoks5qVQljgaJpZM4JLkS0.

rfriese commented 8 years ago

Spring15 and 8TeV samples had what you call "constant nicknames" :)

I for sure won't delete anything from someone else' dCache folder, even if that might be technically possible and Thomas put it in bold letters.

with

gfal-rm srm://dcache-se-cms.desy.de:8443/srm/managerv2?SFN=/pnfs/desy.de/cms/tier2/store/user/... -R

You can now recursively delete folders.

I checked running with Run2Analysis on 1000 events, which is the usual test job size. When reading the file from dcap it took 24s, from /pnfs/ 25s. Even if one might want to step back not using dcap the /pnfs/ paths can be extracted easily.

rfriese commented 8 years ago

I just checked the concern with copying files locally. You can use

gfal-copy dcap://dcache-cms-dcap.desy.de//pnfs/... file:////nfs/...

good thing about that: it works globally no matter where on earth you are