search

DUNE / data-mgmt-ops

3 stars 3 forks source link

protodune SP reco2 files missing from rucio #716

Open hschellman opened 2 months ago

hschellman commented 2 months ago

I'm looking for reco2 files to point to for the tutorials. This run is on the good runs list at: https://wiki.dunescience.org/wiki/ProtoDUNE-SP_datasets#Production_4_Reco_2. so they should be available.

Can someone see what happened to these files? Presumably these are the ones people would be analyzing and for now they need to use sam.

metacat query "files where core.data_tier=reco-recalibrated and 5204 in core.runs limit 1"

dune:np04_raw_run005204_0004_dl10_reco1_38929423_0_20201107T054446Z_reco2_21365431_0_20210622T232038Z.root

rucio list-file-replicas \ dune:np04_raw_run005204_0004_dl10_reco1_38929423_0_20201107T054446Z_reco2_21365431_0_20210622T232038Z.root

2024-09-13 11:23:37,780 ERROR Data identifier not found. Details: Data identifier 'dune:np04_raw_run005204_0004_dl10_reco1_38929423_0_20201107T054446Z_reco2_21365431_0_20210622T232038Z.root' not found

StevenCTimm commented 2 months ago

This is sort of vague, can you at least give us a SAM query of the ones that you found missing?

hschellman commented 2 months ago

I posted the whole thing as an edit. Github got overly excited while I was typing

https://github.com/DUNE/data-mgmt-ops/issues/716 [716.png] protodune SP reco2 files missing from rucio · Issue #716 · DUNE/data-mgmt-opshttps://github.com/DUNE/data-mgmt-ops/issues/716 github.comhttps://github.com/DUNE/data-mgmt-ops/issues/716

On Sep 13, 2024, at 10:30 AM, Steven Timm @.***> wrote:

[This email originated from outside of OSU. Use caution with links and attachments.]

This is sort of vague, can you at least give us a SAM query of the ones that you found missing?

— Reply to this email directly, view it on GitHubhttps://github.com/DUNE/data-mgmt-ops/issues/716#issuecomment-2349348415, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIA37DOII5KVNPRQ567KZ7DZWMHLTAVCNFSM6AAAAABOFWHRRGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBZGM2DQNBRGU. You are receiving this because you authored the thread.Message ID: @.***>

StevenCTimm commented 2 months ago

OK well if a file like that is assigned to "dune" scope in metacat then it was never declared to rucio yet. .this is a bit of a surprise because at least the PDSPPRod4 reco is definitely declared to rucio. Let me have a look.

hschellman commented 2 months ago

samweb list-files "run_number 5204 and data_tier reco-recalibrated with limit 1" -e dune np04_raw_run005204_0046_dl3_reco1_13835485_0_20201109T225723Z_reco2_21365863_0_20210622T232155Z.root [schellma@dunegpvm03 ~]$ [schellma@dunegpvm03 ~]$ samweb locate-file np04_raw_run005204_0046_dl3_reco1_13835485_0_20201109T225723Z_reco2_21365863_0_20210622T232155Z.root -e dune enstore:/pnfs/dune/tape_backed/dunepro/protodune-sp/full-reconstructed/2021/detector/physics/PDSPProd4/00/00/52/04(2167@fl9103l8)

StevenCTimm commented 2 months ago

so the file in question is a 7GeV run The parent reco1 file is in rucio but this reco2 file is not.

I am not sure I was ever informed of the existence of this dataset before now. It appears that we never declared the reco2 files to rucio.. as I said also the "dune" namespace in metacat contains predominantly stuff that didn't fit in any of the known rucio scopes and has not yet been declared to rucio.. you can see that it's quite big. We'll get this eventually but not right away, there are much more important fish to fry.

hschellman commented 2 months ago

Yes, current data taking has priority but I thought this needed to be flagged, Maybe Jake’s CAF’s are so good nobody is even looking at this anymore.

On Sep 13, 2024, at 10:59 AM, Steven Timm @.***> wrote:

[This email originated from outside of OSU. Use caution with links and attachments.]

so the file in question is a 7GeV run The parent reco1 file is in rucio but this reco2 file is not.

I am not sure I was ever informed of the existence of this dataset before now. It appears that we never declared the reco2 files to rucio.. as I said also the "dune" namespace in metacat contains predominantly stuff that didn't fit in any of the known rucio scopes and has not yet been declared to rucio.. you can see that it's quite big. We'll get this eventually but not right away, there are much more important fish to fry.

— Reply to this email directly, view it on GitHubhttps://github.com/DUNE/data-mgmt-ops/issues/716#issuecomment-2349438791, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIA37DMU3TB3Z2MIXRVBIZDZWMKWJAVCNFSM6AAAAABOFWHRRGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBZGQZTQNZZGE. You are receiving this because you authored the thread.Message ID: @.***>