Shorten compressed file name

[mt82]

Develop a script to shorten the compressed file name

[mt82]

from @mattfcs FYI I wrote the script to change file names, as I mentioned in the meeting earlier it's not quite as easy as changing any other info: /exp/icarus/app/home/icaruspro/mateusc_scripts/fix_file-name.sh it basically:

• Picks up a file from the compressed dataset
• creates a new .json file that has the new file name
• uses mv to change the actual file name
• declare the new file
• add file location for the new file
• retire the old one

I tested it in one of the test batches from the compression campaign: run2_compression_test_v09_82_02_01_offbeamnumiminbias_compressed_data that used to have:

compressed_data_dl12_fstrmOffBeamNUMIMINBIAS_run9873_9_20230523T041954_20240426T202846_compression-21590502-929b-4787-afae-d6e331a49711.root
compressed_data_dl10_fstrmOffBeamNUMIMINBIAS_run9704_7_20230329T062701_20240426T205241_compression-c9a70002-2e73-4959-83d7-11c28707412b.root
compressed_data_dl6_fstrmOffBeamNUMIMINBIAS_run9704_8_20230329T102742_20240427T004354_compression-dc92e624-de28-4793-b3be-24dd3452b488.root
compressed_data_dl3_fstrmOffBeamNUMIMINBIAS_run9594_6_20230302T074755_20240427T011428_compression-ed395cb8-a7a1-4df6-a7b8-518aa9d33082.root

and now has

compressed_data_dl12_fstrmOffBeamNUMIMINBIAS_run9873_9_20230523T041954.root
compressed_data_dl10_fstrmOffBeamNUMIMINBIAS_run9704_7_20230329T062701.root
compressed_data_dl6_fstrmOffBeamNUMIMINBIAS_run9704_8_20230329T102742.root
compressed_data_dl3_fstrmOffBeamNUMIMINBIAS_run9594_6_20230302T074755.root

I can't think of any other way to test it. Unless someone screams at me in the next hour I will start changing file names for each stream dataset. Given Promita is in Europe I imagine she will not be able to start reprocessing these files at least until tomorrow.

[mt82]

from @PromitaRoy Thanks @mattfcs. Let me know when you are done. I am not in a hurry since I need to retire/delete all the folders/sam datasets for the existing run2 reprocessing campaigns.

[mt82]

from @mattfcs It took way longer than I expected but the renaming seems to be done. There's a handful of files that got truncated names and I will fix then by hand now.

[mt82]

ok, bad news. My script renamed the files and created the proper metadata but apparently the declaration to SAM failed so now I need to redo it. Writting a script right now for that. I t will take a minute because I will have to make a list of the file locations first. I think I can have it fixed for tonight.

[mt82]

@mattfcs do you think it is worth to push the script /exp/icarus/app/home/icaruspro/mateusc_scripts/fix_file-name.sh?

[mt82]

from @mattfcs it should be done, you can try to run over run2_compression_production_v09_82_02_01_offbeambnbmajority_compressed_data see here