search

rvizarreta / Mx2_AlignmentCalibration

In this repository I'll keep all the code used for the analysis and calibration duties I have for the Mx2 detector, a Near Detector Prototype from the DUNE experiment.
1 stars 0 forks source link

Run alignment calibration source files #6

Closed rvizarreta closed 1 year ago

rvizarreta commented 1 year ago

I got this issue after correcting the paths on make_s2s_script.py:

source master_s2s_script.sh bash: cd: /scripts/s2s: No such file or directory -- s2s script parameters -- -- data_processing directory /minerva/data/users/minervacal/data_processing_s2s_eroica -- data_processing version v10r8pre -- first run 846 2 -- last run 898 11 /minerva/data/users/minervacal/data_processing_s2s_eroica/ Traceback (most recent call last): File "make_playlist.py", line 28, in list = os.listdir(rundir) OSError: [Errno 2] No such file or directory: '/minerva/data/users/minervacal/data_processing_s2s_eroica/'

* Creating plex map **

bash: cd: /options: No such file or directory cp: cannot stat ‘PlexWriter.opts’: No such file or directory bash: [: -le: unary operator expected bash: Gaudi.exe: command not found Could not make plex map! Connection to minervagpvm05.fnal.gov closed.

rvizarreta commented 1 year ago

I create a local variable from my work area location for $ROCKMUONCALIBRATIONROOT. Now I have a new issue:

source master_s2s_script.sh -- s2s script parameters -- -- data_processing directory /minerva/data/users/minervacal/data_processing_s2s_eroica -- data_processing version v10r8pre -- first run 846 2 -- last run 898 11 /minerva/data/users/minervacal/data_processing_s2s_eroica/ Traceback (most recent call last): File "make_playlist.py", line 28, in list = os.listdir(rundir) OSError: [Errno 2] No such file or directory: '/minerva/data/users/minervacal/data_processing_s2s_eroica/'

* Creating plex map **

bash: [: -le: unary operator expected bash: Gaudi.exe: command not found Could not make plex map! Connection to minervagpvm05.fnal.gov closed.

rvizarreta commented 1 year ago

Chris comments about current issue:

OK, so I think master_s2s_script.sh is something that will automatically run through a bunch of IOVs (intervals of validity). This is intended to automate the process once everything is very stable Right now, I don't think you want to do that. You should basically look at what it is doing and do the steps one by one, by had, because you will get lots of errors So looking at my readme, basically you want to skip step 0 for now -- that is making the ntuple files. At some point we will want to integrate this with what Carlos is doing on the production side We probably aren't going to use IOVs for s2s, we will just do it once, so we don't need to do that step either I would basically go to where it says "details of Step 3" and follow that For alignment, you will want to use the output of the ReadNT step

rvizarreta commented 1 year ago

I added a sample .root file to test this calibration, also I am sourcing manually instead of sourcing master_s2s_script.sh. Note that on make_s2s.sh I had to comment on each exit command because it closed my ssh connection. Now I am looking at the error generated by each step:

source make_s2s.sh /minerva/data/users/rvizarr/minervacal v10r8pre 847 2 898 11 -- s2s script parameters -- -- data_processing directory /minerva/data/users/rvizarr/minervacal -- data_processing version v10r8pre -- first run 847 2 -- last run 898 11 rundir:/minerva/data/users/rvizarr/minervacal/ 1 1 /minerva/data/users/rvizarr/minervacal/minervatest_01_01

* Creating plex map **

bash: Gaudi.exe: command not found Could not make plex map! mv: cannot stat ‘plex.root’: No such file or directory

* Reading Ntuples ***

bash: ./ReadNT: No such file or directory Could not read ntuples!

Creating summary tuple

bash: root: command not found Could not make summary tuple!

* Creating constants ****

bash: root: command not found Could not make s2s constants! bash: ./DoFitCorrection: No such file or directory Could not make fit correction file! cp: cannot stat ‘/minerva/data/calibrations/s2s_tables/minerva/v5/s2s_constants_gen2_prime_01.txt’: No such file or directory Traceback (most recent call last): File "IterateS2SConstants.py", line 1, in fold = open('s2s_constants_old.txt', 'r') IOError: [Errno 2] No such file or directory: 's2s_constants_old.txt' Iteration combination step has failed! Traceback (most recent call last): File "MakeTimeHeader.py", line 3, in fin = open( 'time_header.txt', 'r' ) IOError: [Errno 2] No such file or directory: 'time_header.txt' cat: time_header2.txt: No such file or directory cat: s2s_constants.txt: No such file or directory rm: cannot remove ‘s2s_constants.txt’: No such file or directory mv: cannot stat ‘time_header2.txt’: No such file or directory rm: cannot remove ‘time_header.txt’: No such file or directory mv: cannot stat ‘ReadNT.root’: No such file or directory mv: cannot stat ‘error_strips.txt’: No such file or directory mv: cannot stat ‘s2s_constants_new.txt’: No such file or directory mv: cannot stat ‘s2s_constants_old.txt’: No such file or directory mv: cannot stat ‘plex.root’: No such file or directory mv: cannot stat ‘summary.root’: No such file or directory mv: cannot stat ‘FitCorrection.root’: No such file or directory mv: cannot stat ‘fit_corrections.txt’: No such file or directory

***** Done! ****

rvizarreta commented 1 year ago

I forgot to make the make command before the start, that's why I was having some of the previous issues. The make command will essentially run the Makefiel file. After doing this, the current issue is:

source make_s2s.sh /minerva/data/users/rvizarr/minervacal v10r8pre 847 2 898 11 -- s2s script parameters -- -- data_processing directory /minerva/data/users/rvizarr/minervacal -- data_processing version v10r8pre -- first run 847 2 -- last run 898 11 rundir:/minerva/data/users/rvizarr/minervacal/ 1 1 /minerva/data/users/rvizarr/minervacal/minervatest_01_01

* Creating plex map **

bash: Gaudi.exe: command not found Could not make plex map! mv: cannot stat ‘plex.root’: No such file or directory

* Reading Ntuples ***

Getting ntuples... Processing 657 tracks for 40001/1 Make time header... There are 1 files Filling summary histograms... Determining truncated mean with 8 iterations... Filling summary histograms for module 0... Filling summary histograms for module 10... Filling summary histograms for module 20... Filling summary histograms for module 30... Filling summary histograms for module 40... Filling summary histograms for module 50... Filling summary histograms for module 60... Filling summary histograms for module 70... Filling summary histograms for module 80... Filling summary histograms for module 90... Filling summary histograms for module 100... Filling summary histograms for module 110...

Creating summary tuple

root [0] Processing MakeSummaryTuple.C... Error in : file plex.root does not exist Error: illegal pointer to class object plex 0x0 1383 MakeSummaryTuple.C:35: Interpreter error recovered

* Creating constants ****

root [0] Processing PrintConstants.C... Error: illegal pointer to class object nts 0x0 1383 PrintConstants.C:31: Interpreter error recovered Processing 657 tracks for 40001 / 1 Determining truncated mean Fitting... cp: cannot stat ‘/minerva/data/calibrations/s2s_tables/minerva/v5/s2s_constants_gen2_prime_01.txt’: No such file or directory Traceback (most recent call last): File "IterateS2SConstants.py", line 1, in fold = open('s2s_constants_old.txt', 'r') IOError: [Errno 2] No such file or directory: 's2s_constants_old.txt' Iteration combination step has failed! cat: s2s_constants.txt: No such file or directory

***** Done! ****

rvizarreta commented 1 year ago

It seems that now it is trying to read the tuples, but failing due to the plex map error. After the weekend I'll ask my team about how to run Gaudi. I also needed to point the $MINEVA_GEOMETRY environment variable to the following location: /minerva/app/users/rvizarr/cmtuser/Minerva_v22r1p1_2x2/Det/MinervaDDDB/DDDB/

rvizarreta commented 1 year ago

After some discussion with Chris we came to the conclusion that just need to use the part of the code related to the alignment. This is bringing new issues as it seems that the ReadNT.root file is empty.

rvizarreta commented 1 year ago

I am not sure if I can modify ReadNT.C as it is already compiled on a .o file. But I'll try to generate a new root file using the .C file. The problem seems to be definitely here, will do more exploration.

rvizarreta commented 1 year ago

I created a copy of the original ntuple just to see if the problem was related to being only one tuple available, and the ReadNT.cxx file worked now. The root file produced is ReadNT.root now has some interesting plots:

image
rvizarreta commented 1 year ago

The alignment constants file is being generated but it is empty. Now the DoAlignment.C file shows a different error:

root -l DoAlignment.C

root [0] Processing DoAlignment.C... Warning in : file ReadNT.root probably not closed, trying to recover Info in : ReadNT.root, recovered key TProfile3D:moduleLposBase at address 218 Info in : ReadNT.root, recovered key TProfile2D:shifts at address 651812 Info in : ReadNT.root, recovered key TProfile:p_energy_path at address 868509 Info in : ReadNT.root, recovered key TH2I:h_energy_path at address 870560 Info in : ReadNT.root, recovered key TProfile:p_energy_over_path at address 874855 Info in : ReadNT.root, recovered key TH2I:h_energy_over_path at address 876909 Info in : ReadNT.root, recovered key TH2D:TM_energy_channel at address 881533 Info in : ReadNT.root, recovered key TProfile2D:p_energy_channel_all at address 980887 Info in : ReadNT.root, recovered key TH2D:h_zero_fraction at address 1142155 Info in : ReadNT.root, recovered key TH2D:nh_crate0_croc1 at address 1155941 Info in : ReadNT.root, recovered key TProfile2D:mev_crate0_croc1 at address 1159029 Info in : ReadNT.root, recovered key TProfile2D:pe_crate0_croc1 at address 1193388 Info in : ReadNT.root, recovered key TProfile2D:q_crate0_croc1 at address 1228018 Info in : ReadNT.root, recovered key TH2D:nh_crate0_croc2 at address 1262366 Info in : ReadNT.root, recovered key TProfile2D:mev_crate0_croc2 at address 1265411 Info in : ReadNT.root, recovered key TProfile2D:pe_crate0_croc2 at address 1295555 Info in : ReadNT.root, recovered key TProfile2D:q_crate0_croc2 at address 1329180 Info in : ReadNT.root, recovered key TH2D:nh_crate0_croc3 at address 1362510 Info in : ReadNT.root, recovered key TProfile2D:mev_crate0_croc3 at address 1365612 Info in : ReadNT.root, recovered key TProfile2D:pe_crate0_croc3 at address 1399202 Info in : ReadNT.root, recovered key TProfile2D:q_crate0_croc3 at address 1433636 Info in : ReadNT.root, recovered key TH2D:nh_crate0_croc4 at address 1467822 Info in : ReadNT.root, recovered key TProfile2D:mev_crate0_croc4 at address 1470954 Info in : ReadNT.root, recovered key TProfile2D:pe_crate0_croc4 at address 1504420 Info in : ReadNT.root, recovered key TProfile2D:q_crate0_croc4 at address 1538404 Info in : ReadNT.root, recovered key TH2D:nh_crate0_croc5 at address 1571993 Info in : ReadNT.root, recovered key TProfile2D:mev_crate0_croc5 at address 1573648 Info in : ReadNT.root, recovered key TProfile2D:pe_crate0_croc5 at address 1587301 Info in : ReadNT.root, recovered key TProfile2D:q_crate0_croc5 at address 1600958 Info in : ReadNT.root, recovered key TH2D:nh_crate0_croc6 at address 1614746 Info in : ReadNT.root, recovered key TProfile2D:mev_crate0_croc6 at address 1616403 Info in : ReadNT.root, recovered key TProfile2D:pe_crate0_croc6 at address 1629553 Info in : ReadNT.root, recovered key TProfile2D:q_crate0_croc6 at address 1642743 Info in : ReadNT.root, recovered key TH2D:nh_crate0_croc7 at address 1655990 Info in : ReadNT.root, recovered key TProfile2D:mev_crate0_croc7 at address 1656465 Info in : ReadNT.root, recovered key TProfile2D:pe_crate0_croc7 at address 1657175 Info in : ReadNT.root, recovered key TProfile2D:q_crate0_croc7 at address 1657881 Info in : ReadNT.root, recovered key TH2D:nh_crate0_croc8 at address 1658583 Info in : ReadNT.root, recovered key TProfile2D:mev_crate0_croc8 at address 1659057 Info in : ReadNT.root, recovered key TProfile2D:pe_crate0_croc8 at address 1659767 Info in : ReadNT.root, recovered key TProfile2D:q_crate0_croc8 at address 1660473 Info in : ReadNT.root, recovered key TH2D:nh_crate1_croc1 at address 1661175 Info in : ReadNT.root, recovered key TProfile2D:mev_crate1_croc1 at address 1661648 Info in : ReadNT.root, recovered key TProfile2D:pe_crate1_croc1 at address 1662357 Info in : ReadNT.root, recovered key TProfile2D:q_crate1_croc1 at address 1663062 Info in : ReadNT.root, recovered key TH2D:nh_crate1_croc2 at address 1663762 Info in : ReadNT.root, recovered key TProfile2D:mev_crate1_croc2 at address 1664235 Info in : ReadNT.root, recovered key TProfile2D:pe_crate1_croc2 at address 1664945 Info in : ReadNT.root, recovered key TProfile2D:q_crate1_croc2 at address 1665650 Info in : ReadNT.root, recovered key TH2D:nh_crate1_croc3 at address 1666351 Info in : ReadNT.root, recovered key TProfile2D:mev_crate1_croc3 at address 1666825 Info in : ReadNT.root, recovered key TProfile2D:pe_crate1_croc3 at address 1667535 Info in : ReadNT.root, recovered key TProfile2D:q_crate1_croc3 at address 1668240 Info in : ReadNT.root, recovered key TH2D:nh_crate1_croc4 at address 1668941 Info in : ReadNT.root, recovered key TProfile2D:mev_crate1_croc4 at address 1669415 Info in : ReadNT.root, recovered key TProfile2D:pe_crate1_croc4 at address 1670125 Info in : ReadNT.root, recovered key TProfile2D:q_crate1_croc4 at address 1670831 Info in : ReadNT.root, recovered key TH2D:nh_crate1_croc5 at address 1671532 Info in : ReadNT.root, recovered key TProfile2D:mev_crate1_croc5 at address 1672006 Info in : ReadNT.root, recovered key TProfile2D:pe_crate1_croc5 at address 1672715 Info in : ReadNT.root, recovered key TProfile2D:q_crate1_croc5 at address 1673421 Info in : ReadNT.root, recovered key TH2D:nh_crate1_croc6 at address 1674123 Info in : ReadNT.root, recovered key TProfile2D:mev_crate1_croc6 at address 1674597 Info in : ReadNT.root, recovered key TProfile2D:pe_crate1_croc6 at address 1675307 Info in : ReadNT.root, recovered key TProfile2D:q_crate1_croc6 at address 1676013 Info in : ReadNT.root, recovered key TH2D:nh_crate1_croc7 at address 1676715 Info in : ReadNT.root, recovered key TProfile2D:mev_crate1_croc7 at address 1677189 Info in : ReadNT.root, recovered key TProfile2D:pe_crate1_croc7 at address 1677899 Info in : ReadNT.root, recovered key TProfile2D:q_crate1_croc7 at address 1678605 Info in : ReadNT.root, recovered key TH2D:nh_crate1_croc8 at address 1679307 Info in : ReadNT.root, recovered key TProfile2D:mev_crate1_croc8 at address 1679781 Info in : ReadNT.root, recovered key TProfile2D:pe_crate1_croc8 at address 1680491 Info in : ReadNT.root, recovered key TProfile2D:q_crate1_croc8 at address 1681197 Warning in : successfully recovered 73 keys Error: illegal pointer to class object p3ModLposBase 0x0 2158 DoAlignment.C:71: Interpreter error recovered

rvizarreta commented 1 year ago

Take some time to clean your workspace, maybe create new folders dedicated only to the alignment. Is confusing to work with so many files.

rvizarreta commented 1 year ago

The issue was related with a piece of code Chris omitted from his original code. He provided me with an oldest version that makes the 3D TProfile object: https://cdcvs.fnal.gov/cgi-bin/public-cvs/cvsweb-public.cgi/AnalysisFramework/Cal/RockMuonCalibration/scripts/s2s/Attic/ReadNT.C?rev=1.1.2.1;content-type=text%2Fplain;cvsroot=mnvsoft

rvizarreta commented 1 year ago

Wait, it seems like p3ModLposBase (new object) is the same as p_moduleLposBase (current object). Ok I just changed the name on the DoAlignment.C file of the object and started plotting. It fails but now I can manage it.

image
rvizarreta commented 1 year ago

On DoAlignment.C we iterate over the 240 planes. For calibration, do we need to use only the first module? for (int modbin = 1; modbin <= 240; ++modbin). It seems that changing the loop won't run properly. For now let's just run everything and see the final output.

rvizarreta commented 1 year ago

This is working now, we just need to run it again with more ntuples.