search

cta-observatory / protopipe

Prototype data analysis pipeline for the Cherenkov Telescope Array Observatory
https://protopipe.readthedocs.io/en/latest/
Other
5 stars 13 forks source link

write_dl2.py is missing some variables #65

Closed adonini closed 3 years ago

adonini commented 3 years ago

The scriptwrite_dl2.py as it is doesn't work.

Line 79: allowed_tels, cams_and_foclens = prod3b_array(filenamelist[0], site, array) the definition of the variable subarray is missing

Line 219 some values defined in PreparedEvent are missing: dl1_phe_image_mask_reco, dl1_phe_image_1stPass, calibration_status and leakage_dict.

After adding these values the script works fine.

HealthyPear commented 3 years ago

I guess you are using ctapipe 0.8? If yes, this goes into the direction of PR #58

I am performing some tests using the grid because I changed the data training format (until I can use the one from ctapipe). I will add these fixes to the "new" write_dl2 .

many thanks!

adonini commented 3 years ago

I guess you are using ctapipe 0.8?

No, I'm using ctapipe 0.7 and the master branch to analyse my simulations. But it shouldn't change with the 0.8, unless you changed the script.

HealthyPear commented 3 years ago

No, I'm using ctapipe 0.7 and the master branch to analyse my simulations. But it shouldn't change with the 0.8, unless you changed the script.

Aha. Ok, this is a problem then.

Line 79: allowed_tels, cams_and_foclens = prod3b_array(filenamelist[0], site, array) the definition of the variable subarray is missing

I cannot try to launch it now, but I fail to see the bug here. Who is asking for a variable named subarray? The function used at line 79 is defined as prod3b_array(fileName, site, array) and in seems correctly called.

Line 219 some values defined in PreparedEvent are missing: dl1_phe_image_mask_reco, dl1_phe_image_1stPass, calibration_status and leakage_dict.

Yes, this seems to be a problem. @gaia-verna did you find the same inconstistency?

HealthyPear commented 3 years ago

I found the problem with the 1st bug - thanks @adonini !

I will fix this and make a new bugfix release.

@adonini @gaia-verna could you confirm me that you also found this problem and that make_performance.py wasn't crashing for other bug after fixing this? I have problems uploading some models at LCG.GRIF.fr/CEA-USER and I cannot proceed with the DL2 dataset for this version of the analysis (0.2.1dev, but the same problem will present itself for v0.3.0dev...)

adonini commented 3 years ago

Sorry I didn't see the reply. Yes with the two correction, the script works for me.

HealthyPear commented 3 years ago

Sorry I didn't see the reply. Yes with the two correction, the script works for me.

Did you also try the last script with this version (parallel or divergent analyses doesn't matter to me here)?

adonini commented 3 years ago

yes all the analysis were done with the master branch of the latest version. I will analyse the parallel files in the next days, so I can give you a feedback on those too.

HealthyPear commented 3 years ago

the pointing type will not play a role after DL2 (I mean in terms of fatal bugs)

make_performance.py should work in the same way

I just wanted to be sure that after fixing write_dl2 with your suggestions I can be sure that someone tested the DL2 to DL3 step. After this bugfix (and even better after solving #64 ) I think I can do the bugix release