search

CYGNUS-RD / reconstruction

Camera and scope analysis tools
0 stars 25 forks source link

Update and backcompatibility of pedestal search, filepathfinding and winter23-patch2 integration #231

Closed GioDho closed 11 months ago

GioDho commented 11 months ago

Swiftlib.py modified to look for the files in the correct paths on the cloud (no more cygnus disk). Tested on MC, MAN, LNF and LNGS. Reintroduced the JUPITERHUB_USER as now works on the cloud

Utilities.py modified to use the pandas version of the pedestal search for when the runlog database is available. For those without runlog, reintroduced the older method of manually writing a pedfile list with range of runs related to that pedestal.

Reconstruction.py. Modified to more cleanly search for pedestal. Also fixed an issue for which root files could not be run. This adaptation is taken by winter23-patch2. Environmental variables are nevertheless integrated into new system.

configFiles now use tag also for the pedestal and file pathing finding. This makes the use of many configFiles (one for LNGS, one for LNF, one for MANGO etc) not really useful, but I would keep it for the moment as it may be useful later. The MC tag is now 'MC$pathfromcygno-simtofolderabovehistograms_Run', so that the mess of the simulation "folders" on the cloud can be addressed

GioDho commented 11 months ago

Commit Bug correction for debug mode: reconstruction.py. Modification to the lastevent in the range of the ones to be analysed when in debug mode so that, especially for midas files, the code does not go through all the events to the end

GioDho commented 11 months ago

Commit: Modification to getEvent function exploiting the runlog. Reconstruction.py. Modification to function getNEvents which now exploits the runlog to extract the number of events instead of opening the midas file and read it all (about 35 s improvement per run, especially useful for debug mode). Already tested in case the run number is absent from runlog (it then uses previous method) and in case the run number is null (crash of the daq logger).

GioDho commented 11 months ago

Commit: Repair of githash. Now there is a new option --git where you can pass the gihash. Useful in condor queue where the .git folder should not be passed due to its dimensions. Still you can evaluate the githash and pass as an argument. Added a try except to avoid error when no git repository is found by the code. It is somewhat dangerous as the user could manually input a wrong githash. It has the same level of danger as the user changing the parameters which will not match the commit. Creating at runtime a txt file with the parameters used and the githash and upload on sql table may be better (the security of transmitting to the sql though needs to be taken into consideration)

GioDho commented 11 months ago

Commit: Added Piacentini's fix to solve bug in slow control for restricted amount of runs