Wrong base nsb level used in tuning at waveform level

[gabemery]

The tuning at waveform level is done by extracting the relative NSB increase required and getting the original simulated NSB from a Header in simtel files.

Original implementation extracted all occurrences of 'NIGHTSKY_BACKGROUND' in yield_all_subobjects(EventIOFile(filename), [History, HistoryConfig]).

When tested on prod5_tran80 files the output was : [0.25307, 0.24586, 0.25307, 0.25307, 0.25307, 0.1, 0.12] where index 1 was the value for LST-1, index 2 for LST-2, ... index 5 for MAGIC 1 and index 6 for MAGIC 2. Index 0 was ignored and assumed to be some unused default value.

It was then correct to use tel_id to access the correct original NSB level.

This is not the case with the current MC production which has an additional value : [0.24586, 0.217, 0.24586, 0.1, 0.12]. Reported by @jstvdk

The preferred way to solve this issue would be to have a reliable way to find the NSB level effectively used for each telescope.

[maxnoe]

This reliable way exists in the most recent version of sim_telarray and I already opened a corresponding issue in the lst1-sim-config repo two years ago: https://github.com/cta-observatory/lst-sim-config/issues/29

With the metadata, you always get the final configuration value of the configuration item for each telescope.

[gabemery]

Thanks @maxnoe . Sadly it seems the LSTProd2 was not done with the latest version. We have no HistoryMeta.

[gabemery]

Looking a bit more, I have found a way to differentiate between the wrong and correct NSB levels in the config history.

The correct telescope wise entries of 'NIGHTSKY_BACKGROUND' are preceded by an entry 'STORE_PHOTOELECTRONS' while the wrong ones are preceded by 'MIN_PHOTOELECTRONS' in both prod 5 and LSTprod2.

I could thus adapt the implementation of extract_simulation_nsb in lstchain to assume this is true until a better solution is found.

[moralejo]

Hi @gabemery, the current implementation does work for LSTProd2, right?

To make it work for Prod5 too, since the goal is to achieve the same level of fluctuations in MC and data, isn't it possible to obtain empirically the level of fluctuations from the MC waveforms?

[gabemery]

@moralejo Current implementation is wrong for LSTProd2. It was implemented in prod5 (and was correct then).

The issue is that the level of fluctuation in MC vs data informs on the multiplicative factor to the simulated NSB required to reach the level of data. So knowing the original pulse rate in MC is mandatory to know the number of pulse that needs to be added.

[maxnoe]

yield_all_subobjects(EventIOFile(filename), [History, HistoryConfig]).

btw. there is no need for you to do this by hand, the SimTelFile gives you nice access to the config lines:

In [9]: f = SimTelFile("/fefs/aswg/data/mc/DL0/LSTProd2/TestDataset/sim_telarray/node_theta_10.0_az_102.199_/output_v1.4/simtel_corsika_theta_10.0_az_102.199_run1.simtel.gz")

In [10]: for timestamp, line in f.history:
    ...:     line = line.decode('utf-8').strip()
    ...:     if 'nightsky' in line.lower():
    ...:         print(line)
    ...: 
NIGHTSKY_BACKGROUND all:0.24586                % NSB of LST-1 prototype is a bit lower than others due to a different QE curve. For reference NSB.
NIGHTSKY_BACKGROUND all:0.217                                   % 217 MHz NSB p.e. rate
NIGHTSKY_BACKGROUND all:0.24586                % NSB of LST-1 prototype is a bit lower than others due to a different QE curve. For reference NSB.
NIGHTSKY_BACKGROUND all:0.1 % Re-evaluated (KB)
NIGHTSKY_BACKGROUND all:0.12 % Re-evaluated (KB)

[moralejo]

By the way, @gabemery, does the method assume that all noise is from NSB? It is certainly dominant, but there is also electronic noise.

[gabemery]

@moralejo Yes it does assume it is all NSB. If there is a non-negligible contribution of the electronic noise to the waveform variance it would thus be biased.