check what happens when the fixture `verysimple_simulation` is not run with 4000 packets

[antreev-brar]

I would like to work on this issue. Can u provide some more details I couldn't find verysimple_simulation in the codebase and would any number of packets(except 4000 ) be sufficient?

[andrewfullard]

simulation_versimple is the actual name of the test fixture located in tardis/conftest.py. Notice that sim.iterate(4000) is used (the 4000 packets). The current test suite should complete the same with an arbitrary number instead of 4000.

[AyushiDaksh]

Result from running on 4000 packets: pytest tardis 270 passed, 1541 skipped, 15 xfailed, 3 xpassed, 33 warnings

pytest tardis --tardis-refdata 1529 passed, 275 skipped, 17 xfailed, 8 xpassed, 657 warnings

Result from running on 1e5 packet: pytest tardis 270 passed, 1541 skipped, 15 xfailed, 3 xpassed, 33 warnings

pytest tardis --tardis-refdata 1529 passed, 275 skipped, 17 xfailed, 8 xpassed, 657 warnings

Result from running on 1e6 packets: pytest tardis 270 passed, 1541 skipped, 15 xfailed, 3 xpassed, 33 warnings

pytest tardis --tardis-refdata 4 failed, 1525 passed, 275 skipped, 17 xfailed, 8 xpassed, 657 warnings <-------

The failed test cases relate to widgets, specifically TestLineInfoWidgetData and TestLineInfoWidgetEvents.

Investigating this further.

@andrewfullard Is the final goal to have a random number generator (given a certain range) which would initialize simulation_verysimple?

[andrewfullard]

No, it is simply to make sure that our choice of packet count is not hiding test failures. It seems that it does once we reach 1e6, so further investigation into the widget tests would be appropriate.

[AyushiDaksh]

@andrewfullard The issue seems to be happening due to a few outliers (which show up when packet count is very high)

In the LineInfoWidget test cases, there are two wavelength ranges defined:

• [2500, 3500], # Wavelength range with plenty of line interactions
• [16200, 16300], # Wavelength range with no line interactions

Both the cases are handled separately in the test cases. On a high level, when the selected wavelength range is [16200, 16300], the code expects that there should be zero line interactions in that wavelength. Usually this is true, but when we're increasing the packet count to a high number, there seem to be few (<=7) packets in that range which are undergoing line interactions (with Si II). Hence the test is failing due to these packets.

Could this indicate a bug in the Tardis simulation itself?

[AyushiDaksh]

More specifically (please read the previous update too), these are the line interactions causing those failures (notice the de-excitation wavelengths):

Last Line Interaction ............................................................. No. of packets exc. 13-21 (5042.43 A) de-exc. 26-20 (16911.43 A)..........................1 exc. 05-13 (3863.69 A) de-exc. 26-20 (16911.43 A)..........................1 exc. 05-25 (2059.67 A) de-exc. 25-20 (16981.82 A)..........................1 exc. 06-26 (2059.30 A) de-exc. 26-20 (16911.43 A)..........................1 exc. 01-06 (1816.93 A) de-exc. 26-20 (16911.43 A)..........................1 exc. 05-77 (1438.58 A) de-exc. 77-46 (15874.83 A)..........................1 exc. 06-88 (1404.48 A) de-exc. 26-20 (16911.43 A)..........................1

[andrewfullard]

It may not be a bug, but rather simply an unexpected line interaction. Thanks for the thorough investigation. We would need to check to see if those are valid de-excitation wavelengths for Si II given our atomic data to determine if it's a bug.

[AyushiDaksh]

@andrewfullard I created a minimal reproducer for it (this is what exactly is being tested in the test case). The problem is that in the output_nu field of the 'MonteCarloRunner', there are some frequencies corresponding to the wavelength range [16200, 16300]

# Imports
from tardis.io.atom_data.base import AtomData
from tardis.io.config_reader import Configuration
from tardis.simulation import Simulation
from astropy import units as u
import os

# Download the atomic data
atom_data = AtomData.from_hdf("kurucz_cd23_chianti_H_He.h5")

# Get config
cfg_file = "tardis_configv1_verysimple.yml"
path = os.path.abspath(os.path.join("../tardis/io/tests/data/", cfg_file))
config = Configuration.from_yaml(path)

# Run simulation
sim = Simulation.from_config(config, atom_data=atom_data)
sim.iterate(1000000)

# Remove dummy interaction ID from output frequencies
id_mask = sim.runner.last_line_interaction_out_id != -1
output_nus = sim.runner.output_nu[id_mask]

# Convert output frequencies to wavelength
output_wv = output_nus.to(u.Angstrom, equivalencies=u.spectral())

# Filter on wavelength [16200, 16300]
wavelength_filter = [16200 * u.AA, 16300 * u.AA]
wavelength_mask = (output_wv>wavelength_filter[0]) & (output_wv<wavelength_filter[1])

# It prints some wavelengths (ideally this shouldn't happen)
print(output_wv[wavelength_mask])

# Causes the test case to fail
assert len(output_wv[wavelength_mask]) == 0

[AyushiDaksh]

@andrewfullard Can you please guide me a bit how can I investigate this further (to check validity of line interaction wavelengths)?

[andrewfullard]

You could output the lines for Si II from the atomic data set to check if they are present in the 16200 - 16300 wavelength range.