since the carbon-wall training set defined for jet_data_0D, jet_data_1D, jet_data (latter two are basically unused) has missing or incomplete 1D density and temperature profile data for many shots, unlike other data choices. This causes a significant fraction of shots to be excluded during the python guaranteed_preprocessed.py step, even when the 1D profiles are not used.
We should pop the 2x profile entries from the all_signals dictionary automatically when params['paths']['shot_files'] = [jet_carbon_wall].
A tricky, undocumented step when switching to training on
jet_data_0Dis that the user must comment-out the line containing'etemp_profile': etemp_profile, 'edens_profile': edens_profile,in the definition of the dictionary that stores "all signals defined on this machine": https://github.com/PPPLDeepLearning/plasma-python/blob/c82ba61e339882a5af10b1052edc0348e16119f4/data/signals.py#L399-L415before re-building the module and preprocessing the raw shot data. This is unique to this selection of the
conf['paths']['data']in: https://github.com/PPPLDeepLearning/plasma-python/blob/c82ba61e339882a5af10b1052edc0348e16119f4/examples/conf.yaml#L13since the carbon-wall training set defined for
jet_data_0D,jet_data_1D,jet_data(latter two are basically unused) has missing or incomplete 1D density and temperature profile data for many shots, unlike other data choices. This causes a significant fraction of shots to be excluded during thepython guaranteed_preprocessed.pystep, even when the 1D profiles are not used.We should pop the 2x profile entries from the
all_signalsdictionary automatically whenparams['paths']['shot_files'] = [jet_carbon_wall].