maindlt
commented
3 months ago Hi @sthater,
as far as I see from the code, hist_width, hist_bins, and hist_center are only calculated when they are set to 'default' in the config file. If set to other values, DYNAMITE will use these user defined values. The probably easiest way to deal with obvious outliers would therefore be to adjust their respective settings in the config file.
If we need to automatically ignore outliers in calculating hist_width and hist_bins (with hist_center most likely zero), a few questions pop up which we should discuss in one of the next hack sessions:
- How do we detect outliers (based on an $n\sigma$ rule, some other criterion...)?
- Do we always want to eliminate outliers or do we need a config setting for switching it on or off?
- A little provocative: as this might get a bit intricate, do we identify this as a DYNAMITE task or as a data preparation task?
- Based on the answers to the questions above, is it worthwhile to implement this or does it suffice to point to the user-defined settings of
hist_width,hist_bins, andhist_centerin the config file?
Looking forward to discussing and cheers, Thomas
I noticed that hist_width and hist_bins are calculated based on the max V and sigma values in the kinematics. However, sometimes, there are very high outliers in the kinematics (e.g. stars).
These are ignored by DYNAMITE usually, when putting high uncertainties. However, they might drive hist_width and hist_bins to extremely larger values. Usually i use for example 200-500 for hist_bins and because of stars in the kinematics, hist_bins was suddenly 1200.
This can of course easily cause memory issues. I would suggest to calculate hist_width and hist_bins not based on the max values but also check if these values are outliers and then use lower values.
This could also be changed in the automatic plotting routine for kinematic maps, where we have similar issues.