Feature adjustments

[SkepticRaven]

Development branch for merging in new features from Nick

List of new features

• Segmentation Base Featuers
• Signal Processing Features
• Model hyperparameters

[SkepticRaven]

Current state of this effort:

Integrated branches/pulls:

• Original feature request repo: https://github.com/KumarLabJax/JABS-behavior-classifier/pull/9
• Original pull for segmentation work: https://github.com/KumarLabJax/JABS-behavior-classifier/pull/12
• Segmentation rendering bugfix: https://github.com/KumarLabJax/JABS-behavior-classifier/pull/15
• Segmentation animal id sorting bug: https://github.com/KumarLabJax/JABS-behavior-classifier/pull/14
• Pixel scaling + cleanup https://github.com/KumarLabJax/JABS-behavior-classifier/pull/16

Segmentation features integration status:

• 192 segmentation based features calculated
  • 32 base + window
• Features handle pixel units
• Segmentation UI components (plotting)
• Segmentation data is not cached (it's big), so it always hits original pose file
  • Features are cached, so classifiers are export-able

Known issues:

• Backwards compatibility of < pose_v6 projects just crash
  • Feature set checks need to be added

Still TODO:

• Integrate location of ID patch: https://github.com/KumarLabJax/JABS-behavior-classifier/pull/18
• FFT Features: https://github.com/KumarLabJax/JABS-behavior-classifier/pull/11
  • Adjust control for FFT feature use or make more efficient (based on profiling) Profiling branch here: https://github.com/KumarLabJax/JABS-behavior-classifier/pull/20

[SkepticRaven]

Backwards compatibility was resolved with recent merge + patch (9a5e173) Now just need to focus on 2 remaining merges...

[SkepticRaven]

Signal processing code has been integrated and converted to be efficient enough for standard usage, but tests are pretty broken due to internal changes (eg NaN propagation, feature structure changes).

18 still needs to be merged

Almost done with this unreasonably massive branch!

[SkepticRaven]

Attempted a merge of #18, and I think we need to leave that to another timepoint. Moment storage location has drastically changed and will need a new method for propagating centroid information.

[dahhei]

Runtime warnings encountered when calculating features (used initialize.py). Seems to only occur with np.min and np.max

/behavior-classifier/src/feature_extraction/window_operations/window_stats.py:144: RuntimeWarning: All-NaN slice encountered
  return np.min(window_values, axis=1, initial=np.nanmax(values), where=window_masks)
/behavior-classifier/src/feature_extraction/window_operations/window_stats.py:156: RuntimeWarning: All-NaN slice encountered
  return np.max(window_values, axis=1, initial=np.nanmin(values), where=window_masks)

[dahhei]

Error while running initialize.py.

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/behavior-classifier/initialize_project.py", line 52, in generate_files_worker
    _ = features.get_window_features(w, use_social,
  File "/behavior-classifier/src/feature_extraction/features.py", line 346, in get_window_features
    features = self.__compute_window_features(window_size)
  File "/behavior-classifier/src/feature_extraction/features.py", line 468, in __compute_window_features
    self._feature_modules[key].window(self._identity, window_size,
  File "/behavior-classifier/src/feature_extraction/feature_group_base_class.py", line 54, in window
    return {
  File "/behavior-classifier/src/feature_extraction/feature_group_base_class.py", line 55, in <dictcomp>
    name: mod.window(identity, window_size, per_frame_values[name]) for name, mod in
  File "/behavior-classifier/src/feature_extraction/feature_base_class.py", line 139, in window
    signal_features = self.window_signal(identity, window_size, per_frame_values)
  File "/behavior-classifier/src/feature_extraction/feature_base_class.py", line 162, in window_signal
    freqs, ts, Zxx = signal.stft(np.nan_to_num(per_frame, nan=0), fs=self._fps, nperseg=window_size * 2 + 1, noverlap=window_size * 2, window='hann', scaling='psd', detrend='linear')
  File "/usr/local/lib/python3.10/site-packages/scipy/signal/_spectral_py.py", line 1211, in stft
    freqs, time, Zxx = _spectral_helper(x, x, fs, window, nperseg, noverlap,
  File "/usr/local/lib/python3.10/site-packages/scipy/signal/_spectral_py.py", line 1812, in _spectral_helper
    raise ValueError('noverlap must be less than nperseg.')
ValueError: noverlap must be less than nperseg.

[SkepticRaven]

I was able to reproduce the second one, where fft features would crash when windows were larger than the entire file. I was not able to reproduce the warnings on the first, but added some checks that would hopefully silence them. They're only warnings and behavior is to just pass forward a NaN (which is what we want), so can ignore it.