coderabbitai[bot]
commented
1 month ago Walkthrough
The changes in this pull request primarily affect the sleap/nn/tracking.py file, focusing on the Tracker and FlowMaxTracksCandidateMaker classes. A new property max_tracking has been added to the Tracker class, facilitating maximum tracking functionality. The target_instance_count attribute is marked for deprecation, with adjustments made to various methods to accommodate the new max_tracks attribute. Logic for limiting track processing based on max_tracks has been implemented, enhancing the overall tracking capabilities and preparing the codebase for future updates.
Changes
| File Path | Change Summary |
|---|---|
| sleap/nn/tracking.py | - Added max_tracking property to Tracker class.- Deprecated target_instance_count attribute with adjusted usage.- Updated FlowMaxTracksCandidateMaker to limit tracks based on max_tracks.- Enhanced track method in Tracker to handle max_tracking.- Modified final_pass method for flexible connection logic.- Updated make_tracker_by_name method to reflect changes in parameters. |
Poem
🐇 In the meadow where tracks align,
A new path opens, oh so fine!
Withmax_trackingguiding the way,
We hop and skip, come what may.
Old counts fade, new limits rise,
In our world, the future lies! 🌼
codecov[bot]
Description
Replicating the Error
target_instance_countnot present in GUI but is present as argument inmake_tracker_by_namehttps://github.com/talmolab/sleap/blob/1339f0d2c71c1e600f071321034582ac5f89019d/sleap/nn/tracking.py#L836-L872Same error when
max_trackingis setTracing the Error
Tracker.make_tracker_by_namedoes taketarget_instance_countas an argument and it is set to0by default.kf_init_frame_countis set to10whiletarget_instance_countremains0, producing the error "Kalman filter requires target instance count." https://github.com/talmolab/sleap/blob/1339f0d2c71c1e600f071321034582ac5f89019d/sleap/nn/tracking.py#L957-L958target_instance_countandkf_init_frame_counthttps://github.com/talmolab/sleap/blob/1339f0d2c71c1e600f071321034582ac5f89019d/sleap/nn/tracking.py#L947-L954Called here (args parsed from CLI): https://github.com/talmolab/sleap/blob/1339f0d2c71c1e600f071321034582ac5f89019d/sleap/nn/tracking.py#L1564-L1570
presence of
max_trackingdoes change the tracker candidate. Is this a problem?? Will CLI behavior be different than GUI behavior?Related: If we want to cull, we need to add
pre_cull_to_targetoption to the GUI. Right now, nothing will lead to the pre-cull function through the GUI https://github.com/talmolab/sleap/blob/1339f0d2c71c1e600f071321034582ac5f89019d/sleap/nn/tracking.py#L922-L930Getting the following error for
flowmaxtracksonly (notsimplemaxtracks).This is not being caught in our tests.
More tests need to be added since we didn't catch the args for using the Kalman filter not to be present in the GUI.
Codecov results from first push
Started inference at: 2024-10-07 20:55:35.535014 Args: { 'data_path': 'C:/repos/sleap/tests/data/tracks/clip.2node.slp', 'models': [ 'C:/repos/sleap/tests/data/tracks\models\241007_145636.centroid.n=1500\training_config.json', 'C:/repos/sleap/tests/data/tracks\models\241007_154651.centered_instance.n=1500\training_config.json' ], 'frames': '627,-682', 'only_labeled_frames': False, 'only_suggested_frames': False, 'output': 'C:/repos/sleap/tests/data/tracks\predictions\clip.2node.slp.241007_205530.predictions.slp', 'no_empty_frames': True, 'verbosity': 'json', 'video.dataset': None, 'video.input_format': 'channels_last', 'video.index': '0', 'cpu': False, 'first_gpu': False, 'last_gpu': False, 'gpu': 'auto', 'max_edge_length_ratio': 0.25, 'dist_penalty_weight': 1.0, 'batch_size': 4, 'open_in_gui': False, 'peak_threshold': 0.2, 'max_instances': None, 'tracking.tracker': 'flowmaxtracks', 'tracking.max_tracking': True, 'tracking.max_tracks': 2, 'tracking.target_instance_count': None, 'tracking.pre_cull_to_target': None, 'tracking.pre_cull_iou_threshold': None, 2024-10-07 20:55:37.291379: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX AVX2 To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 'tracking.post_connect_single_breaks': 1, 'tracking.clean_instance_count': None, 'tracking.clean_iou_threshold': None, 'tracking.similarity': 'instance', 'tracking.match': 'hungarian', 'tracking.robust': None, 'tracking.track_window': 5, 'tracking.min_new_track_points': None, 'tracking.min_match_points': None, 2024-10-07 20:55:37.799106: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 5957 MB memory: -> device: 0, name: NVIDIA GeForce RTX 2070 SUPER, pci bus id: 0000:01:00.0, compute capability: 7.5 'tracking.img_scale': None, 'tracking.of_window_size': None, 'tracking.of_max_levels': None, 'tracking.save_shifted_instances': None, 'tracking.kf_node_indices': [0, 1], 'tracking.kf_init_frame_count': 10, 'tracking.oks_errors': None, 'tracking.oks_score_weighting': 0, 'tracking.oks_normalization': None }
INFO:sleap.nn.inference:Auto-selected GPU 0 with 6744 MiB of free memory. 2024-10-07 20:55:43.684523: I tensorflow/stream_executor/cuda/cuda_dnn.cc:366] Loaded cuDNN version 8201 Versions: Exception in thread Thread-3: Traceback (most recent call last): File "C:\Users\eb\Miniforge3\envs\sleap_dev\lib\threading.py", line 926, in _bootstrap_inner self.run() File "C:\Users\eb\Miniforge3\envs\sleap_dev\lib\threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "c:\repos\sleap\sleap\nn\inference.py", line 2627, in _object_builder t=frame_ind, File "c:\repos\sleap\sleap\nn\tracking.py", line 1433, in track tracked_instances = self.init_tracker.track(untracked_instances, img, t) File "c:\repos\sleap\sleap\nn\tracking.py", line 714, in track img=img, File "c:\repos\sleap\sleap\nn\tracking.py", line 434, in get_candidates ref_instances, ref_img, ref_t, img, t File "c:\repos\sleap\sleap\nn\tracking.py", line 191, in get_shifted_instances max_levels=self.of_max_levels, File "c:\repos\sleap\sleap\nn\tracking.py", line 330, in flow_shift_instances 0.01, cv2.error: OpenCV(4.6.0) :-1: error: (-5:Bad argument) in function 'calcOpticalFlowPyrLK'
Types of changes
Does this address any currently open issues?
https://github.com/talmolab/sleap/pull/1447
1583, #1980
Outside contributors checklist
Thank you for contributing to SLEAP!
:heart:
Summary by CodeRabbit
New Features
max_trackingfeature for enhanced control over the number of active tracks.Bug Fixes
target_instance_countusage to ensure compatibility with new tracking attributes.Documentation