Open ddrisco11 opened 4 days ago
Hi @ddrisco11!
This could be caused by several things, could you upload a test video and the model weights to google drive and share the link so we can reproduce this? My initial thought is that the creatures are being detected inconsistently, and the object tracker is struggling to refind the tracks if they have been lost for several frames, but there is no way to verify without using your model on your test video.
Thanks
Hi @rolson24 , thanks for the response! Here is a link to a short training video, the full code, and the model weights I am using. Please let me know if you need anything else. https://drive.google.com/drive/folders/11u0m7Koew1D7lPEZngvfSR762Rvz0BNr?usp=drive_link
Thanks for the code, model weights, and video. I have done a few tests and it looks like there is a few things going on.
First off, it looks like there is a small bug in supervision that makes the tracker_id's skip several numbers. This can be solved by using minimum_consecutive_frames=2
for now. This may be part of your confusion.
The other part is that the tracker relies on high confidence detections to determine if it should create a new track. Most of the detections from your model have confidence values of less than 0.3. Generally a good performing model will have confidence values of around 0.8. The confidence values of the detections greatly affect how the tracker performs because the tracker uses it as a metric of how likely the detection will be detected again in the next frame, and thus if it should be an object to track. To increase the confidence values of the detections you will need more training data. I would recommend adding image augmentations to your existing training data and considering using a more powerful foundation model like DETIC with Roboflow autodistill to automatically label images and then train your smaller yolov8 model on those labeled images.
The final thing you can try is to reduce the minimum_matching_threshold
. This parameter determines the minimum threshold of an existing track being matched to a new detection. It essentially combines both the confidence of the detection and how much the track and the detection overlap into one number. By reducing the threshold, you allow the tracker to track objects with lower confidence, but also track detections that happen to overlap with an existing track that corresponds to a different object. By reducing the minimum_matching_threshold
you risk tracks switching between different objects, but you may be able to track lower confidence detections. This is unlikely though, and I would first recommend improving the performance of your object detector.
@rolson24 thanks for taking the time to help out! This was my first time asking a question on GitHub and I very much appreciate the support.
Search before asking
Question
Hello! I'm currently building a program to detect deep sea creatures in submarine video. I am using YOLOv8 to make detections and ByteTrack to assign object IDs to these detections. My output includes both an annotated video (based exclusively on YOLO output) and a csv file of all distinct detections (determined as distinct by ByteTrack). I am having an issue where certain creatures are annotated in the video output, ie. detected by YOLO, but then they are omitted from the csv output ie. not assigned a tracking ID by ByteTrack. Please help! Thanks!
Additional
def process_video(video_path: str, output_path: str, model_path: str, location_path: str, start_time: str, time_col: int, lat_col: int, lon_col: int, depth_col: int, salinity_col: int, oxygen_col: int, altitude_col: int, confidence_threshold: float, iou_threshold: float, track_activation_threshold: float, minimum_matching_threshold: float, lost_track_buffer: int, frame_rate: int, min_box_area: int, aspect_ratio_thresh: float): """Process the video to track objects and save tracking data.""" model = YOLO(model_path) tracker = ByteTrack( track_activation_threshold=track_activation_threshold, minimum_matching_threshold=minimum_matching_threshold, lost_track_buffer=lost_track_buffer ) location_data = get_location_data(location_path, time_col, lat_col, lon_col, depth_col, salinity_col, oxygen_col, altitude_col) start_time_seconds = time_to_seconds(start_time)