ultralytics / hub

Ultralytics HUB tutorials and support
https://hub.ultralytics.com
GNU Affero General Public License v3.0
138 stars 14 forks source link

count on custom model #818

Open rppycv opened 2 months ago

rppycv commented 2 months ago

Search before asking

Question

I have a model trained with custom dataset, named "best.pt" (train with roboflow), the counter from ultralytics work fine!. View https://docs.ultralytics.com/guides/object-counting/#real-world-applications For example, in this line of code, I can print and retrieve the class parameters

counter = solutions.ObjectCounter(view_img=True, reg_pts=region_points, names=model.names, draw_tracks=True, line_thickness=2)

print(model.names) {0: 'Higados', 1: 'vesicula'}

But I need to get from the code the in and out counters for each of the two classes separately, How can I do it?

Additional

The code is as follows

import cv2 from ultralytics import YOLO, solutions

def count_objects_in_region(video_path, output_video_path, model_path): """Count objects in a specific region within a video.""" model = YOLO(model_path) cap = cv2.VideoCapture(video_path) assert cap.isOpened(), "Error reading video file" w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS)) region_points = [(430, 640), (780, 550), (740, 450), (390, 530)] fourcc = int(cap.get(cv2.CAP_PROP_FOURCC)) ### cv2.VideoWriter_fourcc(*"mp4v") BUG REPORTED video_writer = cv2.VideoWriter(output_video_path, fourcc, fps, (w, h)) counter = solutions.ObjectCounter( view_img=True, reg_pts=region_points, names=model.names, draw_tracks=True, line_thickness=2 )

while cap.isOpened():
    success, im0 = cap.read()
    if not success:
        print("Video frame is empty or video processing has been successfully completed.")
        break
    tracks = model.track(im0, persist=True, show=False)
    im0 = counter.start_counting(im0, tracks)
    video_writer.write(im0)

cap.release()
video_writer.release()
cv2.destroyAllWindows()

count_objects_in_region("video_INPUT.mp4", "video_OUTPUT.mp4", "best.pt")

glenn-jocher commented 2 months ago

@rppycv hello,

Thank you for sharing your question and the detailed code snippet! It's great to see you leveraging the Ultralytics solutions for object counting with your custom model. To retrieve the in and out counters for each class separately, you can modify the ObjectCounter class to access the counts directly.

Here's an example of how you can achieve this:

  1. Modify the ObjectCounter class to store counts for each class.
  2. Access the counts after processing each frame.

Below is an updated version of your code with these modifications:

import cv2
from ultralytics import YOLO, solutions

class CustomObjectCounter(solutions.ObjectCounter):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.in_count = {name: 0 for name in self.names.values()}
        self.out_count = {name: 0 for name in self.names.values()}

    def start_counting(self, im0, tracks):
        im0 = super().start_counting(im0, tracks)
        for track in tracks:
            if track['in']:
                self.in_count[self.names[track['cls']]] += 1
            if track['out']:
                self.out_count[self.names[track['cls']]] += 1
        return im0

def count_objects_in_region(video_path, output_video_path, model_path):
    """Count objects in a specific region within a video."""
    model = YOLO(model_path)
    cap = cv2.VideoCapture(video_path)
    assert cap.isOpened(), "Error reading video file"
    w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))
    region_points = [(430, 640), (780, 550), (740, 450), (390, 530)]
    fourcc = int(cap.get(cv2.CAP_PROP_FOURCC))  ### cv2.VideoWriter_fourcc(*"mp4v") BUG REPORTED
    video_writer = cv2.VideoWriter(output_video_path, fourcc, fps, (w, h))
    counter = CustomObjectCounter(
        view_img=True, reg_pts=region_points, names=model.names, draw_tracks=True, line_thickness=2
    )

    while cap.isOpened():
        success, im0 = cap.read()
        if not success:
            print("Video frame is empty or video processing has been successfully completed.")
            break
        tracks = model.track(im0, persist=True, show=False)
        im0 = counter.start_counting(im0, tracks)
        video_writer.write(im0)

    cap.release()
    video_writer.release()
    cv2.destroyAllWindows()

    # Print the counts for each class
    print("In counts:", counter.in_count)
    print("Out counts:", counter.out_count)

count_objects_in_region("video_INPUT.mp4", "video_OUTPUT.mp4", "best.pt")

In this example, the CustomObjectCounter class extends the ObjectCounter class to maintain separate in and out counts for each class. After processing the video, the counts are printed.

Feel free to adjust the code as needed for your specific use case. If you encounter any issues, please ensure you are using the latest versions of the Ultralytics packages.

Happy coding! šŸ˜Š

rppycv commented 2 months ago

@glenn-jocher Hello, thanks for you response, but

When I run the code, I get this error on line 16, it seems it doesn't expect a string

Polygon Counter Initiated.

0: 480x800 2 Higadoss, 2 vesiculas, 508.7ms Speed: 5.0ms preprocess, 508.7ms inference, 2.0ms postprocess per image at shape (1, 3, 480, 800) Traceback (most recent call last): File "C:\Users\root\PycharmProjects\0Nuevos\countClass\cc-API_counterxClass.py", line 55, in count_objects_in_region(folder + video_101, folder + "BORRAR{}Camara101.mp4".format(fechaFormat), "best.pt") File "C:\Users\root\PycharmProjects\0Nuevos\countClass\cc-API_counterxClass.py", line 41, in count_objects_in_region im0 = counter.start_counting(im0, tracks) File "C:\Users\root\PycharmProjects\0Nuevos\countClass\cc-API_counterxClass.py", line 16, in start_counting if track['in']: File "C:\Users\root\PycharmProjects\0Nuevos\countClass.venv\lib\site-packages\ultralytics\engine\results.py", line 287, in getitem return self._apply("getitem", idx) File "C:\Users\root\PycharmProjects\0Nuevos\countClass.venv\lib\site-packages\ultralytics\engine\results.py", line 359, in _apply setattr(r, k, getattr(v, fn)(*args, **kwargs)) File "C:\Users\root\PycharmProjects\0Nuevos\countClass.venv\lib\site-packages\ultralytics\engine\results.py", line 183, in getitem return self.class(self.data[idx], self.orig_shape) TypeError: new(): invalid data type 'str'

Process finished with exit code 1

The full code is:

import cv2 from ultralytics import YOLO, solutions # error fbgemm.dll -> #torch 2.3.0 y torchvision 0.18.0 import datetime # Se crea un objeto de tipo tiempo fechaActual = datetime.datetime.now() fechaFormat = fechaActual.strftime("%Y%m%d__%Hh%Mm")

class CustomObjectCounter(solutions.ObjectCounter): def init(self, *args, *kwargs): super().init(args, **kwargs) self.in_count = {name: 0 for name in self.names.values()} self.out_count = {name: 0 for name in self.names.values()}

def start_counting(self, im0, tracks):
    im0 = super().start_counting(im0, tracks)
    for track in tracks:
        if track['in']:
            self.in_count[self.names[track['cls']]] += 1
        if track['out']:
            self.out_count[self.names[track['cls']]] += 1
    return im0

def count_objects_in_region(video_path, output_video_path, model_path): """Count objects in a specific region within a video.""" model = YOLO(model_path) cap = cv2.VideoCapture(video_path) assert cap.isOpened(), "Error reading video file" w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS)) region_points = [(430, 640), (780, 550), (740, 450), (390, 530)] fourcc = int(cap.get(cv2.CAP_PROP_FOURCC)) ### cv2.VideoWriter_fourcc(*"mp4v") BUG REPORTED video_writer = cv2.VideoWriter(output_video_path, fourcc, fps, (w, h)) counter = CustomObjectCounter( view_img=True, reg_pts=region_points, names=model.names, draw_tracks=True, line_thickness=2 )

while cap.isOpened():
    success, im0 = cap.read()
    if not success:
        print("Video frame is empty or video processing has been successfully completed.")
        break
    tracks = model.track(im0, persist=True, show=False)
    im0 = counter.start_counting(im0, tracks)
    video_writer.write(im0)

cap.release()
video_writer.release()
cv2.destroyAllWindows()

# Print the counts for each class
print("In counts:", counter.in_count)
print("Out counts:", counter.out_count)

folder = "C:/Users/root/Videos/AASA/" video_101 = "Camara_101_HD__20240812_15h.mp4"

count_objects_in_region(folder + video_101, folder + "BORRAR{}Camara101.mp4".format(fechaFormat), "best.pt")

glenn-jocher commented 2 months ago

Hello @rppycv,

Thank you for providing the detailed error message and the full code. It looks like the issue arises from the way the tracks data is being accessed. The track objects might not have the in and out attributes as expected, leading to the TypeError.

To address this, let's ensure that the tracks data is structured correctly and contains the necessary attributes. Here's an updated version of your code with additional checks to handle the track data more robustly:

import cv2
from ultralytics import YOLO, solutions
import datetime

# Create a datetime object for formatting
fechaActual = datetime.datetime.now()
fechaFormat = fechaActual.strftime("%Y%m%d__%Hh%Mm")

class CustomObjectCounter(solutions.ObjectCounter):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.in_count = {name: 0 for name in self.names.values()}
        self.out_count = {name: 0 for name in self.names.values()}

    def start_counting(self, im0, tracks):
        im0 = super().start_counting(im0, tracks)
        for track in tracks:
            if 'in' in track and track['in']:
                self.in_count[self.names[track['cls']]] += 1
            if 'out' in track and track['out']:
                self.out_count[self.names[track['cls']]] += 1
        return im0

def count_objects_in_region(video_path, output_video_path, model_path):
    """Count objects in a specific region within a video."""
    model = YOLO(model_path)
    cap = cv2.VideoCapture(video_path)
    assert cap.isOpened(), "Error reading video file"
    w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))
    region_points = [(430, 640), (780, 550), (740, 450), (390, 530)]
    fourcc = int(cap.get(cv2.CAP_PROP_FOURCC))  ### cv2.VideoWriter_fourcc(*"mp4v") BUG REPORTED
    video_writer = cv2.VideoWriter(output_video_path, fourcc, fps, (w, h))
    counter = CustomObjectCounter(
        view_img=True, reg_pts=region_points, names=model.names, draw_tracks=True, line_thickness=2
    )

    while cap.isOpened():
        success, im0 = cap.read()
        if not success:
            print("Video frame is empty or video processing has been successfully completed.")
            break
        tracks = model.track(im0, persist=True, show=False)
        im0 = counter.start_counting(im0, tracks)
        video_writer.write(im0)

    cap.release()
    video_writer.release()
    cv2.destroyAllWindows()

    # Print the counts for each class
    print("In counts:", counter.in_count)
    print("Out counts:", counter.out_count)

folder = "C:/Users/root/Videos/AASA/"
video_101 = "Camara_101_HD__20240812_15h.mp4"

count_objects_in_region(folder + video_101, folder + "BORRAR___{}___Camara101.mp4".format(fechaFormat), "best.pt")

In this updated code, I've added checks to ensure that the track dictionary contains the in and out keys before attempting to access them. This should help prevent the TypeError you encountered.

If the issue persists, please verify that you are using the latest versions of the Ultralytics packages. You can update them using pip:

pip install --upgrade ultralytics

Feel free to reach out if you have any further questions or run into additional issues. Happy coding! šŸ˜Š

rppycv commented 2 months ago

@glenn-jocher very thanks, an issue, my Pycharm IDE "I suggest the following"

suggested_Pycharm however I did not move this code

I just moved the print under the while loop

while cap.isOpened(): .......

Print the counts for each class

print("In counts:", counter.in_count) print("Out counts:", counter.out_count)

and I only get this result In counts: {'Higados': 0, 'vesicula': 0} Out counts: {'Higados': 0, 'vesicula': 0}

ADD logs LOGcc-v3.py_2024-09-02.txt

Thanks in advance!!!

Full Code:

import cv2 from ultralytics import YOLO, solutions

error fbgemm.dll ->solution: #torch v2.3.0 y torchvision v0.18.0

import datetime # Se crea un objeto de tipo tiempo fechaActual = datetime.datetime.now() fechaFormat = fechaActual.strftime("%Y%m%d__%Hh%Mm")

class CustomObjectCounter(solutions.ObjectCounter): def init(self, *args, *kwargs): super().init(args, **kwargs) self.in_count = {name: 0 for name in self.names.values()} self.out_count = {name: 0 for name in self.names.values()}

def start_counting(self, im0, tracks):
    im0 = super().start_counting(im0, tracks)
    for track in tracks:
        if 'in' in track and track['in']:
            self.in_count[self.names[track['cls']]] += 1
        if 'out' in track and track['out']:
            self.out_count[self.names[track['cls']]] += 1
        return im0

def count_objects_in_region(video_path, output_video_path, model_path): """Count objects in a specific region within a video.""" model = YOLO(model_path) cap = cv2.VideoCapture(video_path) assert cap.isOpened(), "Error reading video file" w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS)) region_points = [(430, 640), (780, 550), (740, 450), (390, 530)] fourcc = int(cap.get(cv2.CAP_PROP_FOURCC)) ### cv2.VideoWriter_fourcc(*"mp4v") BUG REPORTED video_writer = cv2.VideoWriter(output_video_path, fourcc, fps, (w, h)) counter = CustomObjectCounter( view_img=True, reg_pts=region_points, names=model.names, draw_tracks=True, line_thickness=2 )

while cap.isOpened():
    success, im0 = cap.read()
    if not success:
        print("Video frame is empty or video processing has been successfully completed.")
        break
    tracks = model.track(im0, persist=True, show=False)
    im0 = counter.start_counting(im0, tracks)
    video_writer.write(im0)

    # Print the counts for each class
    print("In counts:", counter.in_count)
    print("Out counts:", counter.out_count)

cap.release()
video_writer.release()
cv2.destroyAllWindows()

folder = "C:/Users/root/Videos/AASA/" video_101 = "Camara_101_HD__20240812_15h.mp4"

count_objects_in_region(folder + video_101, folder + "OUTPUTvideo{}Camara101.mp4".format(fechaFormat), "best.pt")

rppycv commented 2 months ago

@glenn-jocher Hello, can we illustrate the question with something simpler, please?

1.- I have an initial video, which I need to tell about, particular classes on Livers (cattle slaughter) 2.- attached video

https://github.com/user-attachments/assets/726b496f-40a8-4095-95ba-cd2fe6e5e3c2

3.- On this video I run the code to count objects, and I get the resulting video plus the logs, which I attach.

https://github.com/user-attachments/assets/0d2a34ec-22eb-4d26-80e8-8d276958e017

LOG_CountClassLine.py_2024-09-02.txt

4.- Finally, what I need is to be able to extract from the code the online counters, shown in the resulting video, attached photo.

Photo-2024-09-02-15h26m42s723

5.- And attach code:

--- IMPORTAMOS LIBRERIAS ---

import cv2 from ultralytics import YOLO, solutions # error fbgemm.dll use #torch 2.3.0 y torchvision 0.18.0 import datetime

Se crea un objeto de tipo tiempo

fechaActual = datetime.datetime.now() fechaFormat = fechaActual.strftime("%Y-%m-%d_%Hh%Mm%Ss")

def count_specific_classes(video_path, output_video_path, model_path): """Count both classes of objects in a video.""" model = YOLO(model_path) cap = cv2.VideoCapture(video_path) assert cap.isOpened(), "Error reading video file" w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS)) line_points = [(430, 640), (780, 550)] fourcc = int(cap.get(cv2.CAP_PROP_FOURCC)) # cv2.VideoWriter_fourcc(*"mp4v") BUG REPORTADO video_writer = cv2.VideoWriter(output_video_path, fourcc, fps, (w, h)) counter = solutions.ObjectCounter(view_img=True, reg_pts=line_points, names=model.names, draw_tracks=True, line_thickness=2)

while cap.isOpened():
    success, im0 = cap.read()
    if not success:
        print("Video frame is empty or video processing has been successfully completed.")
        break
    # Track and count objects for both classes (0: Higados and 1: Vesicula)
    tracks = model.track(im0, persist=True, show=False, classes=[0, 1])
    im0 = counter.start_counting(im0, tracks)
    video_writer.write(im0)

cap.release()
video_writer.release()
cv2.destroyAllWindows()

count_specific_classes("Initial_Video_Camara_101_HD__20240812.mp4", "{}___Camara101_CountLine.mp4".format(fechaFormat), "best.pt")

glenn-jocher commented 2 months ago

Hello @rppycv,

Thank you for providing the detailed context and the attachments. Let's break down the task to make it simpler and ensure we can extract the online counters for the specific classes you are interested in.

Objective

You want to count specific classes (Livers and Vesicula) in a video and extract these counts from the code.

Simplified Approach

  1. Initialize the model and video capture.
  2. Define the region points for counting.
  3. Track and count objects for the specified classes.
  4. Extract and print the counts.

Here's a simplified version of your code to achieve this:

import cv2
from ultralytics import YOLO, solutions
import datetime

# Create a datetime object for formatting
fechaActual = datetime.datetime.now()
fechaFormat = fechaActual.strftime("%Y-%m-%d_%Hh%Mm%Ss")

class CustomObjectCounter(solutions.ObjectCounter):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.in_count = {name: 0 for name in self.names.values()}
        self.out_count = {name: 0 for name in self.names.values()}

    def start_counting(self, im0, tracks):
        im0 = super().start_counting(im0, tracks)
        for track in tracks:
            if 'in' in track and track['in']:
                self.in_count[self.names[track['cls']]] += 1
            if 'out' in track and track['out']:
                self.out_count[self.names[track['cls']]] += 1
        return im0

def count_specific_classes(video_path, output_video_path, model_path):
    """Count both classes of objects in a video."""
    model = YOLO(model_path)
    cap = cv2.VideoCapture(video_path)
    assert cap.isOpened(), "Error reading video file"
    w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))
    line_points = [(430, 640), (780, 550)]
    fourcc = int(cap.get(cv2.CAP_PROP_FOURCC))  # cv2.VideoWriter_fourcc(*"mp4v") BUG REPORTED
    video_writer = cv2.VideoWriter(output_video_path, fourcc, fps, (w, h))
    counter = CustomObjectCounter(view_img=True, reg_pts=line_points, names=model.names, draw_tracks=True, line_thickness=2)

    while cap.isOpened():
        success, im0 = cap.read()
        if not success:
            print("Video frame is empty or video processing has been successfully completed.")
            break
        # Track and count objects for both classes (0: Higados and 1: Vesicula)
        tracks = model.track(im0, persist=True, show=False, classes=[0, 1])
        im0 = counter.start_counting(im0, tracks)
        video_writer.write(im0)

        # Print the counts for each class
        print("In counts:", counter.in_count)
        print("Out counts:", counter.out_count)

    cap.release()
    video_writer.release()
    cv2.destroyAllWindows()

count_specific_classes("Initial_Video_Camara_101_HD__20240812.mp4", "{}___Camara101_CountLine.mp4".format(fechaFormat), "best.pt")

Key Points:

Next Steps:

  1. Run the Code: Execute the provided code to ensure it works with your video and model.
  2. Verify Counts: Check the printed counts to confirm they match the objects in the video.

If you encounter any issues, please ensure you are using the latest versions of the Ultralytics packages. You can update them using pip:

pip install --upgrade ultralytics

Feel free to reach out if you have any further questions or run into additional issues. Happy coding! šŸ˜Š