Detection with Basler Camera

[SJavad]

Search before asking

• [X] I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

Hi ✌ I am working on Face Mask Detection with yolov5 and I want to use basler camera to detect objects. how can I do that and get frames from the Basler camera and make detection? 🤔 I have written some code that gets the frames from the camera:

from pypylon import pylon
import cv2,time

# conecting to the first available camera
camera = pylon.InstantCamera(pylon.TlFactory.GetInstance().CreateFirstDevice())

# Grabing Continusely (video) with minimal delay
camera.StartGrabbing(pylon.GrabStrategy_LatestImageOnly) 
converter = pylon.ImageFormatConverter()

# converting to opencv bgr format
converter.OutputPixelFormat = pylon.PixelType_BGR8packed
converter.OutputBitAlignment = pylon.OutputBitAlignment_MsbAligned

# used to record the time when we processed last frame
prev_frame_time = 0
# used to record the time at which we processed current frame
new_frame_time = 0

index = 0
while camera.IsGrabbing():
    grabResult = camera.RetrieveResult(5000, pylon.TimeoutHandling_ThrowException)

    if grabResult.GrabSucceeded():
        # Access the image data
        image1 = converter.Convert(grabResult)
        image = image1.GetArray()
    grabResult.Release()
    image = cv2.resize(image , (1900,1080))
    cv2.imshow("Image", image)
    if cv2.waitKey(20) == ord("q"):
        break

# Releasing the resource    
camera.StopGrabbing()

cv2.destroyAllWindows()

I think I have to change some lines of code where that loaded image from camera. thanks for your help ❤✌

Additional

No response

[glenn-jocher]

@SJavad hello 👋,

It's great to hear that you're interested in using YOLOv5 for face mask detection with a Basler camera!

Regarding integrating the Basler camera, it looks like you are using pypylon to retrieve the frames. Once you have the frames, you can feed them into YOLOv5 as input.

In your code snippet, you can try replacing cv2.imshow("Image", image) with the line result = detect(image) which will perform object detection using the default yolov5s model and return a list of detected objects in result. You can then process the results as per your use case.

Please let me know if you have any further questions or concerns. We are happy to help!

Best, Glenn Jocher

[SJavad]

@SJavad hello 👋,

It's great to hear that you're interested in using YOLOv5 for face mask detection with a Basler camera!

Regarding integrating the Basler camera, it looks like you are using pypylon to retrieve the frames. Once you have the frames, you can feed them into YOLOv5 as input.

In your code snippet, you can try replacing cv2.imshow("Image", image) with the line result = detect(image) which will perform object detection using the default yolov5s model and return a list of detected objects in the result. You can then process the results as per your use case.

Please let me know if you have any further questions or concerns. We are happy to help!

Best, Glenn Jocher

@glenn-jocher Thank You for your response as always ❤🖐 I Have Tried this Script to stream video frames from Basler Camera to localhost and when I want to run python detect.py I pass the --source switch to http//:localhost:5000/video_feed script:

from flask import Flask, Response
import cv2
import pypylon.pylon as py
import numpy as np
from gevent.pywsgi import WSGIServer

app = Flask(__name__)

icam = py.InstantCamera(py.TlFactory.GetInstance().CreateFirstDevice())
icam.Open()
icam.PixelFormat = "Mono12"

@app.route("/")
def index():
    return "Default Message"
def gen():
    while True:
        image = icam.GrabOne(4000)
        image = image.Array
        ret, jpeg = cv2.imencode(".jpg",image)
        frame = jpeg.tobytes()
        yield (b"--frame\r\n"
               b"Content-Type:image/jpeg\r\n"
               b"Content-Length: "+f"{len(frame)}".encode() + b"\r\n"
               b"\r\n" + frame + b"\r\n")
@app.route('/video_feed')
def video_feed():
    return Response(gen(),mimetype="multipart/x-mixed-replace; boundary=frame")

if __name__ == "__main__":
    http_server = WSGIServer(("",5000), app)
    http_server.serve_forever()        

but It has a very low frame rate. in Pylon Software I get 30 to 40 FPS from the camera but with this script, FPS drops to 15 or lower... I want to change some lines of the detect.py code that can get frames from the camera.

about your solution, I don't have a detect function to pass the image to it and run the inference. also, I import the detect.py file...

how can I access the detect function and pass the data.yaml , weights file, and determine using GPU (--device 0)?

[glenn-jocher]

@SJavad, based on your implementation, it seems that you are using a Flask server to serve the video frames from the Basler camera. This could possibly lead to a low frame rate compared to accessing the camera directly.

Regarding using the detect.py script, you can import the necessary libraries and functions from the script and use the detect() function to obtain the detected objects from the frames. You can modify your script to include the detection function as shown below:

import cv2
from yolov5.detect import detect

# ... camera setup here ...

while True:
    # ... get image from camera ...
    # convert image from camera to BGR format
    image = cv2.cvtColor(image, cv2.COLOR_GRAY2BGR)

    # run detection on the image
    results = detect(image, weights='path/to/weights.pt', device='cpu', conf_thres=0.5)[0]

    # ... process detection results ...

You can import the detection function from the yolov5 repository as shown above. You have to ensure that you have the necessary requirements installed and the weights file is available. You can specify a path to the weights file in the detect function as shown above.

Please let me know if you have any further questions or concerns. We are happy to help!

Best, Glenn Jocher

[lucastack]

Hello @SJavad 👋!

Flask will add an unnecesary bottleneck. Instead, since you can connect directly as you showed in your first script, you just need to preprocess the frame and then run the inference just as is done in detect.py script.

By the way, detect.py has RTSP support to work with live cameras and such. Does your camera support RTSP transmision?

Regards

[SJavad]

@SJavad, based on your implementation, it seems that you are using a Flask server to serve the video frames from the Basler camera. This could possibly lead to a low frame rate compared to accessing the camera directly.

Regarding using the detect.py script, you can import the necessary libraries and functions from the script and use the detect() function to obtain the detected objects from the frames. You can modify your script to include the detection function as shown below:

import cv2
from yolov5.detect import detect

# ... camera setup here ...

while True:
    # ... get image from camera ...
    # convert image from camera to BGR format
    image = cv2.cvtColor(image, cv2.COLOR_GRAY2BGR)

    # run detection on the image
    results = detect(image, weights='path/to/weights.pt', device='cpu', conf_thres=0.5)[0]

    # ... process detection results ...

You can import the detection function from the yolov5 repository as shown above. You have to ensure that you have the necessary requirements installed and the weights file is available. You can specify a path to the weights file in the detect function as shown above.

Please let me know if you have any further questions or concerns. We are happy to help!

Best, Glenn Jocher

@glenn-jocher Thanks for your help ❤🖐 import yolov5.detect does not help me but I try to load a model and make a prediction with this block of code that I see in the yolov5 GitHub repo:

model = torch.hub.load('ultralytics/yolov5', 'custom',"..\FinalModels\Yolov5SG.pt")
while (True):
    image = read.image()
    pred = model(image)
    print(pred)

and now, how can I draw bounding boxes that I get from the pred variable? also, I want to get confidence values like when I type the command python detect.py --source 0. this code which I write only this information:

image 1/1: 1080x1900 1 object-1
Speed: 4.0ms pre-process, 122.7ms inference, 1.0ms NMS per image at shape (1, 3, 384, 640)

[SJavad]

Hello @SJavad 👋!

Flask will add an unnecesary bottleneck. Instead, since you can connect directly as you showed in your first script, you just need to preprocess the frame and then run the inference just as is done in detect.py script.

By the way, detect.py has RTSP support to work with live cameras and such. Does your camera support RTSP transmision?

Regards

Yes you right using Flask for inference with Basler is not recommended but it works by the way. mine is a USB camera and I don't think that supports RTSP protocol.

[SJavad]

@SJavad, based on your implementation, it seems that you are using a Flask server to serve the video frames from the Basler camera. This could possibly lead to a low frame rate compared to accessing the camera directly.

Regarding using the detect.py script, you can import the necessary libraries and functions from the script and use the detect() function to obtain the detected objects from the frames. You can modify your script to include the detection function as shown below:

import cv2
from yolov5.detect import detect

# ... camera setup here ...

while True:
    # ... get image from camera ...
    # convert image from camera to BGR format
    image = cv2.cvtColor(image, cv2.COLOR_GRAY2BGR)

    # run detection on the image
    results = detect(image, weights='path/to/weights.pt', device='cpu', conf_thres=0.5)[0]

    # ... process detection results ...

You can import the detection function from the yolov5 repository as shown above. You have to ensure that you have the necessary requirements installed and the weights file is available. You can specify a path to the weights file in the detect function as shown above.

Please let me know if you have any further questions or concerns. We are happy to help!

Best, Glenn Jocher

@glenn-jocher I have another question in this line:

model = torch.hub.load('ultralytics/yolov5', 'custom',"..\FinalModels\Yolov5SG.pt")

I have to connect to the internet for downloading some repos... how can I load the model in this way without connecting to the internet?

[github-actions[bot]]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐

[glenn-jocher]

@SJavad, you can load the YOLOv5 model without an internet connection by manually providing the model's weights file. To do so, you can directly load the model using load_model function from the models module. Here's an example:

import torch
from models.yolo import Model

weights_path = 'path_to_weights.pt'
model = Model(weights_path, img_size=640)

In the weights_path, you need to provide the local path to the .pt weights file. Make sure you have the necessary requirements installed and the weights file is accessible at the specified path.

Regarding drawing bounding boxes and obtaining confidence values from the predictions, you can access these values directly from the pred object. Here's an example:

for det in pred:
    # Extract class, confidence, and bounding box coordinates
    cls = det[-1]
    conf = det[4]
    bbox = det[:4]

    # Draw bounding box on the image
    cv2.rectangle(image, (bbox[0], bbox[1]), (bbox[2], bbox[3]), (255, 0, 0), 2)

    # Print class, confidence, and bounding box coordinates
    print(f'Class: {cls}, Confidence: {conf}, Bounding Box: {bbox}')

This code snippet demonstrates how to draw bounding boxes and retrieve confidence values from the pred object.

If you have any additional questions or concerns, feel free to ask. We are here to assist you!

[SJavad]

@SJavad, you can load the YOLOv5 model without an internet connection by manually providing the model's weights file. To do so, you can directly load the model using load_model function from the models module. Here's an example:

import torch
from models.yolo import Model

weights_path = 'path_to_weights.pt'
model = Model(weights_path, img_size=640)

In the weights_path, you need to provide the local path to the .pt weights file. Make sure you have the necessary requirements installed and the weights file is accessible at the specified path.

Regarding drawing bounding boxes and obtaining confidence values from the predictions, you can access these values directly from the pred object. Here's an example:

for det in pred:
    # Extract class, confidence, and bounding box coordinates
    cls = det[-1]
    conf = det[4]
    bbox = det[:4]

    # Draw bounding box on the image
    cv2.rectangle(image, (bbox[0], bbox[1]), (bbox[2], bbox[3]), (255, 0, 0), 2)

    # Print class, confidence, and bounding box coordinates
    print(f'Class: {cls}, Confidence: {conf}, Bounding Box: {bbox}')

This code snippet demonstrates how to draw bounding boxes and retrieve confidence values from the pred object.

If you have any additional questions or concerns, feel free to ask. We are here to assist you!

Thank you so much @glenn-jocher your advice was very helpful as always.

[glenn-jocher]

You're welcome, @SJavad! If you have any other questions or need further assistance as you continue with your project, feel free to reach out. Wishing you the best with your face mask detection project using YOLOv5! 👍