Issue while sending live frames from Jetson Nano to PC

[tulbureandreit]

Hello @jeffbass,

So TLDR: I try to send frames from my Jetson nano (running ubuntu18.04) to my PC. They are connected on the same ethernet connection. I use the same syntax as from your /tests/test3-***.py programs. And it does not work, it just freezes without any errors.

Any suggestions?

receiver:

import sys
import cv2
import imagezmq
import numpy as np

image_hub = imagezmq.ImageHub()

print('image hub created')
while True:
    print('entered while')
    image_name, image_buffer = image_hub.recv_jpg()
    print('1')
    image = cv2.imdecode(np.frombuffer(image_buffer, dtype='uint8'), -1)
    print('2')
    cv2.imshow(image_name, image)
    image_hub.send_reply(b'OK')
    print('ok')
    keyCode = cv2.waitKey(30)
    if keyCode == ord('q'):
        break
cv2.destroyAllWindows()

---

sender:

import cv2
import numpy as np
from elements.yolo import OBJ_DETECTION
import base64
import imagezmq
import socket
import sys
import time

Object_classes = ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light',
                'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow',
                'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee',
                'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard',
                'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple',
                'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch',
                'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone',
                'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear',
                'hair drier', 'toothbrush' ]

Object_colors = list(np.random.rand(80,3)*255)
Object_detector = OBJ_DETECTION('weights/yolov5s.pt', Object_classes)

sender = imagezmq.ImageSender(connect_to='tcp://192.168.0.107:5555')
i = 0
jetson_name = socket.gethostname()

def gstreamer_pipeline():
    return (
        "v4l2src device=/dev/video0 ! "
        "video/x-raw, width=640, height=480 !"
        "videoconvert ! appsink" )

time.sleep(1)
jpeg_quality = 95
print(gstreamer_pipeline())
cap = cv2.VideoCapture(gstreamer_pipeline(), cv2.CAP_GSTREAMER)
cap = cv2.VideoCapture(gstreamer_pipeline(), cv2.CAP_GSTREAMER)
if cap.isOpened():
    #window_handle = cv2.namedWindow("CSI Camera", cv2.WINDOW_AUTOSIZE)
    #while cv2.getWindowProperty("CSI Camera", 0) >= 0:
    while True:
        ret, frame = cap.read()
        if ret:
            # detection process
            objs = Object_detector.detect(frame)
            for obj in objs:
                # print(obj)
                label = obj['label']
                score = obj['score']
                [(xmin,ymin),(xmax,ymax)] = obj['bbox']
                color = Object_colors[Object_classes.index(label)]
                frame = cv2.rectangle(frame, (xmin,ymin), (xmax,ymax), color, 2) 
                frame = cv2.putText(frame, f'{label} ({str(score)})', (xmin,ymin), cv2.FONT_HERSHEY_SIMPLEX , 0.75, color, 1, cv2.LINE_AA)
        i = i+1
        print('Seding image' + str(i))
        ret_code, frame_buffer = cv2.imencode(".jpg", frame, [int(cv2.IMWRITE_JPEG_QUALITY), jpeg_quality])
        print('encoded' + str(i))
        sender.send_jpg(jetson_name, frame_buffer)
        print('sent' + str(i))
        cv2.imshow("CSI Camera", frame)
        #time.sleep(1)
        keyCode = cv2.waitKey(30)
        if keyCode == ord('q'):
            break
    cap.release()
    cv2.destroyAllWindows()
else:
    print("Unable to open camera")

[tulbureandreit]

More details, it seems to get stuck at the sender.send_jpg(jetson_name, frame_buffer) line

[jeffbass]

Hi @tulbureandreit, Were you able to run the /tests/test3-***.py programs without changes (other than using your own TCP address)? I would suggest doing that to see if imageZMQ is working OK on your Jetson Nano and your PC hardware with your TCP address and port number. What happens when you run them?

Also, could you format your code using code fencing and syntax highlighting so it is more readable, please. You can edit your existing message rather than re-writing it. GitHub docs on using fenced code blocks with syntax highlighting are here. Thanks, Jeff

[tulbureandreit]

@jeffbass I edited the comment. Sorry.

I will run the tests and update you, but I do not have a PiCamera (I have an USB camera), thus I will still need to modify a little bit the code.

Thanks a lot!

[tulbureandreit]

Hello. I did not manage to run them successfully. As I see, the problem is at the receiver`s end. At:

 image_name, image_buffer = image_hub.recv_jpg()

Because when I debugged the code with prints, this is the only line that did not work. Also, what`s weird is that the While from the sender only looped once, then it blocked ( i think it is waiting for responde from the sender? )

Thank you in advance. Ps: is it easier with a PiCamera?

[jeffbass]

It should not make a difference whether a PiCamera or a USB camera is used. In fact, to facilitate debugging, you may want to borrow code from the imagezmq/tests/test_1_send_images.py program to simply send images that are created to send digit images without a camera. My initial guess is that your receiver has the inbound port blocked somehow. Is your receiving computer running Windows? If so, there are some suggestions for opening the port, etc., in issues #19 & #20. I don't have a Windows computer, so I cannot be much help with Windows.