Sender's closure detection

[ksadura]

Hi everybody. My question is, whether is it possible to detect and handle in the receiver programme - ImageHub() the moment, when sender programme - ImageSender() is being closed (i.e. ctrl + z has been pressed in sender terminal). I have accomplished similar situation, when I worked with normal sockets before, but now with imagezmq and zmq sockets I am struggling. Thanks in advance.

[jeffbass]

imageZMQ is somewhat different than typical socket programs. imageZMQ transports openCV images. Specifically, it transmits (text_message, openCV_image) tuples. The imageZMQ code does not do any checking or interpretation of these tuples; it assumes the sending and receiving code will do all of that. The is no "end of file" marker that imageZMQ reacts to. That is the responsibility of the sending program and the receiving program.

Two of the programs in the tests folder in this repository, timing_send_images.py and timing_receive_images.py, show how to "wrap" image sending in a try/except block to catch a Ctrl-C interrupt. Below I present modified versions of these programs. The sending program demonstrates a very simple way to have the sender send a "stop" message when it catches a Ctrl-C. The receiving program demonstrates a very simple way to react to that stop message by printing the stop message and exiting.

The sending program is a modified version of timing_send_images.py. This modified sending program sends a 'stop' string and a tiny blank image when Ctrl-C is pressed. It requires some ZMQ error checking to make sure that the 'stop' message gets sent if the Ctrl-C occurs during the send statement in image sending loop. In a full program using this approach, additional ZMQ error checking would be required, but this example program runs well enough to illustrate a way to send a 'stop' signal.

"""timing_send_images.py -- send PiCamera image stream.

Modified for example to reply to imageZMQ issue #64
Modified areas have comments # for issue 64

A Raspberry Pi test program that uses imagezmq to send image frames from the
PiCamera continuously to a receiving program on a Mac that will display the
images as a video stream.

This program requires that the image receiving program be running first. Brief
test instructions are in that program: timing_receive_images.py.

This program can turn an LED on and off if needed; assumes BCM pin 18. This
can help with lighting the subject area in front of the PiCamera.
"""

import sys

import socket
import time
import traceback
import cv2
from imutils.video import VideoStream
import imagezmq
import numpy as np  # for issue 64 to make a tiny null image, see below
import zmq # for issue 64 to use some ZMQ error codes

# use either of the formats below to specifiy address of display computer
# sender = imagezmq.ImageSender(connect_to='tcp://jeff-macbook:5555')
# sender = imagezmq.ImageSender(connect_to='tcp://192.168.1.190:5555')
# for issue 64 sending and receiving on same rpi
sender = imagezmq.ImageSender(connect_to='tcp://127.0.0.1:5555') # for issue 64
sender.zmq_socket.setsockopt(zmq.LINGER, 0)  # for issue 64, prevents ZMQ hang on Ctrl-C

rpi_name = socket.gethostname()  # send RPi hostname with each image
picam = VideoStream(usePiCamera=True).start()
time.sleep(2.0)  # allow camera sensor to warm up
try:
    while True:  # send images as stream until Ctrl-C
        image = picam.read()
        # processing of image before sending would go here.
        # for example, rotation, ROI selection, conversion to grayscale, etc.
        reply_from_hub = sender.send_image(rpi_name, image)
        # above line shows how to capture REP reply text from Hub
except (KeyboardInterrupt, SystemExit):
    pass  # Ctrl-C was pressed to end program
except Exception as ex:
    print('Python error with no Exception handler:')
    print('Traceback error:', ex)
    traceback.print_exc()
finally:
    stop_string = 'stop'  # for issue 64, send a stop message and null tiny image
    tiny_image = np.zeros((3, 3, 3), dtype="uint8")  # tiny blank image
    try:  # for issue 64, need to trap possible ZMQErrors while sending stop_string & tiny_image 
        reply_from_hub = sender.send_image(stop_string, tiny_image)
    except (zmq.ZMQError, zmq.ContextTerminated, zmq.Again):
        if 'sender' in locals():
            sender.close()
        sender = imagezmq.ImageSender(connect_to='tcp://127.0.0.1:5555') # for issue 64
        reply_from_hub = sender.send_image(stop_string, tiny_image)
    picam.stop()  # stop the camera thread
    sender.close()  # close the ZMQ socket and context
    sys.exit()

The receiving program is based on timing_receive_images.py. It examines the (text_message, openCV_image) tuple for every image it receives. If the 'text_message' portion is 'stop', then the receiving program exits:

"""timing_receive_images.py -- receive and display images, then print FPS stats

Modified for example to reply to imageZMQ issue #64
Modified areas have comments # for issue 64

A timing program that uses imagezmq to receive and display an image stream
from one or more Raspberry Pi computers and print timing and FPS statistics.

1. Run this program in its own terminal window on the mac:
python timing_receive_images.py

This "receive and display images" program must be running before starting
the RPi image sending program.

2. Run the image sending program on the RPi:
python timing_send_images.py

A cv2.imshow() window will appear on the Mac showing the tramsmitted images
as a video stream. You can repeat Step 2 and start the timing_send_images.py
on multiple RPis and each one will cause a new cv2.imshow() window to open.

To end the programs, press Ctrl-C in the terminal window of the receiving
program first, so that FPS and timing statistics will be accurate. Then, press
Ctrl-C in each terminal window running a Rasperry Pi image sending program.
"""

import sys

import time
import traceback
import cv2
from collections import defaultdict
from imutils.video import FPS
import imagezmq

# instantiate image_hub
image_hub = imagezmq.ImageHub()

image_count = 0
sender_image_counts = defaultdict(int)  # dict for counts by sender
first_image = True

try:
    while True:  # receive images until Ctrl-C is pressed
        sent_from, image = image_hub.recv_image()
        if sent_from == 'stop':  # for issue 64, check for stop message from sender
            print('stop received from sender')  # for issue 64, print stop message
            raise SystemExit # for issue 64, exit program when receiving stop message
        if first_image:
            fps = FPS().start()  # start FPS timer after first image is received
            first_image = False
        fps.update()
        image_count += 1  # global count of all images received
        sender_image_counts[sent_from] += 1  # count images for each RPi name
        cv2.imshow(sent_from, image)  # display images 1 window per sent_from
        cv2.waitKey(1)
        # other image processing code, such as saving the image, would go here.
        # often the text in "sent_from" will have additional information about
        # the image that will be used in processing the image.
        image_hub.send_reply(b"OK")  # REP reply
except (KeyboardInterrupt, SystemExit):
    pass  # Ctrl-C was pressed to end program; FPS stats computed below
except Exception as ex:
    print('Python error with no Exception handler:')
    print('Traceback error:', ex)
    traceback.print_exc()
finally:
    # stop the timer and display FPS information
    print()
    print('Test Program: ', __file__)
    print('Total Number of Images received: {:,g}'.format(image_count))
    if first_image:  # never got images from any RPi
        sys.exit()
    fps.stop()
    print('Number of Images received from each RPi:')
    for RPi in sender_image_counts:
        print('    ', RPi, ': {:,g}'.format(sender_image_counts[RPi]))
    image_size = image.shape
    print('Size of last image received: ', image_size)
    uncompressed_size = image_size[0] * image_size[1] * image_size[2]
    print('    = {:,g} bytes'.format(uncompressed_size))
    print('Elasped time: {:,.2f} seconds'.format(fps.elapsed()))
    print('Approximate FPS: {:.2f}'.format(fps.fps()))
    cv2.destroyAllWindows()  # closes the windows opened by cv2.imshow()
    image_hub.close()  # closes ZMQ socket and context
    sys.exit()

I tested both of the above programs by running them on an RPi with a camera module. I first started the receiving program in one terminal window, then started the sending program in a 2nd terminal window. An OpenCV cv2.imshow() window opened and showed images from the camera. When I pressed Ctrl-C in the terminal window running the sending program, the message "stop received from sender" printed in the terminal window running the receiving program and the receiving program exited. The sending program may need more than 1 press of Ctrl-C to stop it; it would also need additional nested try/except blocks to trap additional errors if used in a production program. But it illustrates one way to send a text "stop" message using imageZMQ. Remember, all imageZMQ does is transmit (text_message, openCV_image) tuples. If you want to send some kind of "stop" message from the sender, these programs show one way to do it using imageZMQ. In my imagenode repository, you can see examples of sending text messages to the imagehub using this approach. The README in my imagehub repository shows what a stream of these messages looks like.

You may want to try something like this as a way to detect and handle closing the sender program. There are many possible variations of this approach.

Jeff