jeffbass / imagezmq

A set of Python classes that transport OpenCV images from one computer to another using PyZMQ messaging.
MIT License
1.01k stars 160 forks source link

Send receive images back to client from server #45

Open sureshgorakala opened 3 years ago

sureshgorakala commented 3 years ago

Hi @jeffbass , first of all, I would like to thank you for writing such as wonderful library. Currently I'm working a POC where video stream will be captured from web cam and sends to a cloud server (AWS) for some kind of face recognition stuff. The processed frame has to be sent back to client for displaying. The flow is shown below pubsub

Question: 1. Will the current implementation handles this?

  1. If yes, can you please provide some kind of pseudo code or existing examples? Thanks for the reply in advance.
jeffbass commented 3 years ago

Hi @sureshgorakala, Short Answers: Question 1: Yes, but only if you can send images to your local machine's IP address (see below). Question 2: I'd be happy to put together an example of your full loop (see below).

Longer Answer, with details: Yes, I believe this could be done, but, because each imageZMQ sender / receiver pair sends images one way only, it would require 2 different imageZMQ pairs. The "send frames to cloud server" imageZMQ pair would use an image sender locally and an image receiver in the cloud. The "send frames back to local" imageZMQ pair would use an image sender in the cloud and an image receiver locally. Each imageZMQ pair would need a unique port number (for example port 5555 for the "send frames to cloud" pair and port 5556 for the "send frames back to local" pair).

The difficult part would be the "send frames back to local" machine's IP address. The IP address of your cloud instance is easy to determine and easy to send to. However, unless your local machine has a fixed IP address that imageZMQ can send to easily, imageZMQ wouldn't work for your round trip design.

You can test the "send frames back to local" IP address pretty easily using one of the imageZMQ tests. Use Test 1 (see the imageZMQ README). Run the send program (which makes a simple test image with a number in a rectangle) on your cloud instance; in that program, you will need to specify the IP address of your local machine. Run the receiving program on your local machine, and it will display the numbers in the rectangle. Remember to start the receiving program on your local machine first. If you haven't run the imageZMQ Test 1 before, I recommend that you run it (both programs) on your local machine and get it working before you try uploading and running the sender program on your cloud instance.

Run imageZMQ Test 1 to "send frames from cloud to local" as described above. Getting it working depends on getting your local machine IP address, local firewall settings, inbound port, etc. working. I think it might be challenging. If you get that working, let me know by posting a comment in this issue. At that point, I will put together a set of test programs that demonstrate your round trip loop above. It is a pretty simple set of modifications to some of the imageZMQ test programs. I would do it as 3 programs. A "sender" program on your local machine; a "receive-process-send back" program on the remote machine, and a "receive-and-display" program on your local machine. I would write the 3 programs using the IP addresses of 2 machines in my local network. I don't do any inbound "through the firewall" IP stuff, so I don't know much about that. But if you know the IP addresses of both machines (and you can send images through the firewall to both of those IP addresses), your program is pretty easy.

Speed would also likely be an issue; I'd recommend using jpgs in production and doing testing at your desired resolution. More comments about speed issues are in issue #41. Jeff