New Feature Request: Bi-Directional Messaging

DJDIngram commented 4 years ago

Hello! Firstly, thank you very much for your work, VidGear is working really well with my project! I'm able to send one way video and other information from a Raspberry Pi (server) to a second computer (client).

Looking at the docs, the "pattern" used to configure Netgear is ZMQ.pair by default - which is said to be bidirectional. Could this be used to send information back to the server?

For instance, the server could send the frame and a string of information over to the client, which could reply with confirmation that it has recieved the data, as well as any other information. In my case, it could send data back to a raspberry pi which can be parsed to turn on an LED or drive a motor.

That would be super useful to my project, and may help others with theirs!

Thank you!

abhiTronix commented 4 years ago

@DJDIngram Thanks for this proposal. I have got a concern related to it as follows

Could this be used to send information back to the server?

Yes but there's a catch at receiver's end in NetGear API that utilizes multithreading to speed up things. This means data is being processed at a separate thread than the main thread at the receiver's end. Hence currently it is difficult to directly send data from recv() back to send() in realtime. We have two approaches to tackle this problem:

Approach 1: Defining a thread-safe fixed-length global queue or variable that takes data from the user directly at the receiver's end and sends it directly to the server's end where you can take care of data received by accessing that global queue/variable. Problem: This has a downside too as if the server is slower than the client(receiver) than this will lead to loss of data as data will get overwritten in the fixed-length queue as we cannot keep track of both queue at the same time.
Approach 2: Returning a status at send() function that returns a true boolean value if data is received successfully and false if don't. This approach has no downside and easier to implement.

What are your thoughts on this?

abhiTronix commented 4 years ago

@DJDIngram, Thanks for your patience, Kindly install the PR(#76) as follows:

git clone https://github.com/abhiTronix/vidgear.git
cd vidgear
git checkout development
sudo pip3 install .
cd

and test this example: https://github.com/abhiTronix/vidgear/wiki/Bidirectional-Mode:-Bidirectional-Data-Transfer-in-NetGear-API#1-bidirectional-data-transfer and see if it works as you expected or not? and then revert here. Good luck!

abhiTronix commented 4 years ago

@DJDIngram I'm hoping you to test this PR as soon as possible as it will be going to be merged by tomorrow.

DJDIngram commented 4 years ago

@abhiTronix I should hopefully get the chance to do so today, else I will definitely get the chance to do so tomorrow

DJDIngram commented 4 years ago

@abhiTronix

I got a chance to have a go at testing the example code, swapping out the test.mp4 with a PiGear stream. I am not getting any output from either the client side or server side.

Server Side Log

[LOG]: Activating Pi camera at index: 0 [LOG]: protocol is not valid or provided. Defaulting to tcp protocol! [LOG]: ZMQ Security Mechanism is disabled for this connection! [LOG]: Bi-Directional Data Transmission is enabled for this connection! [LOG]: Successfully connected to address: tcp://localhost:5555. [LOG]: This device Unique ID is 94E81. [LOG]: Send Mode is successfully activated and ready to send data!

Client Side Log

[LOG]: protocol is not valid or provided. Defaulting to tcp protocol! [LOG]: ZMQ Security Mechanism is disabled for this connection! [LOG]: Bi-Directional Data Transmission is enabled for this connection! [LOG]: Threaded Queue Mode is enabled by default for NetGear. [LOG]: Successfully Binded to address: tcp://*:5555. [LOG]: Multi-threaded Receive Mode is enabled Successfully! [LOG]: Device Unique ID is 68B2E. [LOG]: Receive Mode is activated successfully! Starting Loop

I get the same output when using the specific client address (and configuring the NetGear settings as they were when I was sending the video stream one-way)

Any Ideas?

abhiTronix commented 4 years ago

@DJDIngram is pattern parameter values are same on both server and client end?

julio16101 commented 4 years ago

In the tests I have done, the courier delivery is successful.

Even so, is it possible to send a dictionary type data? to categorize the information sent.

DJDIngram commented 4 years ago

@abhiTronix Yes, pattern parameter values are the same.

I have copied your examples exactly, apart from line 34 on the server side code, where I get an "IdentationError" -> except KeyboardInterrupt:recv_data. I removed the recv_data part to compile and run the code.

abhiTronix commented 4 years ago

@DJDIngram Thanks for testing, I've corrected bug and few typos in code. Kindly reinstall the development branch by executing following commands in your terminal:

git clone https://github.com/abhiTronix/vidgear.git
cd vidgear
git checkout development
sudo pip3 install .
cd

Then retest the fixed example code by following one of the following ways:

Option1: manually as before from here: https://github.com/abhiTronix/vidgear/wiki/Bidirectional-Mode:-Bidirectional-Data-Transfer-in-NetGear-API#usage.
Option2: Otherwise, Download and Extract following test_code.zip file and then go (cd test_code) into the extracted folder and run the python files present in it in the separate terminals: (:warning: Don't change code or move files)

test_code.zip

then see if all works, then report here. Goodluck

abhiTronix commented 4 years ago

@julio16101 Yes, you can send any datatype which includes dict, list, int, bool, string, etc. bidirectionally

abhiTronix commented 4 years ago

@DJDIngram Kindly try the given solution as soon as possible.

DJDIngram commented 4 years ago

@abhiTronix Should be able to test it within the next few hours, sorry for the delay!

DJDIngram commented 4 years ago

@abhiTronix - I was able to reinstall the new vidgear, and ran your code from test_code.zip. I kept all the files in the same directory on both client and server computers. Unfortunately I am still getting the same problems as last time.

Server Side Log

[LOG]: protocol is not valid or provided. Defaulting to tcp protocol! [LOG]: ZMQ Security Mechanism is disabled for this connection! [LOG]: Force termination is enabled for this connection by default! [LOG]: Bi-Directional Data Transmission is enabled for this connection! [LOG]: Threaded Queue Mode is enabled by default for NetGear. [LOG]: Successfully Binded to address: tcp://*:5555 with pattern: 1. [LOG]: Multi-threaded Receive Mode is enabled Successfully! [LOG]: Device Unique ID is A5B7F. [LOG]: Receive Mode is activated successfully!

Client Side Log

[LOG]: protocol is not valid or provided. Defaulting to tcp protocol! [LOG]: ZMQ Security Mechanism is disabled for this connection! [LOG]: Force termination is enabled for this connection by default! [LOG]: Bi-Directional Data Transmission is enabled for this connection! [LOG]: Threaded Queue Mode is enabled by default for NetGear. [LOG]: Successfully Binded to address: tcp://*:5555 with pattern: 1. [LOG]: Multi-threaded Receive Mode is enabled Successfully! [LOG]: Device Unique ID is EBE8B. [LOG]: Receive Mode is activated successfully!

Any ideas?

DJDIngram commented 4 years ago

@abhiTronix This is odd - The previous one-way transmission code is also doing the same thing, not showing the the frames as it used to do. Could this be a problem with cv2 instead?

abhiTronix commented 4 years ago

Server Side Log

[LOG]: protocol is not valid or provided. Defaulting to tcp protocol! [LOG]: ZMQ Security Mechanism is disabled for this connection! [LOG]: Force termination is enabled for this connection by default! [LOG]: Bi-Directional Data Transmission is enabled for this connection! [LOG]: Threaded Queue Mode is enabled by default for NetGear. [LOG]: Successfully Binded to address: tcp://*:5555 with pattern: 1. [LOG]: Multi-threaded Receive Mode is enabled Successfully! [LOG]: Device Unique ID is A5B7F. [LOG]: Receive Mode is activated successfully! Client Side Log

[LOG]: protocol is not valid or provided. Defaulting to tcp protocol! [LOG]: ZMQ Security Mechanism is disabled for this connection! [LOG]: Force termination is enabled for this connection by default! [LOG]: Bi-Directional Data Transmission is enabled for this connection! [LOG]: Threaded Queue Mode is enabled by default for NetGear. [LOG]: Successfully Binded to address: tcp://*:5555 with pattern: 1. [LOG]: Multi-threaded Receive Mode is enabled Successfully! [LOG]: Device Unique ID is EBE8B. [LOG]: Receive Mode is activated successfully!

@DJDIngram Your Server and Client's end logs match exactly. This means you're doing this wrong as you're using the same code for Server-end and Client-end, Kindly check your code again and run server.py and client.py in separate terminals OR use my zip file is given above in my comment that contains both python files.

abhiTronix commented 4 years ago

@julio16101 Can you test my instructions and zip given in this comment, and revert your result with complete log here?

DJDIngram commented 4 years ago

That was embarassing, apologies!

Running the client and server files on seperate terminals on the raspberry pi works fine! The bidirectional sending mode is working perfectly on the same computer.

Just tested it across the raspberry pi and the laptop. I added an address and port to the settings and it worked fine! I'm not sure what I was doing wrong before, but I won't complain.

DJDIngram commented 4 years ago

Just changed the video source to the PiGear camera with no issues. @abhiTronix you are a true hero! Thank you!

abhiTronix commented 4 years ago

Just tested it across the raspberry pi and the laptop. I added an address and port to the settings and it worked fine! I'm not sure what I was doing wrong before, but I won't complain.

@DJDIngram No problem, If you want the full code, here it is to use different machines on a network:

A. Client's End:

Open a terminal on the System(a Client, where you want to display the input frames received from the Server) and execute the following python code: Also, Remember the IP-address of this system(required at Server's end) by executing the command: hostname -I and also replace it in the following code.

# import libraries
from vidgear.gears import NetGear
import cv2

options = {'bidirectional_mode': True}

#change following IP address '192.168.x.xxx' with yours
client = NetGear(address = '192.168.x.xxx', port = '5454', protocol = 'tcp',  pattern = 1, receive_mode = True, logging = True, **options) #Define netgear client at Server IP address.

# infinite loop
while True:

    #prepare data to be sent
    target_data = "Hi, I am a Client here."

    # receive frames & data from server and also send our data
    data = client.recv(return_data = target_data)

    # check if data is None
    if data is None:
        #if True break the infinite loop
        break

    #extract server_data & frame from data
    server_data, frame = data

    # also check if frame is None
    if frame is None:
        #if True break the infinite loop
        break

    # print recieved server data
    if not(server_data is None): print(server_data)

    # do something with frame here

    # Show output window
    cv2.imshow("Output Frame", frame)

    key = cv2.waitKey(1) & 0xFF
    # check for 'q' key-press
    if key == ord("q"):
        #if 'q' key-pressed break out
        break

# close output window
cv2.destroyAllWindows()
# safely close client
client.close()

B. Server End:

Now, Open the terminal on another System(Raspberry Pi with PiCam) and execute the following python code: Also, remember to replace the IP address in the following code with Client's IP address copied earlier

Tip::bulb: You can end streaming anytime on both Server and Client by pressing [Ctrl+C] on your keyboard at Server's end!

# import libraries
from vidgear.gears import NetGear
from vidgear.gears import PiGear
import cv2

options = {"hflip": True, "exposure_mode": "auto", "iso": 800, "exposure_compensation": 15, "awb_mode": "horizon", "sensor_mode": 0} # define tweak parameters

stream = PiGear(resolution=(320, 240), framerate=60, time_delay=1,  logging = True, **options).start() # define various attributes and start the stream

options = {'bidirectional_mode': True}

#change following IP address '192.168.x.xxx' with yours
server = NetGear(address = '192.168.x.xxx', port = '5454', protocol = 'tcp',  pattern = 1, receive_mode = False, logging = True, **options) #Define netgear server at your system IP address.

# infinite loop until [Ctrl+C] is pressed
while True:
        try: 
        frame = stream.read()
        # read frames

        # check if frame is None
        if frame is None:
            #if True break the infinite loop
            break

        # do something with frame here

        #prepare data to be sent
        target_data = 'Hello, I am a Server.'

        # send frame & data and also recieve data from client
        recv_data = server.send(frame, message = target_data)

        #print data just recieved from client
        if not(recv_data is None): print(recv_data)

    except KeyboardInterrupt:
        #break the infinite loop
        break

# safely close video stream
stream.stop()
# safely close server
server.close()

abhiTronix commented 4 years ago

@DJDIngram Thanks you for coming up with this proposal

abhiTronix commented 4 years ago

Successfully resolved and merged in commit: e918d13

abhiTronix / vidgear