Sending kivy android video feed to imagezmq server

[ChesterProgram]

is it possible to send android video feed using kivy and opencv to the imagezmq server?

[jeffbass]

I don't know. I have never used kivy. Looking at its docs, it seems likely that it could work. I don't know if there are any incompatibilities with the combination of kivy, numpy, opencv and ZMQ. I suggest you give it a try with one of the imageZMQ test pairs. Please comment back here with what you find out.

Thanks, Jeff

[ChesterProgram]

So I have a question... does imagezmq only send opencv images or any images?

[jeffbass]

imagezmq sends opencv images or jpg images (or any image that is a "binary blob" that can be placed in a buffer). The send_image and recv_image methods are specific to opencv images. The send_jpg and recv_jpg were implemented to send binary buffers holding jpg images, but any arbitrary binary buffer can be sent using those methods. There are examples in the test folder and the examples folder. The imagezmq API is here.

[ChesterProgram]

This is the Client Code

 from kivy.uix.camera import Camera
from kivy.app import App
from kivy.lang import Builder
from kivy.utils import platform
from kivy.uix.widget import Widget
from kivy.uix.boxlayout import BoxLayout
from threading import Thread
from kivy.clock import Clock
import socket
import imagezmq
import cv2
import numpy as np

if platform == "android":
    from android.permissions import Permission, request_permissions

    request_permissions([Permission.READ_EXTERNAL_STORAGE,
                         Permission.WRITE_EXTERNAL_STORAGE,
                         Permission.INTERNET,
                         Permission.CAMERA])

Builder.load_string("""
<MyLayout>:
    AndroidCamera:
        index: 0
        id: a_cam
        resolution: (640, 480)
        play: True
        canvas.before:
            PushMatrix
            Rotate:
                angle: 0
                origin: self.center
            Scale:
                x: self.cam_ratio
                y: self.cam_ratio
                origin: self.center
        canvas.after:
            PopMatrix
""")

class AndroidCamera(Camera):
    camera_resolution = (640, 480)
    cam_ratio = camera_resolution[0] / camera_resolution[1]

class MyLayout(BoxLayout):
    pass

class MyApp(App):
    rpiName = socket.gethostname()
    sender = imagezmq.ImageSender(connect_to='tcp://192.168.0.124:5555')
    jpeg_quality = 95

    def build(self):
        return MyLayout()

    def send(self, jpg_image):
        self.sender.send_jpg(self.rpiName, jpg_image)

    def on_start(self):
        Clock.schedule_once(self.sending, 1)

    def sending(self, dt):
        cam = self.root.ids.a_cam
        image_object = cam.export_as_image(scale=round((400 / int(cam.height)), 2))
        w, h = image_object._texture.size
        frame = np.frombuffer(image_object._texture.pixels, 'uint8').reshape(h, w, 4)
        BGR = cv2.cvtColor(frame, cv2.COLOR_RGBA2BGR)
        ret_code, jpg_image = cv2.imencode(
            ".jpg", BGR, [int(cv2.IMWRITE_JPEG_QUALITY), self.jpeg_quality])
        Thread(target=self.send, args=(jpg_image, )).start()
        Clock.schedule_once(self.sending, 0.015)

if __name__ == "__main__":
    MyApp().run()

[ChesterProgram]

This is the server code

from imutils import build_montages
import numpy as np
import imagezmq
import cv2

imageHub = imagezmq.ImageHub()

while True:
    rpiName, jpg_image = imageHub.recv_jpg()
    frame = cv2.imdecode(np.frombuffer(jpg_image, dtype='uint8'), -1)
    imageHub.send_reply(b'OK')
    cv2.imshow(rpiName, frame)
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

cv2.destroyAllWindows()

[jeffbass]

Thanks for sharing your code! I'm sure others will find it helpful.

[Eliastoomme]

@jeffbass
Can you please share your buildozer.spec file? I'm also trying to create an android app that sends frame with imagezmq to a server. I've managed to get it working on PC, but not on Android as it when building. I know its been long for this, but I will be really grateful if you help.

[jeffbass]

I don't use Android. I will leave this for @ChesterProgram (who opened this issue & posted the above code) to reply to.