can i not use mplayer to play the EVENT_VIDEO_FRAME

[HoYinChung]

from the example, there is the way top print the data to video_player directly. However, I cant use the mplayer.

is that possible to play the EVENT_VIDEO_FRAME data to like a cv2 frame like the example for video effect? thanks

[hanyazou]

I think you can just use get_video_stream() like video_effect example. The video_effect example uses get_video_stream(). That handle EVENT_VIDEO_FRAME in it. Please see tellopy/_internal/video_stream.py.

If you want to handle video frame and control the drone by user input simultaneously, you may create an additional thread for that.

[HoYinChung]

Thanks, so i think i can follow the usage of get video stream and use it in the handler🙂as using the one for video effect , i do have another usage in the while loop so i think video effect is not suitable. Thank you 👍

[hanyazou]

Here is a threaded example of handing frames. https://github.com/hanyazou/TelloPy/blob/heavy_video_effect/tellopy/examples/video_effect.py

In this case, you can do what you like in the main loop while it displays the video frames. The key point is the additional thread.

[HoYinChung]

what if in the while loop i have another control not regarding frame, for example keyboard event sounds like they will have conflict?

[HoYinChung]

as i would like to control the drone with keyboard like the joystick example one but not with joystick and mplayer thanks

[hanyazou]

It will have conflict if you send takeoff command form a thread and you send land command from another thread at the same time. It might be OK if you handle video frames in a thread and send flight command from another thread. It is recommended that you should send/set/change tello object from one thread and other threads just receive info from the object.

[HoYinChung]

thank you, sounds like i can just put the while loop in the thread, and in the other thread i can do the control thanks

[HoYinChung]

import time
import sys
import tellopy
import keyboard
import pygame
import cv2
import numpy
import av
import threading
import traceback

from pygame.locals import *

prev_flight_data = None
run_recv_thread = True
run_controller_thread = True

def recv_thread():
    global frame
    global run_recv_thread
    global drone
    print('start recv_thread()')
    try:
        container = av.open(drone.get_video_stream())
        frame_count = 0
        while run_recv_thread:
            for f in container.decode(video=0):
                frame_count = frame_count + 1
                # skip first 300 frames
                if frame_count < 300:
                    continue
                frame = f
            time.sleep(0.01)
    except Exception as ex:
        exc_type, exc_value, exc_traceback = sys.exc_info()
        traceback.print_exception(exc_type, exc_value, exc_traceback)
        print(ex)
    finally:
        run_recv_thread = False

def controller_thread():
    global drone
    global run_controller_thread
    print('start controller_thread()')
    try: 
        while run_controller_thread:
            time.sleep(.200)
            # takeoff
            if keyboard.is_pressed('space'):
                drone.takeoff()
            # height and rotate
            if keyboard.is_pressed('left'):
                drone.left(10 )
            if keyboard.is_pressed('up'):
                drone.forward(10)
            if keyboard.is_pressed('right'):
                drone.right(10)
            if keyboard.is_pressed('down'):
                drone.backward(10)
            # direction
            if keyboard.is_pressed('w'):
                drone.up(10)
            if keyboard.is_pressed('a'):
                drone.counter_clockwise(10)
            if keyboard.is_pressed('s'):
                drone.down(10)
            if keyboard.is_pressed('d'):
                drone.clockwise(10)
            # land
            if keyboard.is_pressed('l'):
                drone.land()
            if keyboard.is_pressed('esc'):
                drone.land()
                break
    except KeyboardInterrupt as e:
        print(e)
    except Exception as e:
        print(e)
    finally:
        run_controller_thread = False

def handler(event, sender, data, **args):
    global prev_flight_data
    drone = sender
    if event is drone.EVENT_FLIGHT_DATA:
        if prev_flight_data != str(data):
            print(data)
            prev_flight_data = str(data)
    else:
        print('event="%s" data=%s' % (event.getname(), str(data)))

def main():
    global frame
    global drone
    drone = tellopy.Tello()
    drone.connect()
    drone.start_video()
    drone.subscribe(drone.EVENT_FLIGHT_DATA, handler)
    drone.subscribe(drone.EVENT_VIDEO_FRAME,handler)
    print("Start Running")
    try:
        threading.Thread(target=recv_thread).start()
        threading.Thread(target=controller_thread).start()
        while True:
            if frame is None:
                time.sleep(0.01)
            else:
                image = cv2.cvtColor(numpy.array(frame.to_image()), cv2.COLOR_RGB2BGR)
                cv2.imshow('Original', image)
                cv2.imshow('Canny', cv2.Canny(image, 100, 200))
                cv2.waitKey(1)
                # long deley
                time.sleep(0.5)
                image = None
    except KeyboardInterrupt as e:
        print(e)
    except Exception as e:
        print(e)

    drone.quit()
    exit(1)

if __name__ == '__main__':
    main()

The Above Code I can control the drone but the video thread was quit unexpectedly. Do You have any ideas? thanks

[hanyazou]

"quit unexpectedly" without any output message? how you could know it was terminated? you should share the output of your program.

[HoYinChung]

Tello: 15:40:29.563:  Info: start video thread
Tello: 15:40:29.563:  Info: send connection request (cmd="conn_req:9617")
Tello: 15:40:29.563:  Info: video receive buffer size = 524288
Tello: 15:40:29.564:  Info: state transit State::disconnected -> State::connecting
Tello: 15:40:29.564:  Info: start video (cmd=0x25 seq=0x01e4)
Start Running
start recv_thread()
Tello: 15:40:29.564:  Info: get video stream
start controller_thread()
name 'frame' is not defined
Tello: 15:40:29.565:  Info: start video (cmd=0x25 seq=0x01e4)
Tello: 15:40:29.565:  Info: quit
Tello: 15:40:29.565:  Info: state transit State::connecting -> State::quit
height= 0, fly_mode=0x01, battery_percentage=66, drone_battery_left=0x0000
Tello: 15:40:29.598:  Info: exit from the recv thread.
Tello: 15:40:30.565:  Info: exit from the video thread.
Tello: 15:40:36.480:  Info: counter_clockwise(val=10)
Traceback (most recent call last):
  File "keyboard_test_2.py", line 24, in recv_thread
    container = av.open(drone.get_video_stream())
  File "av/container/core.pyx", line 249, in av.container.core.open
  File "av/container/core.pyx", line 210, in av.container.core.Container.__cinit__
  File "av/container/core.pyx", line 111, in av.container.core.ContainerProxy.__init__
  File "av/container/core.pyx", line 182, in av.container.core.ContainerProxy.err_check
  File "av/utils.pyx", line 103, in av.utils.err_check
av.AVError: [Errno 1094995529] Invalid data found when processing input: 'None'
[Errno 1094995529] Invalid data found when processing input: 'None'

Sorry about that, here it is the output.

[hanyazou]

Could you try to insert "drone.wait_for_connection(60.0)" after "drone.connect()" ?

[HoYinChung]

Tried, the result is the same

[hanyazou]

Please share the output.

[HoYinChung]

Tello: 16:03:02.763:  Info: start video thread
Tello: 16:03:02.763:  Info: send connection request (cmd="conn_req:9617")
Tello: 16:03:02.763:  Info: video receive buffer size = 524288
Tello: 16:03:02.763:  Info: state transit State::disconnected -> State::connecting
Tello: 16:03:02.807:  Info: connected. (port=9617)
Tello: 16:03:02.807:  Info: send_time (cmd=0x46 seq=0x01e4)
Tello: 16:03:02.807:  Info: state transit State::connecting -> State::connected
Tello: 16:03:02.807:  Info: start video (cmd=0x25 seq=0x01e4)
Start Running
start recv_thread()
Tello: 16:03:02.808:  Info: get video stream
start controller_thread()
Tello: 16:03:02.809:  Info: start video (cmd=0x25 seq=0x01e4)
name 'frame' is not defined
Tello: 16:03:02.809:  Info: quit
Tello: 16:03:02.809:  Info: state transit State::connected -> State::quit
<class 'tellopy._internal.video_stream.VideoStream'>.handle_event(DISCONNECTED)
Traceback (most recent call last):
  File "keyboard_test_2.py", line 24, in recv_thread
    container = av.open(drone.get_video_stream())
  File "av/container/core.pyx", line 249, in av.container.core.open
  File "av/container/core.pyx", line 210, in av.container.core.Container.__cinit__
  File "av/container/core.pyx", line 111, in av.container.core.ContainerProxy.__init__
  File "av/container/core.pyx", line 182, in av.container.core.ContainerProxy.err_check
  File "av/utils.pyx", line 103, in av.utils.err_check
av.AVError: [Errno 1094995529] Invalid data found when processing input: 'None'
[Errno 1094995529] Invalid data found when processing input: 'None'
Tello: 16:03:02.858:  Info: recv: ack: cmd=0x34 seq=0x0000 cc 60 00 27 90 34 00 00 00 00 72 a5
Tello: 16:03:02.858:  Info: exit from the recv thread.
Tello: 16:03:03.767:  Info: exit from the video thread.
Tello: 16:03:14.373:  Info: clockwise(val=10)

[hanyazou]

I modified your code slightly because I can't run that on my mac.

import time
import sys
import tellopy
import keyboard
import pygame
import cv2
import numpy
import av
import threading
import traceback

from pygame.locals import *

prev_flight_data = None
run_controller_thread = True

def controller_thread():
    global drone
    global run_controller_thread
    print('start controller_thread()')
    try:
        while run_controller_thread:
            time.sleep(.200)
            # takeoff                                                                                                                                                                          
            if keyboard.is_pressed('space'):
                drone.takeoff()
            # land                                                                                                                                                                             
            if keyboard.is_pressed('l'):
                drone.land()
            if keyboard.is_pressed('esc'):
                drone.land()
                break
    except KeyboardInterrupt as e:
        print(e)
    except Exception as e:
        exc_type, exc_value, exc_traceback = sys.exc_info()
        traceback.print_exception(exc_type, exc_value, exc_traceback)
        print(e)
    finally:
        run_controller_thread = False

def handler(event, sender, data, **args):
    global prev_flight_data
    drone = sender
    if event is drone.EVENT_FLIGHT_DATA:
        if prev_flight_data != str(data):
            print(data)
            prev_flight_data = str(data)
    else:
        print('event="%s" data=%s' % (event.getname(), str(data)))

def main():
    global drone
    drone = tellopy.Tello()
    drone.connect()
    drone.wait_for_connection(60.0)
    drone.start_video()
    drone.subscribe(drone.EVENT_FLIGHT_DATA, handler)
    # drone.subscribe(drone.EVENT_VIDEO_FRAME,handler)                                                                                                                                         
    print("Start Running")
    try:
        # threading.Thread(target=recv_thread).start()                                                                                                                                         
        threading.Thread(target=controller_thread).start()
        container = av.open(drone.get_video_stream())
        frame_count = 0
        while True:
            for frame in container.decode(video=0):
                frame_count = frame_count + 1
                # skip first 300 frames                                                                                                                                                        
                if frame_count < 300:
                    continue
                image = cv2.cvtColor(numpy.array(frame.to_image()), cv2.COLOR_RGB2BGR)
                cv2.imshow('Original', image)
                cv2.imshow('Canny', cv2.Canny(image, 100, 200))
                cv2.waitKey(1)
    except KeyboardInterrupt as e:
        print(e)
    except Exception as e:
        exc_type, exc_value, exc_traceback = sys.exc_info()
        traceback.print_exception(exc_type, exc_value, exc_traceback)
        print(e)

    drone.quit()
    exit(1)

if __name__ == '__main__':
    main()

I can take off and see the video with this code on my Mac.

pygame 1.9.4
Hello from the pygame community. https://www.pygame.org/contribute.html
Tello: 17:38:42.878:  Info: start video thread
Tello: 17:38:42.878:  Info: send connection request (cmd="conn_req:9617")
Tello: 17:38:42.878:  Info: video receive buffer size = 524288
Tello: 17:38:42.878:  Info: state transit State::disconnected -> State::connecting
Tello: 17:38:44.881:  Info: send connection request (cmd="conn_req:9617")
Tello: 17:38:46.883:  Info: send connection request (cmd="conn_req:9617")
Tello: 17:38:48.888:  Info: send connection request (cmd="conn_req:9617")
Tello: 17:38:48.931:  Info: connected. (port=9617)
Tello: 17:38:48.931:  Info: send_time (cmd=0x46 seq=0x01e4)
Tello: 17:38:48.932:  Info: state transit State::connecting -> State::connected
Tello: 17:38:48.932:  Info: start video (cmd=0x25 seq=0x01e4)
Start Running
start controller_thread()
Tello: 17:38:48.932:  Info: get video stream
Tello: 17:38:48.933:  Info: start video (cmd=0x25 seq=0x01e4)
Tello: 17:38:48.971:  Info: recv: ack: cmd=0x34 seq=0x0000 cc 60 00 27 90 34 00 00 00 00 72 a5
Tello: 17:38:48.975:  Info: recv: ack: cmd=0x20 seq=0x0000 cc 60 00 27 b0 20 00 00 00 00 42 b9
Tello: 17:38:48.978:  Info: recv: ack: cmd=0x34 seq=0x0000 cc 60 00 27 90 34 00 00 00 00 72 a5
Tello: 17:38:48.983:  Info: recv: ack: cmd=0x20 seq=0x0000 cc 60 00 27 b0 20 00 00 00 00 42 b9
Tello: 17:38:48.985:  Info: recv: ack: cmd=0x34 seq=0x0000 cc 60 00 27 90 34 00 00 00 00 72 a5
Tello: 17:38:48.986:  Info: recv: ack: cmd=0x20 seq=0x0000 cc 60 00 27 b0 20 00 00 00 00 42 b9
height= 0, fly_mode=0x01, battery_percentage=94, drone_battery_left=0x0000
2018-09-02 17:38:49.138 Python[49320:6441472] pid(49320)/euid(0) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
Tello: 17:38:51.904:  Info: video data 852705 bytes 414.6KB/sec

But I eventually encounter the "[Errno 1094995529] Invalid data found when processing input: 'None'" with the same source code. I feel that this un-stable error is troublesome. On my Mac this error rarely happens. What is your OS/machine environment? How about the result if you try it over and over?

pygame 1.9.4
Hello from the pygame community. https://www.pygame.org/contribute.html
Tello: 17:37:19.274:  Info: start video thread
Tello: 17:37:19.274:  Info: send connection request (cmd="conn_req:9617")
Tello: 17:37:19.274:  Info: video receive buffer size = 524288
Tello: 17:37:19.274:  Info: state transit State::disconnected -> State::connecting
Tello: 17:37:21.276:  Info: send connection request (cmd="conn_req:9617")
Tello: 17:37:27.317:  Info: connected. (port=9617)
Tello: 17:37:27.317:  Info: send_time (cmd=0x46 seq=0x01e4)
Tello: 17:37:27.317:  Info: state transit State::connecting -> State::connected
Tello: 17:37:27.317:  Info: start video (cmd=0x25 seq=0x01e4)
Start Running
start controller_thread()
Tello: 17:37:27.318:  Info: get video stream
Tello: 17:37:27.318:  Info: start video (cmd=0x25 seq=0x01e4)
Tello: 17:37:27.368:  Info: recv: ack: cmd=0x34 seq=0x0000 cc 60 00 27 90 34 00 00 00 00 72 a5
Tello: 17:37:27.369:  Info: recv: ack: cmd=0x20 seq=0x0000 cc 60 00 27 b0 20 00 00 00 00 42 b9
Tello: 17:37:27.373:  Info: recv: ack: cmd=0x34 seq=0x0000 cc 60 00 27 90 34 00 00 00 00 72 a5
Tello: 17:37:27.373:  Info: recv: ack: cmd=0x20 seq=0x0000 cc 60 00 27 b0 20 00 00 00 00 42 b9
Tello: 17:37:27.374:  Info: recv: ack: cmd=0x34 seq=0x0000 cc 60 00 27 90 34 00 00 00 00 72 a5
Tello: 17:37:27.375:  Info: recv: ack: cmd=0x20 seq=0x0000 cc 60 00 27 b0 20 00 00 00 00 42 b9
height= 0, fly_mode=0x01, battery_percentage=86, drone_battery_left=0x0000
height= 0, fly_mode=0x01, battery_percentage=85, drone_battery_left=0x0000
2018-09-02 17:37:27.519 Python[49289:6439591] pid(49289)/euid(0) is calling TIS/TSM in non-main thread environment, ERROR : This is NOT allowed. Please call TIS/TSM in main thread!!!
Traceback (most recent call last):
  File "test.py", line 82, in main
    container = av.open(drone.get_video_stream())
  File "av/container/core.pyx", line 255, in av.container.core.open
  File "av/container/core.pyx", line 211, in av.container.core.Container.__cinit__
  File "av/container/core.pyx", line 112, in av.container.core.ContainerProxy.__init__
  File "av/container/core.pyx", line 183, in av.container.core.ContainerProxy.err_check
  File "av/utils.pyx", line 105, in av.utils.err_check
av.AVError: [Errno 1094995529] Invalid data found when processing input: 'None'
[Errno 1094995529] Invalid data found when processing input: 'None'
Tello: 17:37:30.259:  Info: quit
Tello: 17:37:30.259:  Info: state transit State::connected -> State::quit
<class 'tellopy._internal.video_stream.VideoStream'>.handle_event(DISCONNECTED)
Tello: 17:37:30.259:  Info: exit from the video thread.
Tello: 17:37:30.279:  Info: exit from the recv thread.

[HoYinChung]

Oh so the code changes is basically putting the video handling in the main loop instead of a thread. Will try

I'm using Windows 10.

I do have that unstable error from time to time as well. I have a feeling that it is about the timing of receive and process (try to decode even the stream not really)

Thank you, will try the code👍👍

[hanyazou]

We know that it will occur both Windows and Mac. So it does not depend on platforms. It must be good news.

[HoYinChung]

Your code works on windows as well :+1: Well, the

av.AVError: [Errno 541478725] End of file
[Errno 541478725] End of file

does occur from time to time and sometimes crashes the connection. and it would occur when i move the drone pretty fast.

Im closing this issue 👍 feel free to put the keyboard control as one of your examples as well:)

[hanyazou]

I deleted the control thread and I just repeated executing my program 20 times. The error occurred 3 time. The error rate was 3/20.

[HoYinChung]

Yeah, it occurs pretty often sometimes. And it is not about the control thread. Even that I used video_effect.py example, if i move quick like shake the drone . The same issue will occur. Could it be the issue of the video stream of the av.open ? Just guess.

[hanyazou]

"[Errno 541478725] End of file" will occur when tellopy have no data from drone and returns zero length data to the libav codec. Tellopy wait the data for 5 seconds before it returns zero length data. So the error indicate tellopy did not receive any data from the drone for 5 seconds. It might be a communication error or the drone might be powered off.

The cause of "[Errno 1094995529] Invalid data found when processing input" is unknown. According to google, the same errors are reported in the pyav community. It might not be a issue of the TelloPy.