robagar
commented
3 years ago Yes, those messages are from the h264 video decoder in opencv. I think you always get them until it reaches a key frame it can decipher
Closed jsolderitsch closed 3 years ago
robagar
commented
3 years ago Yes, those messages are from the h264 video decoder in opencv. I think you always get them until it reaches a key frame it can decipher
jsolderitsch
commented
3 years ago OK, that's not a problem then. What about the code not stopping cleanly and needing a keyboard interrupt?
jsolderitsch
commented
3 years ago I am trying to understand waitkey. Once the flight finishes, I click in the opencv window and I get the spinning beachball cursor. Eventually this stops and then pressing the space bar seems to shut the program down. What is the proper way to close the main event loop?
jsolderitsch
commented
3 years ago And sometimes (maybe every other time I try to run the example, so 50% of the time) the opencv window does not appear at all even though I see a successful streamon in the shell where I am running python:
jjsretina:examples jjs$ python video_opencv.py
Please enter the WiFi network name prefix to look for, eg "RMTT". Defaults to "TELLO">
waiting for WiFi (TELLO)...
CONNECT 192.168.10.1
SEND command
RECEIVED ok
SEND battery?
RECEIVED 56
battery: 56%
SEND streamon
RECEIVED ok
SEND takeoff
RECEIVED ok
SEND cw 360
RECEIVED ok
SEND forward 100
RECEIVED ok
SEND ccw 360
RECEIVED ok
SEND back 100
RECEIVED ok
SEND land
RECEIVED ok
SEND streamoff
RECEIVED ok
DISCONNECT 192.168.10.1
OpenCV: Couldn't read video stream from file "udp://0.0.0.0:11111"
^CTraceback (most recent call last):
File "/Users/jjs/Documents/Villanova/CSC9010-Spr2021/Future/TelloTT/asyncio-fork/tello-asyncio/examples/video_opencv.py", line 48, in <module>
if cv2.waitKey(1) != -1:
KeyboardInterruptIn this case there is no opencv window that if selected will respond to the waitkey event. Keyboard interrupt is the only option.
Also, when there is an opencv video window that appears, it pops up after the flight commands are underway. The Tello is already through the clockwise rotation when the video pops into view that seems to show the clockwise rotation from the beginning.
I did add a few more flight commands to lengthen the video segment that appears.
jsolderitsch
commented
3 years ago Basically the same behaviors happen on linux with Python 3.7.3 -- shutdown is problematic and opencv window does not appear at all sometimes.
jsolderitsch
commented
3 years ago Updated to python 3.9.5 in linux and no change in behaviors as described above.
robagar
commented
3 years ago It's just a minimal proof of concept of controlling the drone in a worker thread with the opencv UI, which has to run in the main.
There may be a better way to show the video without the latency, but I'm not an expert on opencv, nor it has to be said, a massive fan.
jsolderitsch
commented
3 years ago Thanks for the clarification. I guess because all of the other examples "just work" that I expected the same for the video ones.
Maybe some words in the documentation on this example can help set expectations.
jsolderitsch
commented
3 years ago Seems like this code is better. The opencv window remains responsive and a key press is seen. Downside is you need to keep the video stream open in the child thread. Worthy of a PR?
#!/usr/bin/env python3
import asyncio
from threading import Thread
import cv2 # requires python-opencv
from tello_asyncio import Tello, VIDEO_URL
##############################################################################
# drone control in worker thread
def fly():
async def main():
drone = Tello()
try:
await drone.wifi_wait_for_network(prompt=True)
await drone.connect()
await drone.start_video(connect=False)
await asyncio.sleep(10)
await drone.takeoff()
await drone.turn_clockwise(360)
await drone.turn_counterclockwise(360)
await drone.land()
finally:
# await drone.stop_video()
await drone.disconnect()
asyncio.run(main())
# needed for drone.wifi_wait_for_network() in worker thread in Python < 3.8
asyncio.get_child_watcher()
fly_thread = Thread(target=fly, daemon=True)
fly_thread.start()
##############################################################################
# Video capture and GUI in main thread
capture = cv2.VideoCapture(VIDEO_URL)
capture.open(VIDEO_URL)
while True:
grabbed, frame = capture.read()
if grabbed:
cv2.imshow('tello-asyncio', frame)
k = cv2.pollKey()
if k == ord('q'):
break
capture.release()
cv2.destroyAllWindows()It's true, handling the video is not easy to get right. I've made another project tello-asyncio-video to do things like decoding the data and making it viewable, but it not quite ready yet.
Thanks for the suggestions- I've added some comments to the example and will add them in before I push
robagar
commented
3 years ago Finally tested out the tidied up example
I see that the previous issue was closed so I am opening a new one.
I fetched the example code change and tried it.
I see:
The code hangs at the end and I used Ctrl-C to interrupt. The opencv window does popup and shows the view despite the h264 errors. Maybe they are normal?
I updated my Mac to Python 3.9.5 last night and just tried the example today. Interestingly I still got the address in use error with the original code and my new 3.9.5 install and that disappeared when I updated the example.