Why is the sequence number always set to 0?

[samlaf]

The tello wiki says that the sequence number is

Either 0 for some types, or ascending for others

In this code base, it's defined in tello.py as self.pkt_seq_num = 0x01e4 and even printed when logging commands, for example takeoff logs log.info('takeoff (cmd=0x%02x seq=0x%04x)' % (TAKEOFF_CMD, self.pkt_seq_num))

However, two lines below, fixup never actually includes that information: pkt.fixup() and since fixup is defined as (in protocol.py) def fixup(self, seq_num=0): the sequence number that is sent to the actual tello is always 0!

Is there a reason for this? What exactly is this sequence number?

[Walt-H]

Remember that this isn't an official package, and just like the Wiki you provided states, these are just reverse engineering's of the lower-level protocols of the Tello.

This repo is a refactoriztion to Python of the Go version: https://gobot.io/blog/2018/04/20/hello-tello-hacking-drones-with-go/ That version too claims to be a hack based on the reverse engineering of the Tello protocol.

With sequences also being featured in the video frame packets, I believe that it is for use at the firmware level for sequencing when need be. Some drones don't use the sequencing number, so maybe it can be used at the application level for those drones (i.e. the Tello).

[samlaf]

Thanks a lot for this, makes sense. Are you also working on a tello-related project? I want to add comments throughout this repo to help other newcomers, but I've seen even your november pull request hasn't been accepted. Any idea if hanyazou has stopped looking at this repo?

[Walt-H]

@samlaf Not as much anymore (using Mavic Air 2 now), but I've been using this repo for Tello dev as of lately: https://github.com/damiafuentes/DJITelloPy

[samlaf]

Ya I've been working with that repo until now, but I've come to this one because I can't get rid of the 1-2s video latency on DJITelloPy. Did you figure out a way to get rid of it?

[Walt-H]

I've seen this commit that discusses your issues, and has been tested by the project keeper with decreased latency, but it's not a perfect solution: https://github.com/damiafuentes/DJITelloPy/commit/5a44e54e82204afb6a4b1aa8e0d28a12b178e236

[samlaf]

That commit seems to be related to controller input latency. I was talking about video latency. (for example if I move the drone, it takes about 2s for the video to update on my computer screen)

[Walt-H]

Oh, the video lag is a known issue, and the WiFi can be interrupted/impeded by a variety of things. I think it also goes back down to the cost of the hardware components and what not (pretty cheap drone).

Using this repo, I was able to get very minimal delay using ffmpeg for my video streaming, for example: ffmpeg -i udp://0.0.0.0:11111 -f sdl "Tello"

My comment here talks about it here for Windows, and my PR talks about how to install it on Linux (as mplayer): https://github.com/hanyazou/TelloPy/blob/112fec56854a3ac3ede813b3d99de20e4e0bc95f/README.md#linux

(Note: mplayer's underlying technology uses ffmpeg)

Hope this helps!

[samlaf]

Oh gosh yes this does seem to help! I was using ffplay before, which also uses ffmpeg + sdl... I'm really confused now why using sdl as an output device to ffmpeg has lower latency than ffplay. I hope there's a way to reproduce this setting in the code (perhaps by giving flag options to opencv's VideoCapture?)

Also I'm guessing you made a mistake since port 11111 is for the TelloPy library (this one uses 6038 for video transmission)? So your workflow for the other repo would look something like:

#!/usr/bin/python3
tello = tellopy.Tello()
tello.connect()

and

#!/bin/bash
ffmpeg -i udp://0.0.0.0:11111 -f sdl "Tello"

?

[Walt-H]

For the speed up, I think it's because that uses ffmpeg directly, where ffplay is a wrapper to ffmpeg. Also, I think it has to do with the parameters that mplayer uses when it uses ffmpeg (see this thread).

As with the ffmpeg stuff, I was using that line as an example, but Tello users with phone apps typically run that line and stream from that to a player; however, in this API, a lot of stuff happens:

• Send Tello command to record data
• Register event to process the expected data at the Connected port (6038)
  • if events are a video received event, for example, process the packet w/ video data structure
• etc...

Links:

• https://github.com/hanyazou/TelloPy/blob/develop-0.7.0/tellopy/_internal/tello.py#L771
• https://github.com/hanyazou/TelloPy/blob/2e3ff77f87448307d6d2656c91ac80e2fb352193/tellopy/_internal/protocol.py#L259
• https://github.com/hanyazou/TelloPy/blob/2e3ff77f87448307d6d2656c91ac80e2fb352193/tellopy/_internal/video_stream.py#L5

Now look at this function in keyboard-video example:https://github.com/hanyazou/TelloPy/blob/2e3ff77f87448307d6d2656c91ac80e2fb352193/tellopy/examples/keyboard_and_video.py#L43

Popen, which is just a wrapper for the bash popen, creates a stream to an mencoder stream (part of mplayer), where we pass this stream (which is treated like any file) to the internal video event handler. This is the video recorder.

Looking here: https://github.com/hanyazou/TelloPy/blob/2e3ff77f87448307d6d2656c91ac80e2fb352193/tellopy/examples/keyboard_and_video.py#L184

, a user defined video frame handler event is created to pass the processed frames from the recorder, add captioning of flight data to the frame, and then display it to the video player (i.e. mplayer).

That was a lot, but I was also reviewing for myself lol

[samlaf]

Thanks for this breakdown, it was useful! The one thing I still don't understand is why the keyboard-video example uses Popen, as opposed to using pyav like the video-effect example does. It seems awfully inefficient to use IPC mechanisms when av already has c bindings for ffmpeg internally, no?

[Walt-H]

Definitely! With this being an example file, I think not much more than the bare minimum was given.

Also, without having to really code anything, I could use mplayer, mencoder, vlc, ffplay or anything in lieu of the recorder/player currently used, since it's like editing a bash command and what not.

[samlaf]

Makes sense! So I guess now the last part that I need to understand is the difference between the stream being sent by the string-protocol (official sdk) and the byte-protocol (unofficial low-level protocol).

The low-level protocol in this repo streams on port 6038 (though that is configurable). From what I understand the Tello encodes using H264, and then streams the frames by breaking them down into packets (typically 6-7/frame), and then sends packets over udp with datapayload: <1 byte frame> <1 byte packet> <1500 bytes of H264> ffmpeg doesn't understand this protocol, which explains why we need to first read it using the video_stream class (which gets rid of the first 2 bytes) and only then pass it to ffmpeg.

On the other hand, the official string-based protocol streams on port 11111 (not configurable). This stream we can pass directly to ffmpeg using the command you showed above ffmpeg -i udp://0.00.0:11111 -f sdl "stream window" Do we know which protocol is used to stream this? Is it the raw H264 stream without the 2 bytes added by the low-level protocol? If so, why does the low-level protocol need to add those 2 bytes?

All in all, I can't figure out for the life of me why the official sdk is giving me a 2s latency whereas the low-level protocol has almost 0 latency. I'm really starting to wonder whether it's not DJI that is purposefully introducing latency in its official sdk to keep an edge with its official tello app (which uses the low-level protocol!)