How does the MATLAB parrot package retrieve it's sensor data?

[BraytonL]

I apologize if this isn't the right place for a post like this- if it is an issue and you know where might be better suited, please let me know. Thanks!

As a part of my university research I am designing a quadcopter flight controller- to learn the basics and then hopefully pursue autonomous flight. Unfortunately as I understand the Parrot SDK does not support sending the required IMU data, such as raw gyroscope angles, much less at any rate useful to a controller (2 Hz). This is not to say it is not useful, I realize a ground station closed feedback loop flight controller is not in any way practical. I really appreciate the work done to develop the Python interface, it was very easy to use and understand. However, I am still faced with this dilemma: how do I access the needed flight data?

I found a MATLAB / Simulink parrot support package (link at bottom) that seems to work well but this is my first time using the software and its not nearly as comprehensive as the Python or C library. However, their custom parrot firmware allows for what seems to be "realtime" speed and orientation data from the IMU (plus extras - see photo). Does anyone know how they're capable of this? According to the Parrot Devs sending this much data would at a high rate is not possible with the current drones.

I understand there is probably not a Python or C interface that has the same access as the one developed for Simulink / MATLAB but does anyone know of a way to actually write and upload custom firmware to say the Mambo? This way the IMU data could be accessed locally by the flight controller.

Also, since I am using the Mambo am I limited by anything? Does the Bebop have more available raw sensor data? Unfortunately it might not be in the budget but I am curious nonetheless.

Thank you for taking the time to read and answer, and for all the work putting together this fantastic Python interface. Without all these open source projects education would be much more difficult.

EDIT: MATLAB Package URL: https://www.mathworks.com/hardware-support/parrot-minidrones.html

[amymcgovern]

These are great questions and I can answer some of them but not all. I am not familiar with the Matlab package so I can't do much with it. But the answer to your other questions is: Parrot limits what it sends out on the Mambo to 2Hz and on the Bebop to 10Hz. If one could get into the firmware, one could presumably find the sensor information at a much better rate! I have deliberately not hacked the firmware because I was writing for K-12 schools as the primary audience and I wanted them to be able to use the COTS drone as it is. But I imagine one can do it and likely there are instructions on the dev board for parrot. Those are sparsely answered these days (parrot has fewer staff due to layoffs, or so I guess from news reports, I don't work for them!) but there is a lot of information in older posts there.

[BraytonL]

These are great questions and I can answer some of them but not all. I am not familiar with the Matlab package so I can't do much with it. But the answer to your other questions is: Parrot limits what it sends out on the Mambo to 2Hz and on the Bebop to 10Hz. If one could get into the firmware, one could presumably find the sensor information at a much better rate! I have deliberately not hacked the firmware because I was writing for K-12 schools as the primary audience and I wanted them to be able to use the COTS drone as it is. But I imagine one can do it and likely there are instructions on the dev board for parrot. Those are sparsely answered these days (parrot has fewer staff due to layoffs, or so I guess from news reports, I don't work for them!) but there is a lot of information in older posts there.

Thank you for the reply- I appreciate the help. It seems like there's always more questions to be answered... so. any feedback you can give me would be awesome. Thank you for your time!

Since the IMU data is not sent out directly, do you have any suggestions on how to create a closed feedback loop for movement? Also is there a place online where I can see more of the projects you've implemented this in? I'm curious to see the potential.

[amymcgovern]

By closed loop, do you mean PID? If so, it is pretty hard at 2Hz and only slightly better at 10Hz. I have done closed loop control using the vision system using PID but the vision runs around 30Hz (more practically around 20 just based on processing power).

As for sample code, my largest projects have been as part of a competition so I haven't released that code. My students also used the library for their AI projects and I could upload the assignments (although not the solutions).

[BraytonL]

By closed loop, do you mean PID? If so, it is pretty hard at 2Hz and only slightly better at 10Hz. I have done closed loop control using the vision system using PID but the vision runs around 30Hz (more practically around 20 just based on processing power).

As for sample code, my largest projects have been as part of a competition so I haven't released that code. My students also used the library for their AI projects and I could upload the assignments (although not the solutions).

Not necessarily PID, just simple spacial awareness, inputting a command to move 10m and moving that amount. I realize over time the error would stack over time though. With vision have you implemented that using ROS at all? I'd love to see the assignments, if you get around to uploading them sometime.

[amymcgovern]

The drift ends up being pretty significant. We definitely did a move_meters command (it was assignment one for them) and you can do it but it ends up off, especially if you are flying faster.

I uploaded the assignments to the coursework subdirectory.

[BraytonL]

Thank you. I've read through them and it all looks like a very interesting challenge.

[BraytonL]

assignment one

I know you are unable to give me the solutions, but I was curious as to how you might approach the problem in assignment one. Your course is on artificial intelligence but developing a coordinate system would require the use of quaternions or Euler angles correct?

[amymcgovern]

There is a quaternion in the mambo already implemented.

https://pyparrot.readthedocs.io/en/latest/minidronecommands.html#mambo-sensors

Also, forgot to mention, here is an ML implementation with pyparrot.

https://github.com/TanerDavis/Gesture-Images-For-Training