RosettaDrone / rosettadrone

MAVlink and H.264 Video for DJI drones
BSD 3-Clause "New" or "Revised" License
298 stars 112 forks source link

Mavic mini2 compability? #31

Closed harshadms closed 1 year ago

m4xw commented 3 years ago

Just finished my PoC.

needs to run with frida as gadget (added to jnilibs and loadlibrary on the connect screen, so it works without root) Confirmed its all working tho DJI SDK Codec Manager has issues on my device, unrelated to the patch. Use it on your own risk.

Fwiw made a little KML parser that pushes waypoints from groundstation export via do_set_motion_absolute :) Works sweet!

I plan to release the scripts after a bit more ironing out.

@The1only Since DJI has no plan to support it in SDK v4, would you be interested in adding my script? It would have quite big deps tho (20MB per arch, frida's libgadget).

Maybe we can figure something, or is it too hot for you? Also would you be interested in adding my KML stuff (tho i plan to do a translation layer between WaypointMission stuff later).

Also fixed a bunch of smaller bugs related to GCS taking images etc. when f.e wrong mode is selected

The1only commented 3 years ago

Great work,

Send push requests and I will diff and add. Some times I rework it a bit to fitt the bigger picture and all other stuff thats beeing added, we too have been working on image taking, but mostly mission work lately.

If you go beyond Rosettadrone strategy I simply reject the push, no problem.

Best regard Terje Nilsen 9Tek AS Norway

On May 6, 2021, at 21:21, m4xw @.***> wrote:

 Just finished my PoC.

needs to run with frida as gadget (added to jnilibs and loadlibrary on the connect screen, so it works without root) Confirmed its all working tho DJI SDK Codec Manager has issues on my device, unrelated to the patch. Use it on your own risk.

Fwiw made a little KML parser that pushes waypoints from groundstation export via do_set_motion_absolute :) Works sweet!

I plan to release the scripts after a bit more ironing out.

@The1only Since DJI has no plan to support it in SDK v4, would you be interested in adding my script? It would have quite big deps tho (20MB per arch, frida's libgadget).

Maybe we can figure something, or is it too hot for you? Also would you be interested in adding my KML stuff (tho i plan to do a translation layer between WaypointMission stuff later).

Also fixed a bunch of smaller bugs related to GCS taking images etc. when f.e wrong mode is selected

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

The1only commented 3 years ago

Ps: Why would you need Frida??

Best regard Terje Nilsen 9Tek AS Norway

On May 6, 2021, at 21:32, Terje Nilsen @.***> wrote:

Great work,

Send push requests and I will diff and add. Some times I rework it a bit to fitt the bigger picture and all other stuff thats beeing added, we too have been working on image taking, but mostly mission work lately.

If you go beyond Rosettadrone strategy I simply reject the push, no problem.

Best regard Terje Nilsen 9Tek AS Norway

On May 6, 2021, at 21:21, m4xw @.***> wrote:

 Just finished my PoC.

needs to run with frida as gadget (added to jnilibs and loadlibrary on the connect screen, so it works without root) Confirmed its all working tho DJI SDK Codec Manager has issues on my device, unrelated to the patch. Use it on your own risk.

Fwiw made a little KML parser that pushes waypoints from groundstation export via do_set_motion_absolute :) Works sweet!

I plan to release the scripts after a bit more ironing out.

@The1only Since DJI has no plan to support it in SDK v4, would you be interested in adding my script? It would have quite big deps tho (20MB per arch, frida's libgadget).

Maybe we can figure something, or is it too hot for you? Also would you be interested in adding my KML stuff (tho i plan to do a translation layer between WaypointMission stuff later).

Also fixed a bunch of smaller bugs related to GCS taking images etc. when f.e wrong mode is selected

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

m4xw commented 3 years ago

@The1only Frida is required because we need to hook DJI SDK internals and tell it its a Mavic Air 2 with a Mavic Mini Camera (Mavic Air 2 as Camera works, actually even better as you dont need to set jpg+raw for example in the DJI fly app first since UX sdk wont allow u to set it, but the UXSDK fails to detect the SD with it so you can't take pics/vid through it, its working via the API calls tho). Its the only stuff really required. This is the runtime method, otherwise we would require a pre-patched SDK which is troublesome because its loaded in-memory from DJI's libDexHelper because stupid SecNeo obfuscator.

I will probably still need a few weeks to get all properly sorted and tested to make sure theres no additional risk involved, but so far its working splendid to be honest, my dayjob is keeping me busy tho. Fwiw if DJI ever adds support for Mini 2, we could simply drop the hooks (pretty much remove 2 lines and the jniLib include) Whats primarily left for me is having it not impact other drone models (thats quite a easy fix tho)

Would you prefer stuff like a "execute/load KML mission" as a Menu entry like the waypoint stuff or directly on screen? Right now I just hijack the button on the left that opens the store with a dummy. I could make this a seperate PR as well, tho I can only test on Mini 2.

m4xw commented 3 years ago

Note tho, the KML stuff is just a stopgap until i got translation layer for the waypoint code in, I want to avoid having to rewrite all the mavlink stuff for it.

The1only commented 3 years ago

Impressive,

That’s what I would call hacking, but in a positive sens. Keep working and we will be first to support Mini2.

Best regard Terje Nilsen 9Tek AS Norway

On May 6, 2021, at 21:47, m4xw @.***> wrote:

 Note tho, the KML stuff is just a stopgap until i got translation layer for the waypoint code in, I want to avoid having to rewrite all the mavlink stuff for it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

m4xw commented 3 years ago

Impressive, That’s what I would call hacking, but in a positive sens. Keep working and we will be first to support Mini2. Best regard Terje Nilsen 9Tek AS Norway On May 6, 2021, at 21:47, m4xw @.***> wrote:  Note tho, the KML stuff is just a stopgap until i got translation layer for the waypoint code in, I want to avoid having to rewrite all the mavlink stuff for it. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

It is, by all means ;) I have a strong background in both PC and Console hacking as well professional pentesting, so this is a small feat for me. The most time i wasted was with the Video feed but that turned out to work on my S6 (vs not working on S21). Exynos video decoder shits itself, I wonder if a MediaDecoder impl would work tho. They refactored all that in the DJI SDK v5 dumped from the dji.go.v5 app and CodecManager doesnt actually exist anymore (and that works on my S21).

I am happy to hear you are not strictly against it, I prefer to avoid forks whenever possible.

Also thank you for developing such a nice base, most mistakes/design oversights I found are pretty trivial (like the wrong ack in takePhoto for groundstation, i guess it has a legacy background tho?) and not having a photoTaken event to sync up the success with a worker thread (but that one i guess would be slightly out of scope for base rosettadrone). I had it take small movements with a smaller dist margin & lower detection in in the MoveTo task and it would end up failing to queue all the takephotos up. Oh and also not switching to RECORD_VIDEO mode if currently in photo mode or whatever when mavlink stuff comes in. Easy to work with ;)

m4xw commented 3 years ago

Spent some time further improving the goto algo, now syncs within a few seconds to a waypoint if close enough by (we probably want varying ranges of accuracy for either goto from gcs and different drone models) but I think the PID Impl neends some more love in general, so will look into that next. That run is with advancedVirtStick and PID reset (strong wind, 5-10m/s) between waypoints: image https://app.airdata.com/share/fRtJPA/GENERALOverview

The spots itself are on-point, the travel paths need some work. Go Home line is perfectly smooth. Its also possible to use the go-home mechanic and feed it updated home points all the time and have it fly smooth that way, if it werent for the RC beeps.... Also got it to go close to sport mode speed wise. Yaw error on mini 2 seems to be <1 across runs too, which is nice. advancedVirtStick really helps a lot when hovering to take pics

m4xw commented 3 years ago

I also noticed significant improvement by decreasing the task scheduler down to 25ms for the GPS calculations and lower the amount the send-virtstick-data task runs when it came to lock onto coordinates without 5-7s delay and overshooting the destination

kripper commented 1 year ago

@m4xw please check latest version and stay up to date. DJI MINI 2 is now officially suported.