opendroneid / receiver-android

Example Android receiver application for unmanned aircraft Remote ID
Apache License 2.0
185 stars 61 forks source link

Crashing every time it detects Mavic 3 Enterprise #99

Open johnnokomis opened 1 year ago

johnnokomis commented 1 year ago

Testing with no drones in the air it's fine. As soon as the app detects my M3E the app crashes. Using LG V60.

Log- session,timestamp (nanos),transportType,macAddress,msgVersion,rssi,payload,idType,uaType,uasId,idType,uaType,uasId,status,heightType,EWDirection,speedMult,direction,speedHori,speedVert,droneLat,droneLon,altitudePressure,altitudeGeodetic,height,horizontalAccuracy,verticalAccuracy,baroAccuracy,speedAccuracy,timestamp,timeAccuracy,distance,descriptionType,operationDescription,operatorLocationType,classificationType,operatorLatitude,operatorLongitude,areaCount,areaRadius,areaCeiling,areaFloor,category,classValue,operatorAltitudeGeo,systemTimestamp,operatorIdType,operatorId,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData, 0,281646140031907,Beacon,60:60:1f:02:d4:f9,2,-65,F1 F2 19 03 02 12 31 35 38 31 46 35 46 48 42 32 32 39 46 30 30 32 30 32 44 52 00 00 00 12 22 63 15 00 7C BA 74 14 54 FC 70 CC 00 00 8F 0A 42 08 3A 02 8B 3D 0A 00 42 01 42 B7 74 14 F7 E0 70 CC 01 00 00 00 00 00 00 00 20 0A F7 C1 99 07 00 00 00 00 00 ,1,2,1581F5FHB229F00202DR,,,,2,0,1,0,99,21,0,343194236,-865010604,0,2703,2114,10,3,0,2,15755,10,66.72754,,,1,0,343193410,-865017609,1,0,0,0,0,0,2592,127517175,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, 0,281649402536177,Beacon,60:60:1f:02:d4:f9,2,-67,00 F2 19 03 02 12 31 35 38 31 46 35 46 48 42 32 32 39 46 30 30 32 30 32 44 52 00 00 00 12 22 63 1F 00 AD BB 74 14 17 F3 70 CC 00 00 8F 0A 42 08 3A 02 AB 3D 0A 00 42 01 3D B7 74 14 A2 E0 70 CC 01 00 00 00 00 00 00 00 20 0A FA C1 99 07 00 00 00 00 00 ,1,2,1581F5FHB229F00202DR,,,,2,0,1,0,99,31,0,343194541,-865012969,0,2703,2114,10,3,0,2,15787,10,45.518757,,,1,0,343193405,-865017694,1,0,0,0,0,0,2592,127517178,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

sxjack commented 1 year ago

Does it work with the Dronetag scanner app?

friissoren commented 1 year ago

Hmm.... That is the csv log file, which only stores the data that the app receives. If it happens to crash before it gets to the part where it dumps the data to the csv file, it is hard to say anything specific.

I guess there is no way to get the actual application logs out for you. You would need to connect via USB and enable adb tracing.

I unfortunately don't have a Mavic 3 Enterprise drone to try with.

I looked in the Google Play Console, which gives some information about crashes that have been logged for the app, but I did not see any related to an LG V60 phone. I don't know how reliable that crash logging mechanism is or how often it gets updated.

Steve's question about trying the Dronetag app is a good one. Probably this is just a bug somewhere in the OpenDroneID app and you will be able to pick up the remote ID signals with the Dronetag app.

sxjack commented 1 year ago

That is the csv log file, which only stores the data that the app receives.

I've had a look at those messages, they seem fine, but, as you say, it's one of the messages after them that crashed it.

sxjack commented 1 year ago

Does it work with the Dronetag scanner app?

The link you want is this one https://play.google.com/store/apps/details?id=cz.dronetag.dronescanner .

Tissy commented 1 year ago

Why is authType | authDataPage | authLastPageIndex | authLength | authTimestamp | authData | repeated 16 times?

Is that what is being broadcast from the UAS or preconfigured in the app?

friissoren commented 1 year ago

Is that what is being broadcast from the UAS or preconfigured in the app?

That is just the header information that the app writes to the csv file. There can be up to 16 authentication messages (according to the standard) so there is room reserved for the worst case.

johnnokomis commented 1 year ago

I just tried it out again today and for a short period my M3E was picked up fine. When I say brief period it was a single time. At first opening of the app it said 0 drones. It took it a minute or 2 to ever see it. Once it did I closed the app and re opened it but the drone was never seen again. I was hovering just 50' away but no sign of it on the app.

I then opened the Dronetag app that was suggested previously. It never saw it even once. It leads me to wonder... Is Android just not capable on a hardware level to be a RID reciever? Or is the FAA rules, guidelines and protocol just this garbage? They'll never enforce any of this RID nonsense. They don't have the manpower or the interest to do anything about it. The manufacturers are the ones responsible for implementing this useless RID. The pilots aren't required by law to do anything different. Crazy.

johnnokomis commented 1 year ago

Oh and the log output is basically just as useless (no offense). The successful attempt today looks identical to the app crashing one in the original post.

lukasbrchl commented 1 year ago

@johnnokomis Dronetag app currently doesn't support reading Wi-Fi Remote IDs from DJI drones. That's why we introduced Drone Scanner. In this video, you can also see a brief explanation why any Wi-Fi RID is poorly seen on Android devices https://www.youtube.com/watch?v=nzV1rJGTyMw

johnnokomis commented 1 year ago

@lukasbrchl Thank you for the video. Even with WiFi scan throttling turned off this is still an issue? Apps like Wigle WiFi scan at very fast rates on all channels. With the exception of Android 9's hard coded throttling. I also don't understand how Bluetooth is being used here since it has a range of about 30 feet in ideal conditions.

lukasbrchl commented 1 year ago

Yes, even with throttling off it is still trouble. The best we have seen is around 1 message per 6 seconds on Wi-Fi RIDs. However, just a limited set of smartphones can achieve that. I don't have many insights regarding the differences between Android versions.

The poor range you mention with Bluetooth is a myth. I understand it comes from using common Bluetooth peripherals, but please note Bluetooth RID is different. Check out this post, where data was received on 2.6km from a drone to a common smartphone at just 6 mW of power. https://www.linkedin.com/feed/update/urn:li:activity:6962664637557346305

sxjack commented 1 year ago

Is Android just not capable on a hardware level to be a RID receiver?

A laptop (or a counter-UAV system) will pick it up and a message every 10 seconds on a phone is fine for getting the serial number.

The pilots aren't required by law to do anything different.

As I understand the US regulations, they are from September unless they are flying in an exempt area.

sxjack commented 1 year ago

Check out this post, where data was received on 2.6km from a drone to a common smartphone at just 6 mW of power.

That is impressive. What antenna do you have on the transmitter?

lukasbrchl commented 1 year ago

We do use ceramic SMD antennas on all our products from multiple vendors

johnnokomis commented 9 months ago

Same drone, same phone, same problem. Instant crash.

session,timestamp (nanos),transportType,macAddress,msgVersion,rssi,payload,idType,uaType,uasId,idType,uaType,uasId,status,heightType,EWDirection,speedMult,direction,speedHori,speedVert,droneLat,droneLon,altitudePressure,altitudeGeodetic,height,horizontalAccuracy,verticalAccuracy,baroAccuracy,speedAccuracy,timestamp,timeAccuracy,distance,descriptionType,operationDescription,operatorLocationType,classificationType,operatorLatitude,operatorLongitude,areaCount,areaRadius,areaCeiling,areaFloor,category,classValue,operatorAltitudeGeo,systemTimestamp,operatorIdType,operatorId,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData,authType,authDataPage,authLastPageIndex,authLength,authTimestamp,authData, 0,93968086029466,Beacon,60:60:1f:02:d4:f9,2,-82,C8 F2 19 03 02 12 31 35 38 31 46 35 46 48 42 32 32 39 46 30 30 32 30 32 44 52 00 00 00 12 22 00 04 00 91 2E 76 14 1D A4 70 CC 00 00 6C 0B FB 08 3A 02 CF 47 0A 00 42 01 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 3E 35 E6 08 00 00 00 00 00 ,1,2,1581F5FHB229F00202DR,,,,2,0,1,0,0,4,0,343289489,-865033187,0,2924,2299,10,3,0,2,18383,10,0.0,,,1,0,0,0,1,0,0,0,0,0,0,149304638,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

janusw commented 4 months ago

Same drone, same phone, same problem.

That does not help much. It would be more exciting to know if you're seeing the problem also with other DJI drones, other phones or other receiver apps.

Also you fail to mention which ODID version you are using. Have you tried the latest v3.5?

I cannot test with a Mavic 3, but I recently observed a DJI M30T with ODID 3.5.1 (and other apps), and I did not see any crashes.

If all else fails, you could try to build the ODID app yourself and run a debugging session to see where the crash happens ...

janusw commented 4 months ago

I then opened the Dronetag app that was suggested previously. It never saw it even once. It leads me to wonder... Is Android just not capable on a hardware level to be a RID reciever? Or is the FAA rules, guidelines and protocol just this garbage?

IMHO what is garbage is DJI's transmitter implementation. Other WiFi-Beacon transmitters can be detected much better on Android (e.g. Parrot, DroneBeacon, ...).

I get the impression that DJI does the bare minimum to comply with the standards, while making sure that their drones are as hard to observe as possible (on mobile devices).

One can argue that it's a weakness of the protocol standards to allow this sort of avoidance ...

johnnokomis commented 4 months ago

Drone Scanner picks up my M3E maybe 60% of the time. I reported this issue over a year ago with OpenDrone ID. Still to this day the app crashes when it is detected. I fly M300/M350's almost daily I have never once seen either one show up on either app. The fact that the FAA tries to sell this as a full proof protocol is crazy. They say things like, if it shows up on your RID app then it's a safe drone. Basically telling people that when a drone doesn't show up then it's an illegal drone. Well, there will be plenty of drones that don't show up because of this BS.

janusw commented 4 months ago

Drone Scanner picks up my M3E maybe 60% of the time. I reported this issue over a year ago with OpenDrone ID. Still to this day the app crashes when it is detected.

Unfortunately it seems like no one could reproduce your issue so far, so it's not clear where the problem is and how to fix it. I made a couple of suggestions already. Make sure that your drone has the latest firmware and that you're using the latest version of the ODID app. Try it with a different receiver device, in order to exclude the possibility that it's a specific problem with that phone. Perform a debugging session with a self-built ODID app, or use a WiFi sniffer to record the messages.

I fly M300/M350's almost daily I have never once seen either one show up on either app. The fact that the FAA tries to sell this as a full proof protocol is crazy. They say things like, if it shows up on your RID app then it's a safe drone. Basically telling people that when a drone doesn't show up then it's an illegal drone.

That's not quite what they say. In fact drones are required to send out messages at least once a second, and with proper network equipment you should be able to see all of them. But detection of those messages is limited on mobile devices:

Some transmitters provide a good WiFi Beacon reception on Android phones (by reacting to WiFi probe requests IIUC), others (such as DJI's) do not.

Well, there will be plenty of drones that don't show up because of this BS.

Yes, the situation is not perfect. Not all drones will show up under every condition. But it's less 'BS' than you make it sound ;)

johnnokomis commented 4 months ago

I'm on the latest firmware even though the M3E has had RID enabled since it released.

That's not quite what they say.

It's not? Here is a quote from the FAA when describing RID: "To demonstrate the use of applications, phone apps or other technology to allow the general public to understand what the drones that they see in their environment, whether or not they're allowed to be there to assure them that it's not a nefarious activity that they need to be concerned about." Source: https://youtu.be/jR3KOUa-Jqo?si=RdIQidau3t9Q4fS1&t=235

with proper network equipment you should be able to see all of them.

Is Karen out in public going to have proper network equipment? This is my whole point here. The FAA is informing people that it's as simple as pulling out any phone when they see a drone to check if it's safe or not. The moment their crap phone doesn't show the drone on it, Karen is dialing 911 to report illegal activity. Or better yet when little Timmy is out flying his brand new Mini that he just got, which isn't required to have RID, and Karen doesn't see it on her app. In her mind that must mean it's flying illegally.

the situation is not perfect. Not all drones will show up under every condition.

Correct. I'm not blaming this app's devs for this either so please don't take this as a personal shot. It's not even close to perfect but that's not the way the FAA is explaining it. It is complete BS and I stand by my statement. Not the app, the way the entire implementation has been handled by authorities.

gabrielcox commented 4 months ago

@johnnokomis , Let's please keep this forum focused supporting the source code (not binary) of this (open source) project. There are plenty of other venues to have discussions the merits and implementation of various regulations. Just like many other systems, it will take time for the issues to be worked out and there's a few chicken/egg issues that certainly need to be sorted. There may be particular devices on the send or receive side that have compatibility issues. If you are able to install the Android SDK, load the source, do a build (running on your device) and contribute to debugging by offering Pull Requests. By performing debug builds, Wireshark captures, you'll be able to observe the line of code on which the program crashes. Then bring that data to this forum or offer a change request (or Pull Request) in this forum.

johnnokomis commented 4 months ago

Let's please keep this forum focused

You're right, this isn't the space for expressing my thoughts on the topic. I sincerely apologize if I stepped out of line. I think this project has potential and there aren't many others like it. I wish everyone involved nothing but the best and I'll be actively keeping up with the progress.