Phantom 4 compatibility

skylow48 commented 7 years ago

Hello,

It seem's to not be compatible with Phantom 4 FlightRecord. How to make it compatible ?

Thanks,

Regards.

lvauvillier commented 7 years ago

FlightRecords are now encrypted by the DJI Go app (since 2.8.4)

More info here

n4ru commented 7 years ago

Was there ever an update on this? Any publicly available way to decode them?

mikeemoo commented 7 years ago

Unfortunately I don't have a drone available at the moment in order to check this out.

If you've got any encrypted log files, feel free to send them over and i'll take a look.

n4ru commented 7 years ago

Here's a couple from the Mavic Pro - encrypted using the same app as the Phantom 4 and Inspire 2.

lvauvillier commented 7 years ago

@mikeemoo i already rewrite some part of the lib to work properly with encrypted dataframes. My only missing piece is the getKey() function which i understand the logic but doesn't work...

I also collect some frames on which the key is easy to reverse (frames containing fields with zero values, which a xor return the key)

Feel free to contact me in private.

tonymorris commented 7 years ago

Hi @lvauvillier I would like to help you with this, but I am unfamiliar with the new DJI format. Any pointers to get going?

lvauvillier commented 7 years ago

@tonymorris

The "detail" frame remains unchanged and raw.
For the others, the logic of parsing the frame type and frame length remains also the same, but there is is two more bytes in the data frame : one at the beginning and one at the end. Each data frames are encrypted with a specific key.
To decode the frame, the 8 bytes key is simply XORed across the data.

The getKey() function is my only missing part. The key is computed from the first byte, a fixed seed and the frame length using a CRC64. I'm not sure for the frame length, but i can say that two different frames with the same first byte may have not the same key. This is my only missing part.

To go further i suggest you to read the two linked posts in my previous reply.

SwissDroneHome commented 7 years ago

@lvauvillier I was planning on going public with my (free) website for DJI logs just when DJI started encrypting logs. I got as far as you did and have observed:

Position of frame within file is not relevant, only the parameters you have mentioned are needed to generate the key, e.g. frames with same message type and length have always the same key
Identical frames have the same additional second byte, but frames of same message type and different data have not. Makes me believe it is some kind of checksum
I've asked one of the guys who managed to decode the encryption in the phantom pilots forum for help, but they won't provide the solution nor did he answer a PN for hints. I personally find that a bit annoying since ot was the same group who asked for public help on the reverse engineering site.
The fact that it did not take them very long to decode the messages as soon as they knew what we know here makes me think it can't be too complicated but I could be wrong.
Happy to support so that we can solve this

skandragon commented 7 years ago

I'm interested in getting the flight data logs decoded as well.

Neilfau commented 7 years ago

Anyone have any luck with this?

abaschen commented 6 years ago

The XOR key is generated for every possible value so you have 4096 possibilities. Which can be obtained by calling the function in the C lib. Hope it helps :)

lvauvillier commented 6 years ago

Now fixed by #15 !

Thanks a lot @TecHunter for the tip !

skylow48 commented 6 years ago

Hi @lvauvillier how to get your #15 release ? (sorry I don't realy know GitHub)

lvauvillier commented 6 years ago

I'm not the owner. I cannot release it on npm. You can download the working project from my repo and add it manually to your project.

skylow48 commented 6 years ago

Great thanks. Did you succeed in decoding a recent encrypted DJI log ? I don't succeed in it with your update.

lvauvillier commented 6 years ago

Can you send me your log file ? I'll give a try.

lvauvillier commented 6 years ago

It works for me. Are you running it within your browser or using nodejs ?

skylow48 commented 6 years ago

Good new thanks :) I'm running it with my browser

lvauvillier commented 6 years ago

@skylow48 Fixed !

skylow48 commented 6 years ago

It works :) thanks a lot

skylow48 commented 6 years ago

Sorry I tested the wrong file... it doesn't work, but I'm not sure to run the right function : I test it from "exemples/browser-example.html" : is it correct ?

lvauvillier commented 6 years ago

Yes it is correct. But the sample use a compiled version of the library (browser.js) and i push only the modified source files.

I just add a build script and an updated browser.js version to my repo.

Everything should work for you now.

skylow48 commented 6 years ago

You are the best, thanks :)

lvauvillier commented 6 years ago

I also update my online visualisation demo with the new parser version : https://flighttrack.now.sh

You can easily check your encrypted flight logs and report bug if it is not working.

skylow48 commented 6 years ago

Very nice! Did you manage to recover the battery SN ? And the pictures ? How to report a bug out of Github (do you have a bug reporting tool ?)

lvauvillier commented 6 years ago

Pictures are my next step. Stay tuned

lvauvillier commented 6 years ago

First Image extraction is working :) You can test it with my repo's code. (code sample in example.js)

@skylow48 you can report bugs opening issues here or on my repo. @mikeemoo are you still working on this project ? If you have no time for it, i propose to help you to maintain this library.

skylow48 commented 6 years ago

Nice ! :) But I don't know how to use node.js, do you have a way to test your code in HTML/javascript or better : in C# ?

lvauvillier commented 6 years ago

I update the browser sample with an image extraction code

skylow48 commented 6 years ago

That works perfectly :) thanks ! Do you have an easy way to convert your script into C# to run it in a winform ?

RogueWarrior65 commented 6 years ago

Hello all, I'm trying to convert this algorithm into C/C++ but I'm clearly missing something. I'd appreciate it if someone would take a look at some data and tell me where I'm going wrong.

I have a log file from a flight that I know starts at the following coordinate: 38.813482 -120.080589. I've tried this file out on https://flighttrack.now.sh/ as well as the TXTlogToCSVTool so I know the file works.

The first 100 bytes of the log look like this: 22 5C 14 00 00 00 00 00 90 01 08 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Then I have what appears to be an OSD frame: 01 37 F6 10 E8 8D 41 8E 7E 7C 90 A5 0A F9 FF CE 17 99 6F 4A 34 42 E7 BD BA 7C 50 A1 CB 5F E7 6B BB 7A 59 51 E4 D6 E7 B2 B3 7C 50 68 34 42 E7 BC A4 F6 5F 41 34 42 E7 BD 4D

The frame type is 1, the length is 0x37 or 55 bytes. The key byte is 0xF6 or 246. Since this is a Type 1 frame, key = keys[((tId - 1) * 256) + byteKey] should give me row 246 of the table: 0x4a | 0x34 | 0x42 | 0xe7 | 0xbd | 0xba | 0x7c | 0x50

The actual data is supposed to begin with the 0x10 byte which is at position 103. The first field of the frame is supposed to be the longitude in radians and it's a double (8 bytes). In the line: var decodedByte = tmpBuffer.readUint8(i) ^ this.key[(offset + i) % 8]; i would be zero because it's the first byte. offset would be 103. Modulus 8 gives 7. So the key value should be 0x50. XORing that with 0x10 gives 0x40.
But that's not anywhere close to what a IEEE-754 64-bit floating point conversion should be.

I'm not able to decode anything. So, where did I go wrong?

RogueWarrior65 commented 6 years ago

Okay, I figured it out. The offset variable is the offset into the payload of the frame itself. Also, 64-bit floats are stored in memory in reverse byte order.

MikeBenza commented 6 years ago

I'm trying to port this to phantom-decoder, but I don't understand how key selection works. Some frame types are > 16, but keys[((tId - 1) * 256) + byteKey] evaluates to > 4096 and there are only 4096 keys. Does anyone have a better understanding of this?

lvauvillier commented 6 years ago

@MikeBenza We don't have model and mapping for frames having a tId > 16. Thats why we don't decode more than 4096 keys. Just ignore them to make your code working as we do in the switch instruction of index.js.

rsfinlayson commented 6 years ago

FYI, I have developed an Open Source parser for the newest versions of DJI's 'TXT' log file. You can find this online (along with a developers' mailing list) at: http://djilogs.live555.com/ Your "dji-log-parser" application was a big help in developing this.

Like the previous posters, I'm trying to figure out the values of the 'scramble table' (i.e., the table whose rows are XORed with the encrypted record payload to decrypt it) beyond row 0x1000 (i.e., 4096). I'm curious, though: How did you figure out the first 0x1000 (4096) rows of this table to begin with? The DJI Go 4 app itself doesn't seem to store this table; instead, it seems to generate each row programmatically - e.g., using some sort of hash? But somehow, you managed to generate the first 0x1000 rows of the table. How? And could the remaining rows of the table be generated the same way??

lvauvillier commented 6 years ago

Hi, nothing magical here. The table was generated using the dji decrypt lib contained in the android app. Create an empty project, try to link the decrypt.so lib, loop and call the getkey() function.

I think you could get all the next keys you need using this method.

Le mer. 15 août 2018 à 11:24, rsfinlayson notifications@github.com a écrit :

FYI, I have developed an Open Source parser for the newest versions of DJI's 'TXT' log file. You can find this online (along with a developers' mailing list) at: http://djilogs.live555.com/ Your "dji-log-parser" application was a big help in developing this.

Like the previous posters, I'm trying to figure out the values of the 'scramble table' (i.e., the table whose rows are XORed with the encrypted record payload to decrypt it) beyond row 0x1000 (i.e., 4096). I'm curious, though: How did you figure out the first 0x1000 (4096) rows of this table to begin with? The DJI Go 4 app itself doesn't seem to store this table; instead, it seems to generate each row programmatically - e.g., using some sort of hash? But somehow, you managed to generate the first 0x1000 rows of the table. How? And could the remaining rows of the table be generated the same way??

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mikeemoo/dji-log-parser/issues/14#issuecomment-413082957, or mute the thread https://github.com/notifications/unsubscribe-auth/AAIO_SPuErw-z2uUmfTT5pKw3DjHAquLks5uQ5RTgaJpZM4Lq4kB .

susdru5 commented 5 years ago

Did we ever get this working for new models? Having trouble

rsfinlayson commented 5 years ago

Did we ever get this working for new models? Having trouble

I don't think this software is being updated anymore. Instead, you could try the newer software that I document here: http://djilogs.live555.com/ If you have any questions about that software, you'll need to subscribe and post to the mailing list that's described at the link above; I won't be conducting conversations on random 3rd-party web sites like GitHub.

simonwiles commented 5 years ago

http://djilogs.live555.com/ seems to be dead, and the mailing list likewise. Is this a temporary situation, or has something gone more seriously wrong?

rsfinlayson commented 5 years ago

On Mar 28, 2019, at 6:44 PM, Simon Wiles notifications@github.com wrote:

http://djilogs.live555.com/ seems to be dead, and the mailing list likewise. Is this a temporary situation, or has something gone more seriously wrong?

It’s a temporary situation, but won’t get fixed until mid April. In the meantime, we have an archived copy of the documentation online at http://djilogs.live555.com/ (which is up once again)

Ross Finlayson Live Networks, Inc. http://www.live555.com/

cutec-chris commented 4 years ago

Hi, nothing magical here. The table was generated using the dji decrypt lib contained in the android app. Create an empty project, try to link the decrypt.so lib, loop and call the getkey() function. I think you could get all the next keys you need using this method. Le mer. 15 août 2018 à 11:24, rsfinlayson notifications@github.com a écrit : … FYI, I have developed an Open Source parser for the newest versions of DJI's 'TXT' log file. You can find this online (along with a developers' mailing list) at: http://djilogs.live555.com/ Your "dji-log-parser" application was a big help in developing this. Like the previous posters, I'm trying to figure out the values of the 'scramble table' (i.e., the table whose rows are XORed with the encrypted record payload to decrypt it) beyond row 0x1000 (i.e., 4096). I'm curious, though: How did you figure out the first 0x1000 (4096) rows of this table to begin with? The DJI Go 4 app itself doesn't seem to store this table; instead, it seems to generate each row programmatically - e.g., using some sort of hash? But somehow, you managed to generate the first 0x1000 rows of the table. How? And could the remaining rows of the table be generated the same way?? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#14 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AAIO_SPuErw-z2uUmfTT5pKw3DjHAquLks5uQ5RTgaJpZM4Lq4kB .

what decrypt.so ? In what Version of DJI GO App ? In actual DHI GO 4 i dont see any *decrypt.so lib

divyaprabhu15 commented 4 years ago

how to put console output in .csv file?

console.log(obj.getLatitude() ); its displaying latitudes in console i want it in csv file

rsfinlayson commented 4 years ago

On Feb 18, 2020, at 4:54 PM, divyaprabhu15 notifications@github.com wrote:

how to put console output in .csv file?

If you’re referring to the “djiparsetxt” application described here http://djilogs.live555.com/ then support for that application is via the mailing list described here: http://djilogs.live555.com/#mailing-list (I generally don’t have time to engage in one-on-one email conversations outside the mailing list.)

Ross Finlayson Live Networks, Inc. http://www.live555.com/

rsfinlayson commented 4 years ago

On Feb 19, 2020, at 1:19 AM, divyaprabhu15 notifications@github.com wrote:

http://djilogs.live555.com/ http://djilogs.live555.com/doxygen/html/index.html how to execute this?

Once again: Questions about the “djiparsetxt” application should be asked on our mailing list, which is described here: http://djilogs.live555.com/#mailing-list

I WILL NOT answer questions outside the mailing list, nor on third-party web sites (like GitHub).

Ross Finlayson Live Networks, Inc. http://www.live555.com/

mikeemoo / dji-log-parser

Phantom 4 compatibility #14