starkillerOG / reolink_aio

Reolink NVR/camera API PyPI package
MIT License
68 stars 13 forks source link

FYI: filename encoding documented - full list of flags #72

Open sven337 opened 1 month ago

sven337 commented 1 month ago

Hello, FYI, I looked at the Reolink FW and figured out the encoding of the file names in its entirety. Of interest in particular would be the list of attributes: https://github.com/sven337/ReolinkLinux/wiki/Figuring-out-the-file-names#attributes

starkillerOG commented 4 weeks ago

@sven337 that is very instresting, thanks a lot for sharing! I will defenitly look into this to improve the Reolink itegration. At the moment a bit bussy with some other Reolink imporvements, but will keep this issue open untill I find some time to work on it.

Did you perhaps also look into the file names of the Reolink Home Hub? That uses a even newer file naming standard from which I did not figure out what the flags mean....

sven337 commented 4 weeks ago

Let me know where to find the firmware and show me the filename pattern, and I may be able to figure it out.

starkillerOG commented 3 weeks ago

The firmware is on the reolink download center: https://reolink.com/download-center/?srsltid=AfmBOor3BU5F6iWv9p1mMWavhnPrtjGFN2QigYLOH4eaepdvwtnU-eMS

Device name: Reolink Home Hu Hardware version: BASE_WENNT6NA5

I will give you a filenmae pattern as soon as possible, I don't have my Home Hub plugged in right now and am working on way too many issues at the same time at the moment...

sven337 commented 3 weeks ago

Thanks, I have the firmware bits, and it looks like the relevant functions are indeed different. I need to see the pattern to help with RE. Please tell me everything you know/have and what you're missing, and I should be able to RE that.

As for the flags from the camera, here's how I'm decoding them: https://github.com/ReolinkCameraAPI/reolinkapipy/blob/master/examples/video_review_gui.py#L28 feel free to reuse

starkillerOG commented 2 weeks ago

@sven337 sorry for the delay, this is the filename of a recording from the Reolink Home Hub (I replaced the UID and camera name): "/mnt/sda/<UID>-<NAME>/Mp4Record/2024-08-27/RecM02_DST20240827_090302_090334_0_800_800_033C820000_61B6F0.mp4"

First I thought the last 6 chars would be the tags so I made a list of those tags for diffrent events:

21DCCD - 001000011101110011001101 - pet + person
182721 - 000110000010011100100001 - person
18DF13 - 000110001101111100010011 - motion + person
188FF9 - 000110001000111111111001 - pet
17DD6E - 000101111101110101101110 - pet
156701 - 000101010110011100000001 - motion
1F25E5 - 000111110010010111100101 - motion

But I could not find a logical pattern in this....

starkillerOG commented 2 weeks ago

I will defenitely use the decoding you wrote, it looks way nicer. May take a while untill I get to it though.... Lots of things I want to implement and improve.

starkillerOG commented 2 weeks ago

For a Argus Track (dual lens camera) the recording on the Home Hub has the following names:

RecC02_DST20240623_163915_163926_0_380_200_033C800000_8655E.mp4
RecE02_DST20240623_163915_163925_0_780_438_033C820000_F6362.mp4
RecM02_DST20240623_163915_163924_0_F00_870_033C820000_34B93A.mp4
RecS02_DST20240623_163915_163924_0_380_200_033C800000_7EB2A.mp4

For the sub/main stream of both lenses

sven337 commented 2 weeks ago

Here are my findings: https://github.com/sven337/ReolinkLinux/wiki/Figuring-out-the-file-names#filename-structure-for-reolink-home-hub as far as I can tell and just like for camera, the last hex group is the filesize!

In my link you can find the flags you are looking for:


flags_mapping_v2 = {
        'resolution_index': (32,    7),
        'tv_system': (31,           1),
        'framerate': (24,           7),
        'audio_index': (22,         2),
        'ai_pd': (21,               1),  # person detection
        'ai_fd': (20,               1),  # face detection
        'ai_vd': (19,               1),  # vehicle detection
        'ai_ad' : (18 ,             1),
        'ai_other' : (16,           2), 
        'encoder_type_index': (15,  1),
        'is_schedule_record' : (14, 1),
        'is_motion_record': (13,    1),
        'is_rf_record': (12,        1),
        'is_doorbell_record': (11,  1),
        'picture_layout_index': (4, 7),
        'package_delivered' : (3,   1),
        'package_takenaway': (2,    1),
        'package_event' : (1,       1),
        'upload_flag' : (0,         1),
}```
starkillerOG commented 2 weeks ago

@sven337 thank you very much for figuring this all out, I wrote some initial code here: https://github.com/starkillerOG/reolink_aio/blob/6d22f8edaa2baf6c3f8754027e292e17f5d8fe29/reolink_aio/typings.py#L403-L544

It works on "normal" Reolink cams, but on the Home Hub it does not work. I noticed the hex_flag is always 033C800000 no matter what type of events triggered the recording. This may be a firmware bug in the Home Hub though.... not sure.

sven337 commented 2 weeks ago

As far as I can tell your implementation of flags isn't quite correct: the home hub and cameras use different flags. Version 2 and 3 (I have both on my Trackmix, 3 is after the recent firmware upgrade) on cameras differ only by is_doorbell_press_record. In fact in my table you can track all 3 known versions of flags on cameras.

The table of versions for the hub is different (it's a different namespace, in a way).

Your flag is sometimes 0x33C820000 in your own examples, which IIRC corresponds to ai_other (don't ask me what that is! it's not animal, not face, not vehicle and not person....). But I agree it is strange that they all use the same. The resolution index being the same value despite resolution being obviously different is also a ????, but I am reasonably certain I got this right. (After checking, I am confused and need to figure this out, sorry for the noise)

Can you share more examples of hub filenames please, along with framerates for each video? It seems that resolution index is supposed to be hardcoded to 1, yet my decoder sees 3. I've been over the list of flags in the code and don't see what I would have forgotten, but decoding 0x33C820000 as the following makes more sense:

{'resolution_index': 1,
 'tv_system': 1,
 'framerate': 30,
 'audio_index': 1,

since I'm reasonably certain your videos are not at 60FPS like originally decoded. :) Though even 30 seems like a lot.

FPS info would therefore help me align things. I do not know why you don't get trigger data.

sven337 commented 5 days ago

Note, the cameras have been updated to a new version of the encoding, version 9. It is not the same as the home hub (!!!). Support for the new encoding is here: https://github.com/sven337/reolinkapipy/blob/af748158785774c556d4b799445103421bd757c3/examples/video_review_gui.py#L53

I haven't yet decoded the extra set of flags, they're always zero on my camera anyway so I do not care much. Probably same flags as the home hub (package events).

starkillerOG commented 6 hours ago

@sven337 I have updated the decoding of the recording names in the reolink_aio lib: https://github.com/starkillerOG/reolink_aio/blob/cbcc1ab32832afc049adbdd57ee3cca89cd8a3aa/reolink_aio/typings.py#L430-L563

I think it should now work a bit better.

The most noticible improvement I have made is to reverse the binary hex value and adjust all mappings acordingly. That makes sure the first bit accross all versions is always the resolution_index and so on, newer version just append more stuff at the higher bits.

Therefore the offset does not need to be adjusted between versions anymore.

https://github.com/starkillerOG/reolink_aio/blob/cbcc1ab32832afc049adbdd57ee3cca89cd8a3aa/reolink_aio/typings.py#L465

https://github.com/starkillerOG/reolink_aio/blob/cbcc1ab32832afc049adbdd57ee3cca89cd8a3aa/reolink_aio/typings.py#L471

I am not so good in all the binary operations, so this can proably be done more ellegantly, faster and more compact, but in any case this works and should make the decoding a lot more resilient against future version changes.

sven337 commented 5 hours ago

Nice job, it is better to have flipped things indeed. My version is the product of the reverse engineering but I wasn't exactly focused on doing something nice either, I only wanted something that worked :)