search

ZoneMinder / pyzm

Python API, Log, Event Server and Memory wrapper for ZoneMinder
GNU General Public License v2.0
31 stars 20 forks source link

update ZMMemory for 1.36 version of shared memory struct #48

Closed jantman closed 6 months ago

jantman commented 6 months ago

The current version of this package, when used with ZoneMinder 1.36, gives wildly incorrect values for many of the ZMMemory shared data fields. i.e. on a system running ZM 1.36.33 where Monitor 1 has 296 total events and the latest Event ID is 1203:

# python3
>>> from pyzm import ZMMemory
>>> import json
>>> z = ZMMemory.ZMMemory(mid=1)
>>> z.alarm_state()
{'id': 1, 'state': 'STATE_PREALARM'}
>>> print(json.dumps(z.get(), sort_keys=True, indent=4))
{   
    "shared_data": {
        "action": 4050334703,
        "active": true,
        "alarm_cause": "",
        "alarm_x": -1,
        "alarm_y": -1,
        "brightness": 1076098633,
        "color": 0,
        "contrast": 0,
        "control_state": "A\u0611e",
        "epadding1": 4294967295,
        "format": true,
        "hue": 1203,
        "imagesize": 4294967295,
        "last_event": 4621808440055641071,
        "last_read_index": 622028,
        "last_read_time": 4294975296,
        "last_write_index": 4,
        "last_write_time": 8294400,
        "signal": true,
        "size": 760,
        "startup_time": 576743335381696511,
        "state": 1,
        "valid": true
    },
    "trigger_data": {
        "padding": 0,
        "size": 0,
        "trigger_cause": "",
        "trigger_score": 0,
        "trigger_showtext": "",
        "trigger_state": 0,
        "trigger_text": ""
    }
}
>>> z.is_alarmed()
False
>>> z.last_event()
4621818432625040276

I tracked this down to the fact that the shared memory struct format in ZoneMinder::Memory changed in v1.36 but this package was not updated with those changes, resulting in almost all of the fields here in ZMMemory being incorrect.

This PR updates this package to match what's currently in the latest 1.36 tag (1.36.33). However, this does expose a larger issue, as it both breaks compatibility with previous versions and also will not be compatible with the next major release, as there are already changes to the memory map structure in the current ZM master branch. I'm not sure how best to handle this as my experience with memory maps is pretty limited, but the only things I can really think of are to attempt to infer the memory map "version" based on the size of the file (which may or may not be a reliable indication) or to force ZMMemory to also have an API connection, and use that to query the ZoneMinder version.

I'm happy to attempt one of those fixes if given some guidance on what direction you'd like me to take.

baudneo commented 6 months ago

Structs change again in 1.37, this codebase is basically unmaintained. This would require classes with structs defined for version ranges and a mechanism to detect version and apply the proper class of structs. Probably easier to write your own custom lib if you're scraping SHM.

If you're looking for fast event data, DB calls might be better, that's what I do and it's fast. Although I believe the frames table is only updated on bulk frames (configurable, default is 100 IIRC). I haven't noticed any stale event data myself, but YMMV.

 '01/01/24 10:36:44.0136' ML:Client[3500570] DEBUG db:873 -> perf:zmdb:: Grabbing DB info took 0.01701 s ----> Monitor ID: 5 Monitor Nam
  e: Alley Monitor PreEventCount: 75 Monitor PostEventCount: 75 Monitor FPS: 14.980000000000000426325641456060111522674560546875 Event Ca
  use: Motion:Drive Lane Event Notes: Motion: Drive Lane Event Storage Scheme: Medium Event Storage ID: 3 Event StartDateTime: 2024-01-01
   10:36:37 Event Storage Path: /zoneminder Monitor Width: 1080 Monitor Height: 1920 Monitor Color: 4 Monitor ImageBufferCount: 5

perf:zmdb:: Grabbing DB info took 0.01701 s

Edit 2: If you're looking to run a script on motion events, 1.37 has EventStartCommand and EventEndCommand configurable for each monitor. ZM will run a command on event start and/or end, no SHM scanning necessary.

IMO, SHM scraping isn't necessary any more unless you're looking for the image ring buffer, which can also be accessed using zms cgi endpoint and the following params

f"?mode=single&monitor={mid}&connkey={random.randint(100000, 999999)}"

Example: https:://zm.example.com/zm/cgi-bin/nph-zms?mode=single&monitor=1&connkey=879364651

Should give you an image from that monitors live stream.

jantman commented 6 months ago

Thanks for the info, @baudneo.

In this case, I'm using the SHM scraping to monitor the internals of ZM itself. Specifically, the things I'm reading from SHM that I couldn't find elsewhere in any of the APIs and I've found useful in the past are:

Other things that I'm grabbing, just because they're there are maybe could prove useful:

Regarding the structs and this codebase... I could make a number of suggestions for how to handle this with low effort on a largely unmaintained codebase... the easiest of them being just keeping the latest release of pyzm compatible with the latest version of ZM itself, and if you need to read SHM from an older ZM version, use an older pyzm version. Maybe even changing the version numbering of pyzm to match the major.minor of ZM itself. Or else just add something to the README and documentation (I'd be willing to open a PR for this) to clarify that ZMMemory only works with 1.34, when 1.36 has been out for over 2 years.

Alternatively, I'd be happy to take a crack at an implementation to support the structs for 1.34, 1.36, and either what's in master now or whatever the final struct will look like in 1.37. If it was acceptable to limit it to those versions and any future versions, we could probably just use the size of the struct to figure out which version it is.

At the moment, this seems to be the official Python API bindings for ZM... it's certainly in the ZoneMinder organization and relied on by a bunch of things in the ecosystem (i.e. the ZMES hooks), and there's nothing in the documentation or README that says that it's intentionally deprecated/unmaintained or shouldn't be used...

connortechnology commented 6 months ago

@jantman Thanks for this, and please, by all means, feel free to continue improving it. We don't support 1.34 anymore and once 1.38 is released and stabilised, we likely won't actively support 1.36 either. We do need this code to handle 1.37, but that's not too many cases.

jantman commented 6 months ago

Ok cool, and thanks for the merge. I can't commit to any real high level of support/contribution, but I'll at least keep opening PRs to keep this stuff functional, as I'm relying on it now (and plan on following the stable releases of ZM).