Autotracking with Axis Camera

[alexyao2015]

Background I am attempting to use autotracking with an Axis ptz camera (AXIS M5525-E), but it started spewing out errors even though the camera has all the onvif capabilities and is supported. Since Axis was one of the founders of the onvif standard, I was pretty confident that their implementation should have been correct.

Debugging Running the test script showed 3 YES logs, however the service_capabilities was empty. When running Frigate, it crashed at this line where obtained the service_capabilities and received a None value. I removed this check and was able to get past it.

Next I noticed my logs giving some error relating to the zoom being out of bounds at this line. I determined that my camera responded with 0-1 as the values Frigate checks for zoom interpolation, however my camera supported from 0-0.499. I was able to fix this using the following:

zoom = numpy.interp(
    zoom,
    [0, 1],
    [
        self.cams[camera_name]["zoom_limits"]["Range"]["XRange"]["Min"],
        self.cams[camera_name]["zoom_limits"]["Range"]["XRange"]["Max"],
    ],
)

After these two changes, autotracking started working normally.

Fix

1. We may want to use a different check instead of service_capabilities to verify movement status. Why wasn't GetStatus simply used and a check to see if that was idle? Is there a reason why a completely different value is checked?
2. For the camera interpolation, I would suggest using the above lines of code to determine the zoom range to interpolate. I only have 1 camera, so I cannot verify that this works across other cameras and would need others to help verify if this is a good solution.

CC: @hawkeye217

[hawkeye217]

Thanks for the info.

For the first issue: There was a recent discussion about GetServiceCapabilities and AXIS cameras.

The ONVIF protocol says here that the GetServiceCapabilities call should return True if the camera supports MoveStatus information. This MoveStatus parameter is used by Frigate's autotracking to determine when your PTZ is moving.

The reason why this parameter was chosen was:

1. It is following the ONVIF standard.
2. Some cameras report bogus values for GetStatus - they may show MOVING or IDLE but because GetServiceCapabilities doesn't indicate support for MoveStatus, the values from GetStatus are untrustworthy. Additionally, some users with Hikvision cameras had a MoveStatus value that remained as MOVING even after the PTZ motor stopped.

Other cameras we've tested with and that users have reported success with all have correctly reported GetServiceCapabilities as expected by the ONVIF standard.

For the second issue: the ONVIF code in Frigate saves the absolute zoom limits from the camera here and then uses them in calculations elsewhere when absolute zooming is used. In my testing, the zoom_limits values are not always the same as the absolute zoom limit values reported in the ptz profile's Spaces.AbsoluteZoomPositionSpace object.

In short, I've found that the ONVIF "standard" is not quite as standard as one would hope. Basic things (continuous move, presets, streams, etc) seem to be widely standardized, but the implementation of some of the finer parameters seem to vary widely from manufacturer to manufacturer, despite them saying they "support ONVIF".

[alexyao2015]

Thanks for your comments! I understand the concerns now with some cameras not conforming to the ONVIF spec. I still do find it odd that a manufacturer like AXIS produces cameras that doesn't confine to a spec they helped create. Ultimately, I as well as others likely would want their cameras to work with this new feature, so I would like to toss out a couple of ideas for solving these issues.

For GetServiceCapabilities, could we leave this as an optional config options to simply ignore this check? I did see that in my specific case, disabling this check allowed the camera to still function as GetStatus did return the correct MoveStatus values.

As for the zoom limits, I know you mention that zoom_limits are different, but does this cause issues with other cameras if it were used instead of the Spaces.AbsoluteZoomPositionSpace object? I know it certainly worked on my camera, but I cannot speak to other cameras.

Additionally as I was taking look as this, I think I found a bug. The usage of np.interp is inverted here. Since the values are getting interpolated back from the camera's view to 0-1, the camera's zoom should come first.

Hopefully we can figure out a solution to getting something working in for everyone. I have my frigate version monkey patched with the few changes I mentioned and they have been working well on my camera.

[hawkeye217]

Thanks for your comments! I understand the concerns now with some cameras not conforming to the ONVIF spec. I still do find it odd that a manufacturer like AXIS produces cameras that doesn't confine to a spec they helped create. Ultimately, I as well as others likely would want their cameras to work with this new feature, so I would like to toss out a couple of ideas for solving these issues.

For GetServiceCapabilities, could we leave this as an optional config options to simply ignore this check? I did see that in my specific case, disabling this check allowed the camera to still function as GetStatus did return the correct MoveStatus values.

Adding features to work around individual camera issues or manufacturer's firmware bugs and incomplete ONVIF support isn't something that makes a lot of sense from a support perspective, especially when myself and the other maintainers don't have those cameras to test with.

As for the zoom limits, I know you mention that zoom_limits are different, but does this cause issues with other cameras if it were used instead of the Spaces.AbsoluteZoomPositionSpace object? I know it certainly worked on my camera, but I cannot speak to other cameras.

Additionally as I was taking look as this, I think I found a bug. The usage of np.interp is inverted here. Since the values are getting interpolated back from the camera's view to 0-1, the camera's zoom should come first.

This is incorrect and not a bug. Absolute zoom values from the camera get mapped to a value between 0 and 1. Frigate uses a value of 0-1 internally to keep track of cameras using absolute zooming, so the code as written is working as intended.

However, it may be possible to better catch errors or unexpected values with cameras like yours. Could you enable debug logging, restart Frigate, and paste the initial log output as well as your Frigate config here?

logger:
  default: info
  logs:
    frigate.ptz.autotrack: debug
    frigate.ptz.onvif: debug

[alexyao2015]

@hawkeye217

Sorry about the long delay here. I wanted to give an update on this as I have been patching frigate manually each release and have been using autotracking on these cameras without issue since I made the initial post. I am confident that these cameras can be supported, even if they do not support the "correct" ONVIF standard. With these patches applied, autotracking is perfect and I've observed it track people for months without any fail.

Attached in the zip file, I've included the relevant log snippets and config snippets as well as the git patches to onvif.py and autotrack.py that I've been using. I would imagine that the onvif profile from the logs and the patches will likely prove to be the most useful. The config is pretty standard and functions just fine with my camera, post patch.

I'm hoping that these files will allow something to make it into the main branch, but they will not suffice as I have them written today because they assume some specifics about my particular cameras.

Thanks for all your work on this! data.zip

[alexyao2015]

Some answers to your response:

especially when myself and the other maintainers don't have those cameras to test with

I am willing to help with the code and test this if necessary if there was a selection that allows the selection of this zoom_limits in the ONVIF properties compared to the absolute_zoom_range that is used today. I am very confident that this is all that is required to support at least one other camera type.

Absolute zoom values from the camera get mapped to a value between 0 and 1. Frigate uses a value of 0-1 internally to keep track of cameras using absolute zooming, so the code as written is working as intended.

Here is a simple demonstration showing that the code is wrong. If this is incorrect, please let me know which of my assumptions are incorrect. Additionally, as mentioned I've been running a patched version with this exact code and haven't had any adverse side effects whereas it was completely nonfunctional before.

import numpy as np
# In this example, our zoom level is 0.25 within a camera zoom range of 0-0.5 where 0 is 0% zoom and 0.5 is 100% zoom
# 0.25 is 50% of our zoom so we expect the result of this operation to return 0.5.

# Before (Code in repo today)
# Here the result of this operation is 0.25 because we are scaling to the camera zoom level rather than frigate's zoom level
np.interp(
  0.5, # our zoom level to interpolate
  [
    0, # frigate min zoom -> always 0
    1, # frigate max zoom -> always 1
  ],
  [
    0, # camera min zoom -> read onvif from camera
    0.5, # camera max zoom -> read onvif from camera
  ]
)
# After (my changes)
# Here the result of the operation is 0.5 as we expect
np.interp(
  0.25, # our zoom level to interpolate
  [
    0, # camera min zoom -> read onvif from camera
    0.5, # camera max zoom -> read onvif from camera
  ],
  [
    0, # frigate min zoom -> always 0
    1, # frigate max zoom -> always 1
  ]
)

[hawkeye217]

Thanks for this! A few thoughts:

I am willing to help with the code and test this if necessary if there was a selection that allows the selection of this zoom_limits in the ONVIF properties compared to the absolute_zoom_range that is used today. I am very confident that this is all that is required to support at least one other camera type.

The absolute_zoom_range is a property that I assign in code directly from the onvif property AbsoluteZoomPositionSpace, per the onvif spec. See the code here: https://github.com/blakeblackshear/frigate/blob/dev/frigate/ptz/onvif.py#L297

Some cameras have the ability to configure zooming limits in firmware, so the ZoomLimits onvif property reflects this. Again, from the onvif spec:

ZoomLimits – The zoom limits element should be present for a PTZ node that supports absolute zoom. If the element is present it signals the supports for configurable zoom limits. If limits are enabled the zoom movements shall always stay within the specified range

On my Dahua, the AbsoluteZoomPositionSpace property works as I'd expect and mirrors ZoomLimits. However on other cameras I've seen, ZoomLimits is not always present (again, because of a bad onvif implementation), hence why the code uses AbsoluteZoomPositionSpace to determine the usable range for absolute zooming.

Your cameras do report values for AbsoluteZoomPositionSpace, so you shouldn't need to patch that code. But because your camera's onvif implementation doesn't support GetServiceCapabilities, you'll need to continue to patch for that as Frigate will continue to follow the onvif standard there.

As for the interpolation logic, it does seem to be reversed, thanks for catching that. I'll have a PR up to fix that shortly.

[hawkeye217]

Once again I just have to say that the onvif standard and the implementation of it from camera manufacturers is a mess. I did a little more reading and I can see how ZoomLimits relates to AbsoluteZoomPositionSpace, but the fact that it's up to the camera manufacturer to determine how these two values should practically correlate leaves developers who want to use the standard across various cameras hanging out to dry.

The general direction Frigate has taken with the codebase is to avoid hardware or camera-specific code to work around bugs or problems. Since you're already familiar with patching, that's our recommendation to keep your cameras working. I don't suspect there will be any major changes to the autotracking or onvif code, so you shouldn't have a difficult time patching from release to release.