Receive MetadataStream from Camera

[tofrye]

Hi,

how is it possible to fetch MetaData from a Camera with this library? I am able to connect to the Camera, get the DeviceCapabilites etc. but I dont know how to proceed with Metadata.

With the Software ONVIF Device Manager I am able to receive this data from my camera so the camera is capable of this. The output looks like this:

<tt:MetadataStream>
  <tt:VideoAnalytics>
    <tt:Frame UtcTime="2018-05-14T07:34:15.8280">
      <tt:Object ObjectId="16740">
        <tt:Appearance velocity="0.00" area="5.98">
    <tt:Shape>
       <tt:BoundingBox bottom="-0.78" top="0.93" right="0.96" left="0.32"/>
         <tt:CenterOfGravity x="0.64" y="0.09"/>
           <tt:Polygon>
             <tt:Point x="0.61" y="0.78"/>
             <tt:Point x="0.69" y="0.93"/>
           </tt:Polygon>
          </tt:Shape>
          <tt:Class>
            <tt:ClassCandidate>
             <tt:Type>Human</tt:Type>
             <tt:Likelihood>0.61</tt:Likelihood>
            </tt:ClassCandidate>
           </tt:Class>
         <tt:GeoLocation lon="9.4508070672052553" lat="52.5539138110899864" elevation="44.50"/>
        </tt:Appearance>
      </tt:Object>
    </tt:Frame>
  </tt:VideoAnalytics>
</tt:MetadataStream>

Thank you

[scottlamb]

The ONVIF Streaming Specification describes retrieving this via RTSP/RTP, as you would for a video or audio stream. I believe this is the only way.

You first must configure the stream via the ONVIF SOAP interface. This is the part that python-onvif might be able to help with. I'm a little fuzzy on the details; I haven't gotten it to work correctly with my cameras.

If you're able to get this to work with other software, you could get a packet capture with Wireshark and see what it does. I'd be very interested to see a packet capture of it setting it up from a factory reset camera to a usable metadata stream.

[agateo]

After the release of profile M in June 2021 I believe that this 'problem' remains; there is no operation to fetch the kind of data yhat you're available to fetch with Onvif Device Manager. If someone has an approach in order to do this, please share! 🤖

[UPDATE -> SOLUTION] The approach is the following one:

• First, we must get the Stream URI which will be used to fetch the metadata

-Then, we get the metadata via the RTSP describe method. You can test this functionality using a tool like openRTSP. In my case GetStreamUri answered the following rtsp query. This concrete tool will create two separate files: one with the raw binary metadata, and another one with the Onvif XML metadata.

To sum up, in order to implement it in other applications, you just have to fetch the URI via the previous command and then make a RTSP request with the obtained URL. Enjoy 🤡

[scottlamb]

You also need the metadata configuration to actually include the analytics and to be enabled on the media profile that matches your RTSP URL. Your camera might do those things straight out of the box or maybe some other software you ran set it up for you. Often that isn't true, though, which is what I was referring to above. Since writing that comment I figured out how to do this with python-onvif-zeep. It's probably similar if not identical with python-onvif.

c = ONVIFCamera(...)

c.create_devicemgmt_service()
c.create_media_service()

config = c.media.GetMetadataConfigurations()[0]
config.Analytics = True
req = c.media.create_type('SetMetadataConfiguration')
req.Configuration = config
req.ForcePersistence = True
c.media.SetMetadataConfiguration(req)

for profile in c.media.GetProfiles():
    print('profile %s' % profile.token)
    if not hasattr(profile, 'MetadataConfiguration'):
        c.media.AddMetadataConfiguration({
            'ProfileToken': profile.token,
            'ConfigurationToken': config.token,
        })
    resp = c.media.GetStreamUri({
        'StreamSetup': {'Stream': 'RTP-Unicast', 'Transport': {'Protocol': 'RTSP'}},
        'ProfileToken': profile.token,
    })
    print(resp.Uri)

[jimmycfa]

I ended up doing what @scottlamb recommended then running the following using GStreamer to get the stream:

The location comes from the printed out resp.Uri in his code:

gst-launch-1.0 rtspsrc location=<Resp.Uri from Above> ! application/x-rtp, media=application ! fakesink dump=true

[lchunleo]

Have the same requirements to get the video analytics meta data. Are the ways to get it via some api or ffmpeg stuff instead from gstreamer ? Thanks

[andyg2]

As of 2022.04.12 from the live555 change log

• Updated the "openRTSP" application (RTSP command-line client) to add an option '-L', meaning: receive only an "application" (e.g., 'metadata') track, if present, outputting the data to 'stdout'. (This is analogous to the existing '-a' and '-v' options, for receiving only audio or video.)

e.g. openRTSP -L rtsp://service:Agata1234.@192.168.12.51/rtsp_tunnel?p=0&h=26x=4&vcd=2 :boom: