search

agsh / onvif

ONVIF node.js implementation
http://agsh.github.io/onvif/
MIT License
681 stars 231 forks source link

VideoEncoderConfiguration was refactored, but the doc was not #233

Open sjkummer opened 2 years ago

sjkummer commented 2 years ago

Seems like the H264 attributes of VideoEncoderConfiguration was moved to $ but the doc is unchanged. Was this refactoring intended (its a breaking change)?

See https://github.com/agsh/onvif/blob/master/lib/media.js#L122

current_dump
agsh commented 2 years ago

@sjkummer Hi! No, nothing changed. I think that something strange in your cam response. Can you send the response? In tests I rely on this sample: https://github.com/agsh/onvif/blob/master/test/serverMockup/media.GetVideoEncoderConfiguration.xml

sjkummer commented 2 years ago

Camera: AXIS M2026-LE Mk II Firmware: 9.80.1

Maybe related to Media2 API changes? @RogerHardiman
media2Support is set to true

XML:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope>
  <SOAP-ENV:Header/>
  <SOAP-ENV:Body>
    <tr2:GetProfilesResponse>
      <tr2:Profiles fixed="true" token="profile_1_h264">
        <tr2:Name>profile_1 h264</tr2:Name>
        <tr2:Configurations>
          <tr2:VideoSource token="0">
            <tt:Name>user0</tt:Name>
            <tt:UseCount>4</tt:UseCount>
            <tt:SourceToken>0</tt:SourceToken>
            <tt:Bounds height="1520" width="2688" y="0" x="0"/>
            <tt:Extension>
              <tt:Rotate>
                <tt:Mode>ON</tt:Mode>
                <tt:Degree>180</tt:Degree>
              </tt:Rotate>
            </tt:Extension>
          </tr2:VideoSource>
          <tr2:VideoEncoder Profile="Main" GovLength="32" token="default_1_h264">
            <tt:Name>default_1 h264</tt:Name>
            <tt:UseCount>1</tt:UseCount>
            <tt:Encoding>H264</tt:Encoding>
            <tt:Resolution>
              <tt:Width>2688</tt:Width>
              <tt:Height>1520</tt:Height>
            </tt:Resolution>
            <tt:RateControl>
              <tt:FrameRateLimit>30</tt:FrameRateLimit>
              <tt:BitrateLimit>2147483647</tt:BitrateLimit>
            </tt:RateControl>
            <tt:Multicast>
              <tt:Address>
                <tt:Type>IPv4</tt:Type>
                <tt:IPv4Address>0.0.0.0</tt:IPv4Address>
              </tt:Address>
              <tt:Port>0</tt:Port>
              <tt:TTL>5</tt:TTL>
              <tt:AutoStart>false</tt:AutoStart>
            </tt:Multicast>
            <tt:Quality>70</tt:Quality>
          </tr2:VideoEncoder>
        </tr2:Configurations>
      </tr2:Profiles>
      <tr2:Profiles fixed="true" token="profile_1_jpeg">
        <tr2:Name>profile_1 jpeg</tr2:Name>
        <tr2:Configurations>
          <tr2:VideoSource token="0">
            <tt:Name>user0</tt:Name>
            <tt:UseCount>4</tt:UseCount>
            <tt:SourceToken>0</tt:SourceToken>
            <tt:Bounds height="1520" width="2688" y="0" x="0"/>
            <tt:Extension>
              <tt:Rotate>
                <tt:Mode>ON</tt:Mode>
                <tt:Degree>180</tt:Degree>
              </tt:Rotate>
            </tt:Extension>
          </tr2:VideoSource>
          <tr2:VideoEncoder token="default_1_jpeg">
            <tt:Name>default_1 jpeg</tt:Name>
            <tt:UseCount>1</tt:UseCount>
            <tt:Encoding>JPEG</tt:Encoding>
            <tt:Resolution>
              <tt:Width>2688</tt:Width>
              <tt:Height>1520</tt:Height>
            </tt:Resolution>
            <tt:RateControl>
              <tt:FrameRateLimit>30</tt:FrameRateLimit>
              <tt:BitrateLimit>2147483647</tt:BitrateLimit>
            </tt:RateControl>
            <tt:Multicast>
              <tt:Address>
                <tt:Type>IPv4</tt:Type>
                <tt:IPv4Address>0.0.0.0</tt:IPv4Address>
              </tt:Address>
              <tt:Port>0</tt:Port>
              <tt:TTL>5</tt:TTL>
              <tt:AutoStart>false</tt:AutoStart>
            </tt:Multicast>
            <tt:Quality>70</tt:Quality>
          </tr2:VideoEncoder>
        </tr2:Configurations>
      </tr2:Profiles>
      <tr2:Profiles fixed="true" token="profile_1_h265">
        <tr2:Name>profile_1 h265</tr2:Name>
        <tr2:Configurations>
          <tr2:VideoSource token="0">
            <tt:Name>user0</tt:Name>
            <tt:UseCount>4</tt:UseCount>
            <tt:SourceToken>0</tt:SourceToken>
            <tt:Bounds height="1520" width="2688" y="0" x="0"/>
            <tt:Extension>
              <tt:Rotate>
                <tt:Mode>ON</tt:Mode>
                <tt:Degree>180</tt:Degree>
              </tt:Rotate>
            </tt:Extension>
          </tr2:VideoSource>
          <tr2:VideoEncoder Profile="Main" GovLength="32" token="default_1_h265">
            <tt:Name>default_1 h265</tt:Name>
            <tt:UseCount>1</tt:UseCount>
            <tt:Encoding>H265</tt:Encoding>
            <tt:Resolution>
              <tt:Width>2688</tt:Width>
              <tt:Height>1520</tt:Height>
            </tt:Resolution>
            <tt:RateControl>
              <tt:FrameRateLimit>30</tt:FrameRateLimit>
              <tt:BitrateLimit>2147483647</tt:BitrateLimit>
            </tt:RateControl>
            <tt:Multicast>
              <tt:Address>
                <tt:Type>IPv4</tt:Type>
                <tt:IPv4Address>0.0.0.0</tt:IPv4Address>
              </tt:Address>
              <tt:Port>0</tt:Port>
              <tt:TTL>5</tt:TTL>
              <tt:AutoStart>false</tt:AutoStart>
            </tt:Multicast>
            <tt:Quality>70</tt:Quality>
          </tr2:VideoEncoder>
        </tr2:Configurations>
      </tr2:Profiles>
      <tr2:Profiles fixed="false" token="profile0">
        <tr2:Name>yauto</tr2:Name>
        <tr2:Configurations>
          <tr2:VideoSource token="0">
            <tt:Name>user0</tt:Name>
            <tt:UseCount>4</tt:UseCount>
            <tt:SourceToken>0</tt:SourceToken>
            <tt:Bounds height="1520" width="2688" y="0" x="0"/>
            <tt:Extension>
              <tt:Rotate>
                <tt:Mode>ON</tt:Mode>
                <tt:Degree>180</tt:Degree>
              </tt:Rotate>
            </tt:Extension>
          </tr2:VideoSource>
          <tr2:VideoEncoder Profile="Main" GovLength="32" token="1">
            <tt:Name>user1</tt:Name>
            <tt:UseCount>1</tt:UseCount>
            <tt:Encoding>H264</tt:Encoding>
            <tt:Resolution>
              <tt:Width>2688</tt:Width>
              <tt:Height>1520</tt:Height>
            </tt:Resolution>
            <tt:RateControl>
              <tt:FrameRateLimit>25</tt:FrameRateLimit>
              <tt:BitrateLimit>2048</tt:BitrateLimit>
            </tt:RateControl>
            <tt:Multicast>
              <tt:Address>
                <tt:Type>IPv4</tt:Type>
                <tt:IPv4Address>0.0.0.0</tt:IPv4Address>
              </tt:Address>
              <tt:Port>0</tt:Port>
              <tt:TTL>5</tt:TTL>
              <tt:AutoStart>false</tt:AutoStart>
            </tt:Multicast>
            <tt:Quality>70</tt:Quality>
          </tr2:VideoEncoder>
        </tr2:Configurations>
      </tr2:Profiles>
    </tr2:GetProfilesResponse>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

JSON:

[
  {
    "$": {
      "fixed": true,
      "token": "profile_1_h264"
    },
    "name": "profile_1 h264",
    "videoSourceConfiguration": {
      "$": {
        "token": 0
      },
      "name": "user0",
      "useCount": 4,
      "sourceToken": 0,
      "bounds": {
        "$": {
          "height": 1520,
          "width": 2688,
          "y": 0,
          "x": 0
        }
      },
      "extension": {
        "rotate": {
          "mode": "ON",
          "degree": 180
        }
      }
    },
    "videoEncoderConfiguration": {
      "$": {
        "Profile": "Main",
        "GovLength": 32,
        "token": "default_1_h264"
      },
      "name": "default_1 h264",
      "useCount": 1,
      "encoding": "H264",
      "resolution": {
        "width": 2688,
        "height": 1520
      },
      "rateControl": {
        "frameRateLimit": 30,
        "bitrateLimit": 2147483647
      },
      "multicast": {
        "address": {
          "type": "IPv4",
          "IPv4Address": "0.0.0.0"
        },
        "port": 0,
        "TTL": 5,
        "autoStart": false
      },
      "quality": 70
    }
  },
  {
    "$": {
      "fixed": true,
      "token": "profile_1_jpeg"
    },
    "name": "profile_1 jpeg",
    "videoSourceConfiguration": {
      "$": {
        "token": 0
      },
      "name": "user0",
      "useCount": 4,
      "sourceToken": 0,
      "bounds": {
        "$": {
          "height": 1520,
          "width": 2688,
          "y": 0,
          "x": 0
        }
      },
      "extension": {
        "rotate": {
          "mode": "ON",
          "degree": 180
        }
      }
    },
    "videoEncoderConfiguration": {
      "$": {
        "token": "default_1_jpeg"
      },
      "name": "default_1 jpeg",
      "useCount": 1,
      "encoding": "JPEG",
      "resolution": {
        "width": 2688,
        "height": 1520
      },
      "rateControl": {
        "frameRateLimit": 30,
        "bitrateLimit": 2147483647
      },
      "multicast": {
        "address": {
          "type": "IPv4",
          "IPv4Address": "0.0.0.0"
        },
        "port": 0,
        "TTL": 5,
        "autoStart": false
      },
      "quality": 70
    }
  },
  {
    "$": {
      "fixed": true,
      "token": "profile_1_h265"
    },
    "name": "profile_1 h265",
    "videoSourceConfiguration": {
      "$": {
        "token": 0
      },
      "name": "user0",
      "useCount": 4,
      "sourceToken": 0,
      "bounds": {
        "$": {
          "height": 1520,
          "width": 2688,
          "y": 0,
          "x": 0
        }
      },
      "extension": {
        "rotate": {
          "mode": "ON",
          "degree": 180
        }
      }
    },
    "videoEncoderConfiguration": {
      "$": {
        "Profile": "Main",
        "GovLength": 32,
        "token": "default_1_h265"
      },
      "name": "default_1 h265",
      "useCount": 1,
      "encoding": "H265",
      "resolution": {
        "width": 2688,
        "height": 1520
      },
      "rateControl": {
        "frameRateLimit": 30,
        "bitrateLimit": 2147483647
      },
      "multicast": {
        "address": {
          "type": "IPv4",
          "IPv4Address": "0.0.0.0"
        },
        "port": 0,
        "TTL": 5,
        "autoStart": false
      },
      "quality": 70
    }
  },
  {
    "$": {
      "fixed": false,
      "token": "profile0"
    },
    "name": "yauto",
    "videoSourceConfiguration": {
      "$": {
        "token": 0
      },
      "name": "user0",
      "useCount": 4,
      "sourceToken": 0,
      "bounds": {
        "$": {
          "height": 1520,
          "width": 2688,
          "y": 0,
          "x": 0
        }
      },
      "extension": {
        "rotate": {
          "mode": "ON",
          "degree": 180
        }
      }
    },
    "videoEncoderConfiguration": {
      "$": {
        "Profile": "Main",
        "GovLength": 32,
        "token": 1
      },
      "name": "user1",
      "useCount": 1,
      "encoding": "H264",
      "resolution": {
        "width": 2688,
        "height": 1520
      },
      "rateControl": {
        "frameRateLimit": 25,
        "bitrateLimit": 2048
      },
      "multicast": {
        "address": {
          "type": "IPv4",
          "IPv4Address": "0.0.0.0"
        },
        "port": 0,
        "TTL": 5,
        "autoStart": false
      },
      "quality": 70
    }
  }
]
sjkummer commented 2 years ago

If I enforce media2Support to be false, it's working fine (again).

With media2Support set to true, the attributes are in the XML as follows:

<tr2:VideoEncoder Profile="Main" GovLength="32" token="1">

To me, it seems like the media 2 support is not yet complete and breaks some things in unexpected places.

Maybe, we should introduce an option to enable/disable the media 2 support when initializing a cam. What do you think, would such a PR be welcome? @agsh

RogerHardiman commented 2 years ago

Hi there Thanks for investigating the issue.

Maybe, we should introduce an option to enable/disable the media 2 support There are no CCTV viewing systems I've seen where you need decide if you want to use the Media(1) API or the Media2 API. (I use Milestone, Genetec, Avigilon, Bosch and IndigoVision and a few others less well known ones at work)

Bosch firmware 7.x has moved the OSD (On Screen Displaty) functions out of Media(1) API and only support them in Media2 API as I found out recently on a client project that uses OSD and this NodeJS library

So let's worth through and figure out what is wrong.

I thought the issue was a Hanwha camera, but in another post you mentioned an Axis camera with 9.80 firmware.

Can I check which camera had the Media2 problem please?

Thanks Roger

sjkummer commented 2 years ago

In chapter 5.12.14 of https://www.onvif.org/specs/srv/media/ONVIF-Media2-Service-Spec-v100.pdf I found the following definiton for VideoEncoder2Configuration

<xs:complexType name="VideoEncoder2Configuration">
 <xs:element name="Encoding" type="xs:string"/>
 <xs:element name="Resolution" type="tt:VideoResolution2"/>
 <xs:element name="RateControl" type="tt:VideoRateControl2"
minOccurs="0"/>
 <xs:element name="Multicast" type="tt:MulticastConfiguration"
minOccurs="0"/>
 <xs:element name="Quality" type="xs:float"/>
 <xs:attribute name="GovLength" type="xs:int"/>
 <xs:attribute name="Profile" type="xs:string"/>
</xs:complexType>

Note: GovLength and Profile are defined as attributes, not elements. That looks like the discrepancy.

Both, the Axis and the Hanwha/Samsung cameras behave the same regarding VideoEncoder2Configuration

Btw: As a workarround, I just set cameraObject.media2Support = false; for that use case. This can be set from outside without modifying the lib.

RogerHardiman commented 2 years ago

I've just used SOAP UI for testing (it is loaded with the ONVIF WSDLs) and pointed it at a Bosch camera. I can see the differente in GovLength and Profile and agree with you, They have moved to become attributes of "Configurations" and not elements.

MEDIA (1)

  <trt:GetVideoEncoderConfigurationsResponse>
         <trt:Configurations token="H26x_L1S1">
            <tt:Name>HD Image Optimized</tt:Name>
            ....
            <tt:H264>
               <tt:GovLength>60</tt:GovLength>
               <tt:H264Profile>Main</tt:H264Profile>
            </tt:H264>

MEDIA 2

      <tms:Configurations token="H26x_L1S1" GovLength="255" Profile="Main">
        ...
RogerHardiman commented 2 years ago

So looks like what we need to add to the Documentation is details of the returned data if Media Version is 1 and details of the returned data if Media Version is 2

sjkummer commented 2 years ago

I think there is a general decision to make:

Should the API of this lib change depending on the Media version used (breaking change, harder for the application developers who integrat the lib), or should we try to build an abstraction layer (transforming the data) to make sure, the JSON objects passed to, and returned from this lib stays the same.

From what I see, this will affect requests and responses for Media2 API calls.

What do you think @agsh @RogerHardiman ?

sjkummer commented 2 years ago

PS: If we can achive Media2 support without breaking changes would be great. If we can import all XSL schemes (XSD) from the onvif specs and generate TypeScript files from that would be great and more foolproof. But it will be quite some work, probably.

RogerHardiman commented 2 years ago

Hi @sjkummer I added just enough Media2 API code for my own needs - all I needed was JPEG URIs and RTSP Stream URIs. I did not care about getting a list of Video Sources, Video Codec Instances, Audio Codec Instances etc. So in my case, it made sense to make the replies from Media2 appear like Media1 replies.

But there are problems. The raw XML is different. Also some APIs are only in Media1 (and not in Media2)

So perhaps the best option is to remove the compatibility shim I used, where I send commands to Media2 if possible and translate the results. Perhaps the best option is this 1) Library has existing APIs that only use Media1 (even if camera supports Media2) 2) Library has new Media2 APIs called Media2_GetStreamURI and Media2_GetSnapshotURI etc

Then old software will 1) User connects to camera 2) Use Media1 with no change

For new software the code will 1) User connects to the camera 2) software can check if .media2support is true3 3) software can call either Media1 or Media2 APIs and it is the programmer who has to decide which to call. This means existing software will not benefit from Media2 support.

Software would need to be updated to handle Media1 or Media2 with the programmer making the decision.

When I first added Media2 support, I tried to hide the decision of Media1 or Media2, and it works OK for GetSteamURI and GetSnapshotURI, but this design model falls over when we want to use all the new APIs or parse the raw XML ourselves

sjkummer commented 2 years ago

Introducing new methods like Media2_GetStreamURI sounds like a good and safe approach to me.

agsh commented 2 years ago

PS: If we can achive Media2 support without breaking changes would be great. If we can import all XSL schemes (XSD) from the onvif specs and generate TypeScript files from that would be great and more foolproof. But it will be quite some work, probably.

@sjkummer Hi! You can fork this branch which I'm trying to rewrite in ts and add schemas there. Any help is appreciated :)

@RogerHardiman Yes, I think we should separate APIs to media.js and media2.js modules. Also refactor methods to use namespaces like this: cam.media2.getStreamURI. And cam.media.getStreamURI which is just reference to cam.prototype.getStreamURI to save backward compatibility. This is a huge work