search

maciej-or / hikvision_next

Home Assistant integration for Hikvision NVRs and IP cameras
90 stars 16 forks source link

Client error '403 Forbidden' for url 'http://xxx.xxx.xxx.xxx/ISAPI/Streaming/channels/y0y but browser with this url returns '200 OK' #176

Closed Nikolay-Ch closed 1 week ago

Nikolay-Ch commented 2 months ago

Hi,

When I connect NVR, this addon try to check all channels of the all cameras, that were connected to the NVR. And then in log file of the HA I see this messages:

Client error '403 Forbidden' for url 'http://xxx.xxx.xxx.xxx/ISAPI/Streaming/channels/y0y

But if I try to call HTTP-request with this URL and credentials that I enter into addon, in browser I get something else like this:

<ResponseStatus xmlns="http://www.hikvision.com/ver20/XMLSchema" version="1.0">
<requestURL>/ISAPI/Streaming/channels/103</requestURL>
<statusCode>4</statusCode>
<statusString>Invalid Operation</statusString>
<subStatusCode>notSupport</subStatusCode>
</ResponseStatus>

And addon logs 403 errors, when each camera returns something useful information about channell or status "Not Supported".

I have HiWatch DS-N308(C) NVR, and DS-I400(C) cameras...

maciej-or commented 2 months ago

A HTTP response with a status code of 403 can indeed include a JSON body. By initialization the integration relies on status code only to probe available channels. Many cameras have no 3rd and 4th channels. The level of log entry is WARNING so you shouldn't worry about that.

Nikolay-Ch commented 2 months ago

A HTTP response with a status code of 403 can indeed include a JSON body. By initialization the integration relies on status code only to probe available channels. Many cameras have no 3rd and 4th channels.

Above I wrote that my camera returns “200 OK” to this URL and XML, which contains the status “Not supported”... And returning “403 Forbidden” probably indicates that the addon is accessing the camera with incorrect credentials. Can I log the credentials used by addon to create HTTP-Request?

The level of log entry is WARNING so you shouldn't worry about that.

But I have another logs... Every minute for each camera... My log full of this messages (see below)

WARNING (MainThread) [custom_components.hikvision_next.isapi] Unexpected exception | Cannot fetch state for io | Client error '403 Forbidden' for url 'http://**IP_ADDRESS_OF_NVR**/ISAPI/System/IO/inputs/401'

And when I run this URL in browser, NVR returns this XML (not 403 Error)

<ResponseStatus xmlns="http://www.hikvision.com/ver20/XMLSchema" version="1.0">
<requestURL>/ISAPI/System/IO/inputs/401</requestURL>
<statusCode>4</statusCode>
<statusString>Invalid Operation</statusString>
<subStatusCode>invalidOperation</subStatusCode>
</ResponseStatus>

Can you check - with what credentials you create HTTP-request to NVR?

maciej-or commented 2 months ago

Credentials are always the same but I can check specific request because some of my cameras by initialization respond 403 as well, especially for 3rd stream.

maciej-or commented 2 months ago

So, in the browser I get 403 as well and XML response. Credentials are in Authorization header of any request. Regarding errors on System/IO/inputs/401 you could check admin account, maybe some permissions are missed for your current user. If not pls share diagnostic data of your NVR it would allow debug more in tests environment.

Nikolay-Ch commented 2 months ago

So, in the browser I get 403 as well and XML response. Credentials are in Authorization header of any request. Regarding errors on System/IO/inputs/401 you could check admin account, maybe some permissions are missed for your current user. If not pls share diagnostic data of your NVR it would allow debug more in tests environment.

But I get no 403 error - I get 200. With the same credentials. Maybe you create wrong request for HiWatch NVR - because it is a cheaper product-line of the HikView? Can you clarify the code-line where your request is generated? I try to run this code in command line to check it with my cameras and NVR.

maciej-or commented 2 months ago

It is not straightforward as it seems. Basically any request goes from here https://github.com/maciej-or/hikvision_next/blob/main/custom_components/hikvision_next/isapi.py#L760 deeper is hikvisionapi library where http request is generated. I would recommend to run and debug homeassistant and the integration in dev container. https://developers.home-assistant.io/docs/development_environment

Nikolay-Ch commented 2 months ago

Look, I understand. HIk library returns code 403, but in http-body returns some usefull info in xml. And you can analyze it.

And another thing... Your addon every minute request for unsupported inputs. and every minute I get in logs messages about it. Can you change log-level of you logging in this (and similar) case? Because. if you check something and waiting for error it is not right pattern to log this in WARNING level. Log it at DEBUG or INFO level... Debug at WARNING level you need only if somethig going wrong.

maciej-or commented 2 months ago

I don't need to analyze payload of failed /Streaming/channels/y0y requests, if failed it does mean the stream entity wont be created. If the integration retries requests every minute it is not initialized properly. Did you try use admin to exclude missing permissions case?

Nikolay-Ch commented 2 months ago

If the integration retries requests every minute it is not initialized properly. Did you try use admin to exclude missing permissions case?

What rights I need to add to user? Below all logs from HA log-file. So you can see that warnings repeated every two minutes.

2024-04-29 17:47:54.056 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/103
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/103'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:47:54.132 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/104
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/104'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:47:54.503 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/203
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/203'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:47:54.719 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/204
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/204'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:47:56.533 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/303
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/303'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:47:56.855 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/304
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/304'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:47:59.770 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/403
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/403'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:48:00.254 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/404
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/404'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:48:01.970 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/503
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/503'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:48:02.281 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/504
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/504'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:48:02.439 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/601
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/601'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:48:02.838 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/602
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/602'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:48:03.112 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/603
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/603'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:48:03.285 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/604
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/Streaming/channels/604'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:48:05.481 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/401
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/401'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:48:05.542 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/501
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/501'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:50:06.736 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/401
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/401'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:50:06.737 WARNING (MainThread) [custom_components.hikvision_next.isapi] Unexpected exception | Cannot fetch state for io | Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/401'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:50:06.796 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/501
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/501'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:50:06.796 WARNING (MainThread) [custom_components.hikvision_next.isapi] Unexpected exception | Cannot fetch state for io | Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/501'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:52:07.758 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/401
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/401'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:52:07.759 WARNING (MainThread) [custom_components.hikvision_next.isapi] Unexpected exception | Cannot fetch state for io | Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/401'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:52:07.826 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/501
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/501'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:52:07.826 WARNING (MainThread) [custom_components.hikvision_next.isapi] Unexpected exception | Cannot fetch state for io | Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/501'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:54:08.829 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/401
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/401'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:54:08.830 WARNING (MainThread) [custom_components.hikvision_next.isapi] Unexpected exception | Cannot fetch state for io | Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/401'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:54:08.888 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/501
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/501'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:54:08.889 WARNING (MainThread) [custom_components.hikvision_next.isapi] Unexpected exception | Cannot fetch state for io | Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/501'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:56:10.773 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/401
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/401'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:56:10.773 WARNING (MainThread) [custom_components.hikvision_next.isapi] Unexpected exception | Cannot fetch state for io | Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/401'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:56:10.847 WARNING (MainThread) [custom_components.hikvision_next.isapi] --- [GET] http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/501
Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/501'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

2024-04-29 17:56:10.847 WARNING (MainThread) [custom_components.hikvision_next.isapi] Unexpected exception | Cannot fetch state for io | Client error '403 Forbidden' for url 'http://IP_ADDRESS_OF_NVR/ISAPI/System/IO/inputs/501'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403
maciej-or commented 2 months ago

it is definitely wrong behaviour, please share diagnostic data, then I could debug more image

TheEric7 commented 2 months ago

Is there any update on this? I'm experiencing the same issue.

Nikolay-Ch commented 2 months ago

Is there any update on this? I'm experiencing the same issue.

I have not yet had time to prepare the information requested by the author (diagnostic information). Maybe you can send it before me?

code-monkey-71 commented 3 weeks ago

I've been dealing with a similar issue. What I've discovered for my setup is that I can't get still images directly from the NVR using the URL in the integration. I get see the same error as @Nikolay-Ch. I found another thread that led me to a different URL for still images: http:///ISAPI/ContentMgmt/StreamingProxy/channels/101/picture

albeit lower image quality than going to the camera directly, it produces the desired image. This is the thread that led me to the URL above (It's towards the bottom): https://community.home-assistant.io/t/hikvision-still-image/183682/8

I'm not sure if this helps, but I wanted to share because I've been wrestling with this issue for a few months.

maciej-or commented 3 weeks ago

@code-monkey-71 pls check authentication mode for RTSP and WEB, here is a sample setup https://github.com/maciej-or/hikvision_next/issues/124#issuecomment-1815402298 if you still have the issue just share diagnostic data, it's fully anonymised

code-monkey-71 commented 2 weeks ago

I have set both authentication modes to digest, but I don't have the algorithm configuration setting. Was there something else from the post that I was supposed to do? I have followed the directions for giving user the appropriate permissions, but I haven't been able to get a snapshot from any of the cameras even if I configure the integration to use the admin user.
image

Here is the diagnostic data: config_entry-hikvision_next-b8b35a802a2cfaf2ceb4883891e8493f.json

debug logging: home-assistant_hikvision_next_2024-07-02T16-11-55.771Z.log

I have two cameras that are connected to the NVR from another switch and those 2 cameras get their IP addresses from my DHCP server and are therefore able to be connected directly from HA. If I connect the NVR and the cameras directly to HA via the integration, I can get a snapshot from the instance that's directly connected to HA, but I get the failure above if I try to go through the NVR. Video recording seems to work either way for what it's worth.

Here is the image created by the integration if I call the Take Snapshot camera service: hikvision_next.jpg.txt

code-monkey-71 commented 2 weeks ago

One other data point is that I can create a generic camera that connects through the NVR and it's able to get a snapshot and also stream correctly: Screenshot 2024-07-02 125349

These are the URLs: Still Image URL: http:///ISAPI/ContentMgmt/StreamingProxy/channels/201/picture

Stream Source URL: rtsp://:554/ISAPI/Streaming/Channels/201/

I can cycle through all 9 cameras this way, even the ones that are connected with IP addresses accessible by HA. However, the image quality is lower than what I get if I pull an image directly from those cameras. I'm still trying to unravel this mystery for some automations and notifications, so I hope this helps. My next step was going to be looking at the code to see if a configuration option may be achievable.

maciej-or commented 2 weeks ago

Thanks for details of your setup. You shouldn't worry about failed request for 'http://IP_NVR/ISAPI/Streaming/channels/103', 104. Any camera can provide several steams and these requests check what is available on specific device. So in your case the 1st camera supports main stream 101 - 2560x1440, 2nd stream 102 - 640x480, and does not support 103 and 104, it's fine.

For still images the integration uses: ISAPI/Streaming/channels/101/picture?videoResolutionWidth=2560&videoResolutionHeight=1440 ISAPI/Streaming/channels/102/picture?videoResolutionWidth=640&videoResolutionHeight=480

ISAPI/ContentMgmt/StreamingProxy/channels/x0y/picture should work as well, for better quality you can add size params

It is mystery indeed why ISAPI/Streaming/channels/101/picture does not work for cameras connected via NVR. If you are able to modify integration code feel free replace this line https://github.com/maciej-or/hikvision_next/blob/main/custom_components/hikvision_next/isapi.py#L857 with chunks = self.isapi.ContentMgmt.StreamingProxy.channels[stream.id].picture(method=GET, type="opaque_data", params=params) and restart HA

code-monkey-71 commented 2 weeks ago

Thanks for saving me time reading through the code. I was starting to dig through isapi.py, but hadn't quite found the right spot.

This code change solved the snapshot issue and everything else appears to still be working correctly. I'll do a little more testing through the day and post back any findings. I'm also going to look into potential chipset anomalies, as I recall having to get a special firmware update from Hikvision that wasn't publicly available. I'll try to dig up that email and see if it helps explain the problem.

maciej-or commented 2 weeks ago

Cool, I compared your diagnostic data with mine but haven't found particular differences regarding using another url for still images. Ideally would be find a condition when to use 2nd url and improve the integration.

code-monkey-71 commented 1 week ago

Just reporting back that I haven't seen any issues with this workaround in my environment. Everything is now working as expected.

Thanks, Chris