System stopped working (Required request header 'x-installationNumber' not present) -- Error with login again (seems so similar to #166)

[danise76]

Hi.

Since yesterday the login stopped working in the Home Assistant plugin. It's the last version installed (v2.4.0.4). Seems similar to what happend in #166

We got 2 errors. First and more general one

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 361, in _async_setup_platform
    await asyncio.shield(task)
  File "/config/custom_components/securitas/alarm_control_panel.py", line 61, in async_setup_entry
    current_state: CheckAlarmStatus = await client.update_overview(
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/securitas/__init__.py", line 397, in update_overview
    status.status,
    ^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'status'

But the main thing, is that it ask for a header. I have 2 alarms under the same account:

Logger: custom_components.securitas.securitas_direct_new_api.apimanager
Source: custom_components/securitas/securitas_direct_new_api/apimanager.py:159
Integration: Securitas Direct (documentation, issues)
First occurred: 10:50:27 (1 occurrences)
Last logged: 10:50:27

Required request header 'x-installationNumber' for method parameter type String is not present

To be sure, i've removed the installation, readded it clean, configured (2-FA works fine). But after the configuration it show no devices as it can't update them, and give the errors in the logs.

Daniel

[RenZan]

Hello, I have the same error, I tried adding this x-installationNumber in _execute_request of apimanager.py but without success, I still have the error.

[KoolLSL]

I also have the same 'x-installationNumber' error since yesterday. Same with v2.4.0.2, v2.4.0.3, v2.4.0.4.

[guerrerotook]

There was a change in the API, working on it.

[KoolLSL]

Not idea if it is related but in apimanager.py I see ""callby": "OWA_10"" and if checking traffic while logging via web, I see ""callby": "OWP_10"

[guerrerotook]

I just fixed the issue and works on my installation.

[RenZan]

I just fixed the issue and works on my installation.

Nice, can you open a PR for us to test as well ?

thank you very much

[guerrerotook]

I just published a new version https://github.com/guerrerotook/securitas-direct-new-api/releases/tag/v2.5.0.0

[KoolLSL]

Many thanks @guerrerotook ! v2.5.0.0 solved this issue, the entity was found again and shows correct status.

[chabou-san]

Confirmed here too it's working again, thank you so much for your reactivity ! 🚀 🎊

[danise76]

All is working again with the update

[chabou-san]

Actually, I still see the error (missing header) 2 hours and 20 minutes after having restarted HomeAssistant. So it seems you fixed the issue preventing to start correctly, but maybe you missed something ? My alarm status stayed deactivated the whole time, so I guess it happens when the status is polled even when no change is happening at all. Am I the only one ?

[danise76]

Hi. I think that still a bug around. Like it happend while ago, after some time with the integration being up, the errors appears again and not update any status

Last logged: 17:50:55

Required request header 'x-installationNumber' for method parameter type String is not present

If you reload the integration or restart HA, it works again for a while

[guerrerotook]

Can you share the logs for the error? I only add this parameter to one of the operations, but maybe there are more who required it.

[danise76]

Hi. I think that still a bug around. Like it happend while ago, after some time with the integration being up, the errors appears again and not update any status

Last logged: 17:50:55

Required request header 'x-installationNumber' for method parameter type String is not present

If you reload the integration or restart HA, it works again for a while

Just noticed. When it appears, it appears 2 times in the log, same time but lile 2 errors logged.

[danise76]

Hi. I think that still a bug around. Like it happend while ago, after some time with the integration being up, the errors appears again and not update any status

Last logged: 17:50:55

Required request header 'x-installationNumber' for method parameter type String is not present

If you reload the integration or restart HA, it works again for a while

Just noticed. When it appears, it appears 2 times in the log, same time but lile 2 errors logged.

By the way. I have 2 installations under the same account, if it may help

[chabou-san]

Can you share the logs for the error? I only add this parameter to one of the operations, but maybe there are more who required it.

Here are the logs from "download full logs" button. I hope this will be enough to debug.

home-assistant_2023-12-16T17-39-16.588Z.log

[guerrerotook]

Thanks for sharing the logs, the most important thing is the call stack so I can see what api operation need the capability parameter. Thanks !

[skamaleo]

With the latest update 2.5.0.0, the first status update runs flawlessly. However, 20 minutes from the first call, the update operation shows the 'x-installationNumber' warning... Could you look into it? Thank you!

[guerrerotook]

I started working on the issue but having trouble decoding the jwt token. Will keep working on this.

[guerrerotook]

I created this new release, https://github.com/guerrerotook/securitas-direct-new-api/releases/tag/v2.5.1.0 please share any errors in your instalations.

[skamaleo]

@guerrerotook with 2.5.1.0 update, HA is shooting an update call every minute. Can you confirm, please?

[KoolLSL]

2.5.1.0 seems to work better regarding the status when arming/disarming with an external device. Actually the status got updated few seconds after, when it was 20 minutes in 2.4 versions. However there are errors in the logs:

2023-12-18 14:38:15.480 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/config/custom_components/securitas/alarm_control_panel.py", line 146, in async_update_status alarm_status = await self.client.update_overview(self.installation) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/securitas/init.py", line 404, in update_overview await self.get_services(installation) File "/config/custom_components/securitas/init.py", line 374, in get_services return await self.session.get_all_services(instalation) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/securitas/securitas_direct_new_api/apimanager.py", line 361, in get_all_services response: ClientResponse = await self._execute_request(content, "Srv") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/securitas/securitas_direct_new_api/apimanager.py", line 133, in _execute_request error_login: bool = await self._check_errros(response_text) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/securitas/securitas_direct_new_api/apimanager.py", line 155, in _check_errros response = json.loads(value) ^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/init.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

and

2023-12-18 15:25:15.689 ERROR (MainThread) [custom_components.securitas.securitas_direct_new_api.apimanager] Required request header 'x-installationNumber' for method parameter type String is not present 2023-12-18 15:25:15.690 ERROR (MainThread) [custom_components.securitas] Required request header 'x-installationNumber' for method parameter type String is not present

[RenZan]

On my part I seem to have an issue with the jwt token: Everything works fine for hours, and then I have these kind of error logs:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 698, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 959, in async_device_update
    await self.async_update()
  File "/config/custom_components/securitas/alarm_control_panel.py", line 327, in async_update
    alarm_status: CheckAlarmStatus = await self.client.update_overview(
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/securitas/__init__.py", line 395, in update_overview
    token = jwt.decode(
            ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jwt.py", line 210, in decode
    decoded = self.decode_complete(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jwt.py", line 151, in decode_complete
    decoded = api_jws.decode_complete(
              ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jws.py", line 198, in decode_complete
    payload, signing_input, header, signature = self._load(jwt)
                                                ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jws.py", line 254, in _load
    raise DecodeError(f"Invalid token type. Token must be a {bytes}")
jwt.exceptions.DecodeError: Invalid token type. Token must be a <class 'bytes'>

[KoolLSL]

I also noticed that at first all seems to work fine for hours. Then it stays blocked on "Disarming" and I had to restart the integration. There are errors at various frequency. The 981 times is about jwt.

Required request header 'x-installationNumber' for method parameter type String is not present 12:50:07 – (ERROR) Securitas Direct (custom integration) - message first occurred at 18 December 2023 at 15:25:15 and shows up 12 times Required request header 'x-installationNumber' for method parameter type String is not present 12:50:07 – (ERROR) Securitas Direct (custom integration) - message first occurred at 18 December 2023 at 15:25:15 and shows up 12 times Error doing job: Task exception was never retrieved 11:49:14 – (ERROR) Securitas Direct (custom integration) - message first occurred at 18 December 2023 at 19:29:13 and shows up 981 times Update for alarm_control_panel.securitas_a fails 11:37:11 – (ERROR) helpers/entity.py - message first occurred at 18 December 2023 at 19:37:11 and shows up 50 times

[danise76]

Hi.

I can confirm it also as Renzan said. Inside is not happy with the token.

Logger: homeassistant.helpers.entity
Source: helpers/entity.py:698
First occurred: December 18, 2023 at 19:15:21 (2640 occurrences)
Last logged: 15:47:42

Update for alarm_control_panel.calle_XXXX_XXXX_XXXX fails
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 698, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 959, in async_device_update
    await self.async_update()
  File "/config/custom_components/securitas/alarm_control_panel.py", line 327, in async_update
    alarm_status: CheckAlarmStatus = await self.client.update_overview(
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/securitas/__init__.py", line 395, in update_overview
    token = jwt.decode(
            ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jwt.py", line 210, in decode
    decoded = self.decode_complete(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jwt.py", line 151, in decode_complete
    decoded = api_jws.decode_complete(
              ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jws.py", line 198, in decode_complete
    payload, signing_input, header, signature = self._load(jwt)
                                                ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jws.py", line 254, in _load
    raise DecodeError(f"Invalid token type. Token must be a {bytes}")
jwt.exceptions.DecodeError: Invalid token type. Token must be a <class 'bytes'>

[pganansia]

Hi,

I get exactly the same difficulty as danise76. Rgds

[guerrerotook]

Thanks everyone for the feedback. I'm currently working in trying to fix this issue, and I am trying that on my local home assistant instance.

[KoolLSL]

Meanwhile, as a workaround, I added in the scripts which arm or disarm the alarm a reload, with a small delay before. So far it seems to update the status correctly instead of staying on arming or disarming.

  - delay:
      hours: 0
      minutes: 0
      seconds: 4
      milliseconds: 0
  - service: homeassistant.reload_config_entry
    target:
      entity_id: alarm_control_panel.securitas_home
    data: {}

[samatild]

+1 with same issue, running v2.5.1.0

2023-12-26 20:38:49.997 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/config/custom_components/securitas/alarm_control_panel.py", line 146, in async_update_status
    alarm_status = await self.client.update_overview(self.installation)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/securitas/__init__.py", line 395, in update_overview
    token = jwt.decode(
            ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jwt.py", line 210, in decode
    decoded = self.decode_complete(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jwt.py", line 151, in decode_complete
    decoded = api_jws.decode_complete(
              ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jws.py", line 198, in decode_complete
    payload, signing_input, header, signature = self._load(jwt)
                                                ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jws.py", line 254, in _load
    raise DecodeError(f"Invalid token type. Token must be a {bytes}")
jwt.exceptions.DecodeError: Invalid token type. Token must be a <class 'bytes'>

Along with other errors that are a consequence of this one (piled requests that are not processed, etc...)

Going to try @KoolLSL suggested workaround... seems reasonable as it's the exact same thing I'm doing (but manually)

[cantupaz]

@guerrerotook I'm not seeing this problem myself, so I cannot offer logs, but I stumbled upon the x-installationNumber while debugging something, so this may be useful. When the token expires, the code gives it a minute before forcing an update. That could cause problems. Imagine that the token has actually expired, but less than a minute has elapsed. If something calls an API function that needs the token that function would not work (and show the x-installationNumber issue). In init.py:

    if "exp" in token:
        expiration: datetime = datetime.fromtimestamp(token["exp"])
        if datetime.now() + timedelta(minutes=1) > expiration:
            # if the token is expired get a new one
            await self.get_services(installation)

Getting rid of the extra minute is not going to solve all the problems, but it will help.

I think you could hack a fix moving the logic that checks if the token is expired to the apimanager in a way similar like how you handle the login expiring.

[guerrerotook]

@guerrerotook I'm not seeing this problem myself, so I cannot offer logs, but I stumbled upon the x-installationNumber while debugging something, so this may be useful. When the token expires, the code gives it a minute before forcing an update. That could cause problems. Imagine that the token has actually expired, but less than a minute has elapsed. If something calls an API function that needs the token that function would not work (and show the x-installationNumber issue). In init.py:

    if "exp" in token:
        expiration: datetime = datetime.fromtimestamp(token["exp"])
        if datetime.now() + timedelta(minutes=1) > expiration:
            # if the token is expired get a new one
            await self.get_services(installation)

Getting rid of the extra minute is not going to solve all the problems, but it will help.

I think you could hack a fix moving the logic that checks if the token is expired to the apimanager in a way similar like how you handle the login expiring.

So, what is your suggestion then? When you mean getting rid of the extra minute, what is exactly that? thanks @cantupaz

[cantupaz]

I'm tired and I realize my previous message was not clear. I was misreading the condition in the if statement, but I still think you could try moving:

    token = jwt.decode(
        installation.capabilities,
        algorithms=["HS256"],
        options={"verify_signature": False},
    )
    if "exp" in token:
        expiration: datetime = datetime.fromtimestamp(token["exp"])
        if datetime.now() + timedelta(minutes=1) > expiration:  
            # if the token is expired get a new one
            await self.get_all_services(installation)

to a function in apimanager.py and then call that function from the other functions in API manager where it makes sense (check_alarm, etc). See: https://github.com/cantupaz/securitas-direct-new-api/blob/exceptions/custom_components/securitas/securitas_direct_new_api/apimanager.py

I don't know why, but I'm not seeing the errors others are reporting in the HA integration. I can force the error calling the API directly, but not in HA.

hope this helps

[samatild]

Adding the timer + reload doesn't seem to work

Logger: homeassistant.helpers.entity
Source: helpers/entity.py:698
First occurred: 8:08:32 AM (42 occurrences)
Last logged: 9:28:32 PM

Update for alarm_control_panel.<redacted > fails
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 698, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 959, in async_device_update
    await self.async_update()
  File "/config/custom_components/securitas/alarm_control_panel.py", line 327, in async_update
    alarm_status: CheckAlarmStatus = await self.client.update_overview(
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/securitas/__init__.py", line 395, in update_overview
    token = jwt.decode(
            ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jwt.py", line 210, in decode
    decoded = self.decode_complete(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jwt.py", line 151, in decode_complete
    decoded = api_jws.decode_complete(
              ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jws.py", line 198, in decode_complete
    payload, signing_input, header, signature = self._load(jwt)
                                                ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jws.py", line 254, in _load
    raise DecodeError(f"Invalid token type. Token must be a {bytes}")
jwt.exceptions.DecodeError: Invalid token type. Token must be a <class 'bytes'>

For reference... here's my arm/disarm script

alias: securitas_armdisarm
sequence:
  - if:
      - condition: device
        device_id: <redacted>
        domain: alarm_control_panel
        entity_id: <redacted>
        type: is_disarmed
    then:
      - device_id: <redacted>
        domain: alarm_control_panel
        entity_id: <redacted>
        type: arm_away
        enabled: true
    else:
      - if:
          - condition: device
            device_id: <redacted>
            domain: alarm_control_panel
            entity_id: <redacted>
            type: is_armed_away
        then:
          - device_id: <redacted>
            domain: alarm_control_panel
            entity_id: <redacted>
            type: disarm
            code: "<redacted>"
  - delay:
      hours: 0
      minutes: 0
      seconds: 4
      milliseconds: 0
  - service: homeassistant.reload_config_entry
    target:
      entity_id:
        - alarm_control_panel.<redacted>
icon: mdi:shield-home-outline
mode: single

[KoolLSL]

@samatild, this workaround doesn't prevent some error messages but allows to have the correct updated alarm status after each arming/disarming. In addition, as the issue happens after some hours, I put an automation to reload every 3 hours. This time is OK for my situation, there are surely other ways to do it. It seems it needs a reload not too long before the next arming/disarming, and also after to get the updated status:

trigger:
  - platform: time_pattern
    hours: /3
condition: []
action:
  - service: homeassistant.reload_config_entry
    target:
      entity_id: alarm_control_panel.securitas_home
    data: {}

[RenZan]

Happy new year to everyone :) . Any progress by anyone on this subject ? ^^

[KoolLSL]

New v2.6.0.0 seems to work very well since installed 3 days ago (without the reload_config_entry workaround for v2.5). There are no errors or warnings in logs. Many thanks to @guerrerotook and @cantupaz.

[RenZan]

I agree with @KoolLSL, everything seems to be working good now. Thanks a lot @cantupaz and @guerrerotook

[guerrerotook]

I think in here @cantupaz is the hero that did all the work! Thanks for that.

[samatild]

well, a huge thanks in advance 👍 very well done. I can confirm it's working like it was before Cheers @cantupaz and @guerrerotook

[cantupaz]

well, let's hope Securitas doesn't change their API again....

I think in here @cantupaz is the hero that did all the work! Thanks for that.

I did the last 5% on top of all the work of @guerrerotook