Discovered again?

[dennidoubles]

I received a discovered device in HA:

Are you aware of this and why is this, Inhad it already installed and working?

[arjenbos]

Hi @dennidoubles ,

This is really odd, I didn't add any support for discovery. Which version of HA are you using? Maybe there is an issue on the backlog of HA which is related. For now, I do not know how to fix this.

[dennidoubles]

Running 2024.1 Will check the log if I can find anything. Strange, right?

[dennidoubles]

Well, I reconfigured and it is working again. Still strange it got discovered again and needed a re-config, I've heard it also from others. Thanks anyway!

[arjenbos]

This is really odd, since I do not do anything other than following the guides from HA I do not think it's the integration, but HA that does this strange behaviour. I will close this issue for now, but if it happens again then I will re-open it.

[dennidoubles]

Well, another discovered notification came in today Arjen:

Normally I could just ignore it, but the result is that my card isn't working anymore:

I work with version 1.2.0 and the following:

[stijnb1234]

Well, another discovered notification came in today Arjen:

Normally I could just ignore it, but the result is that my card isn't working anymore:

I work with version 1.2.0 and the following:

Same for me

[stijnb1234]

Maybe this is helpful.

Error fetching PostNL data: Cannot connect to host login.postnl.nl:443 ssl:default [Try again]
Error fetching PostNL data: Expecting value: line 1 column 1 (char 0)
Error fetching PostNL data: 500, message='Internal Server Error', url=URL('https://login.postnl.nl/xxxxxx/login/token')

[arjenbos]

This certainly helps; I will try to see if I can reproduce it.

[arjenbos]

I tested a couple of things. I am not a experienced Python developer, for me this is a hobby. I may have some strange things in my code that are against the principles of Python. So, if people see some strange code in my upcoming changes, please create a PR 😄 .

It's a combination of things that's happening:

1. In the error log you see that PostNL returned a 500 Internal Server Error. This indicates that their API is broken. That's something I would like to solve, but can't 😅 .
2. That your Lovelace card isn't working is intended behavior if I follow examples from official integrations. If data isn't returned by external API, then the integration must throw an UpdateFailed exception. Due to this exception, the entity is unavailable.
3. Handling exceptions isn't done properly in the integration. Due to this it broke completely.
4. The lovelace isn't able to handle an entity which is unavailable, due to this you see the error cannot convert undefined or null to object in the lovelace. This error is raised in Javascript/Typescript.
5. I cannot reproduce the Discovery part. The integration doesn't support this at all and I do not know where to start.

I made some changes and I am kindly asking you if you can test it.

1. The integration now catches more exceptions. Hopefully this will make sure that the integration will try again after it broke down.
2. I made a small change in getting the data. If a package is delivered, then I do not call jouw.postnl.nl anymore. This will reduce the number of calls with a lot. For me this went down from 40 to 1. And, 0 if I do not an incoming package.

The changes can be inspected in the linked PR. I will merge PR into the main branch. Please checkout this branch via HACS.

[AarvdH]

It's not really a discovery. I have the same thing, but it happens if a restore a backup. It says two things, " Discovered "and " Reconfigure " So I have this with your PostNL integration but also with my FritzBox smarthome integration, Then it loses a repeater, says, Discovered, and the same reconfigure, I only must put in the password of that repeater, and all is fine again. Maybe it helps you to find what is happening.

[dennidoubles]

One week later, I experienced it again today.

If I can help you with more info, just let me know. I don't see anything in my logs, which is even more strange ...

[arjenbos]

One week later, I experienced it again today.

If I can help you with more info, just let me know. I don't see anything in my logs, which is even more strange ...

Could you please check if you have multiple credentials (/config/application_credentials) for the PostNL integrations?

[FeikoJoosten]

I'm pretty sure that it's not really discovering the integration, but that it's failing to connect to the API causing someone to have to reconfigure the integration before it works again. I think that it's caused by the authentication token being expired causing the integration to fail to connect to the post nl servers.

For me it seems that I have to reconfigure weekly, which would somewhat match with an expiration date for these kind of tokens. @arjenbos I'm not sure about the post nl api, but normally if the API returns a refresh token, that token should be used to create a new authentication token.

Regarding your question for the application_credentials. This is what's stored for me:

{
    "id": "postnl",
    "domain": "postnl",
    "client_id": "-",
    "client_secret": "-",
    "name": "HomeAssistant PostNL"
}

[stijnb1234]

I'm pretty sure that it's not really discovering the integration, but that it's failing to connect to the API causing someone to have to reconfigure the integration before it works again. I think that it's caused by the authentication token being expired causing the integration to fail to connect to the post nl servers.

For me it seems that I have to reconfigure weekly, which would somewhat match with an expiration date for these kind of tokens. @arjenbos I'm not sure about the post nl api, but normally if the API returns a refresh token, that token should be used to create a new authentication token.

Regarding your question for the application_credentials. This is what's stored for me:

{
    "id": "postnl",
    "domain": "postnl",
    "client_id": "-",
    "client_secret": "-",
    "name": "HomeAssistant PostNL"
}

Yup, it's not a discovery, but it gives an error and then HomeAssistants decides it should be reconfigured.

[arjenbos]

I'm pretty sure that it's not really discovering the integration, but that it's failing to connect to the API causing someone to have to reconfigure the integration before it works again. I think that it's caused by the authentication token being expired causing the integration to fail to connect to the post nl servers.

For me it seems that I have to reconfigure weekly, which would somewhat match with an expiration date for these kind of tokens. @arjenbos I'm not sure about the post nl api, but normally if the API returns a refresh token, that token should be used to create a new authentication token.

Regarding your question for the application_credentials. This is what's stored for me:

{
    "id": "postnl",
    "domain": "postnl",
    "client_id": "-",
    "client_secret": "-",
    "name": "HomeAssistant PostNL"
}

That's indeed default oAuth2 behaviour which is developed and controlled by Home Assistant core. I will have another look and see if something goes wrong in combination with the access_token and refresh_token.

[arjenbos]

I think I found it. This behaviour is initiated due to a ConfigEntryAuthFailed exception. I cannot reproduce why you guys hit this exception.

The exception was thrown when the API returned an HTTP response status code between 400 and 500. I changed the code in the main branch to throw a HomeAssistantError exception instead and I added some debug logging. Hopefully we can figure out why the PostNL API returns an error code between the 400 and 500.

@stijnb1234 @FeikoJoosten could you please pull the latest version of the main branch and enable debug logging?

[stijnb1234]

I think I found it. This behaviour is initiated due to a ConfigEntryAuthFailed exception. I cannot reproduce why you guys hit this exception.

The exception was thrown when the API returned an HTTP response status code between 400 and 500. I changed the code in the main branch to throw a HomeAssistantError exception instead and I added some debug logging. Hopefully we can figure out why the PostNL API returns an error code between the 400 and 500.

@stijnb1234 @FeikoJoosten could you please pull the latest version of the main branch and enable debug logging?

Installed the main branch version again, will let you know when I have anything.

[FeikoJoosten]

Same here!

[dennidoubles]

Don't understand what to do next, so I guess I'll wait for your feedback

[arjenbos]

No feedback received, so I assume it's fixed?

[FeikoJoosten]

So far no issues on my end. Seems to work like a charm

[dennidoubles]

Is the new version a fix for this issue? That would great work!

[stijnb1234]

No feedback received, so I assume it's fixed?

Same for me, no issues.

[arjenbos]

Nice, the latest release includes the fixes.