search

iMicknl / python-postnl-api

Python wrapper for the PostNL API, a way to track packages using their online portal.
MIT License
21 stars 5 forks source link

Access Denied #20

Open icepick3000 opened 4 years ago

icepick3000 commented 4 years ago

I saw that an update was posted here.. but i still get the same error.. 

_LOGGER.debug(f"Access denied. Failed to refresh, attempt {count} of {max}.")

I read reports this fixes the problem for some users. I wonder if they blocked my IP or something.. but on the other hand.. it still works from my smartphone.

iMicknl commented 4 years ago

Which version of the library are you using?

icepick3000 commented 4 years ago

I am using 1.2.3... i updated today as soon as you posted it..

iMicknl commented 4 years ago

Currently I am not able to do extensive testing. Could you have a look at the exact error your receive? What if you do the curl request.

0x00-0xFF commented 4 years ago

Fix in: https://github.com/iMicknl/python-postnl-api/pull/22

peternijssen commented 4 years ago

I had a call with PostNL about the issue. Unfortunately they have no solution for us yet in the near future and after talking with them, we came to the conclusion to remove PostNL as a component from HA. See my message over here: https://community.home-assistant.io/t/lovelace-postnl/112433/281

iMicknl commented 4 years ago

@peternijssen is #22 not fixing this? As an unofficial work around?

peternijssen commented 4 years ago

@iMicknl I asked PostNL if there was a work around that we could use, even the web login and they said no. On the other hand, HA changed some rules and they don't like components that break all the time, which kinda happened lately to PostNL.

As I understood, either the web login has or will have also some bot protection.

peternijssen commented 3 years ago

I had a conversation with PostNL again as people still request a HA integration on a regular basis. Basically the answer is that there is still no official solution, but we can use the web login from now on as suggested here earlier. The only requirement is that we do max once an hour a request, to avoid being seen as a bot.

In that essence, we can update this library and create a custom component for HA and I can update the Lovelace card. Anyone up to this?

iMicknl commented 3 years ago

I am happy to update the Python library and make it async in the coming weeks, however I am not sure about the current behaviour of PostNL. If they are against it and actively block the requests, it will be a cat-and-mouse game.

peternijssen commented 3 years ago

Agree. This was the response from the head of digital development when I asked about to use the web login; 1x per uur zie ik niet direct een issue in, zal het nog even nalopen. Zeker niet als de toegang via het postnl account verloopt.

1ntroduc3 commented 3 years ago

Following this work with eager :-)

DieterKoblenz commented 3 years ago

Following too

ChefkeGremmen commented 3 years ago

Yup, ik volg ook (following as well ;-) )

iMicknl commented 3 years ago

I have had a look this week, however it will be hard to build and maintain if they don't give us access... There current login mechanism is well protected, and I don't feel like scraping their website for the tokens and than simulate the request + catching callback.

ChefkeGremmen commented 3 years ago

Using your own credentials doesn't work? (sorry, for maybe my lack of knowledge)

IcyPalm commented 3 years ago

I am looking into this as well, rooted an android phone and intercepted all the traffic. It is not just the credentials, you initially send the credentials and then you get a token back, then you need to store the token, refresh the token once in a while(and that is simplyfing it quite a bit). It is quite some flow.

Especially since were are NOT using a official and described API but a self-discovered way of working with the data it is not as simple as just implementing an API. First discover how it works, trying to mimic it with a self-written script and then when it finally works you need to pray that PostNL doesn't change their flow because then the process starts all over again for us.

ChefkeGremmen commented 3 years ago

ah, clear. That is indeed pretty cumbersome. Thanks for the explanation!

iMicknl commented 3 years ago

Indeed, @IcyPalm. Their API is pretty nice and even has a GraphQL endpoint nowadays, however this will always be based on reverse engineering.

It seems that they are using some kind of OAuth flow for web + mobile. I tried reverse engineering it, but you are not allowed to set any redirect URL, which will even complicate it more. So for now, the only possibility I would see is to have an OAuth application from them, with the Nabu Casa redirect url.

IcyPalm commented 3 years ago

OAuth via Nabu Casa is not going to happen as long as it's not an official API though 😢

web+Mobile API looks pretty nice as well, was quite some ways figuring it out but then my test-android crashed so need to set that one up again.

For now I really do not see a realistic option for creating a (semi) stable implementation. I wished PostNL just created a (very simple) API

peternijssen commented 3 years ago

I've reached out to PostNL and pointed them to this thread.

peternijssen commented 3 years ago

So far no response unfortunately :(

umutcelebi commented 3 years ago

Ik volg en ben zeer benieuwd wat er uit gaat komen....

Melantrix commented 3 years ago

For now I really do not see a realistic option for creating a (semi) stable implementation. I wished PostNL just created a (very simple) API

This would be great. They have an api already, if they could open it up to be able to get the deliveries from your own account and nothing more, it would be golden. Even with a rate limit like once an hour it would still be a much better solution then reverse engineering

bliekp commented 2 years ago

Would be totally awesome if PostNL opened up and lets third party developers create cool things with their API. Come on PostNL, step into the 21st century! :)

bafplus commented 2 years ago

Any news yet?

K4CZP3R commented 2 years ago

I've reverse engineered their iOS app, I'm able to perform login action and get the token needed for API calls.

They changed the authentication flow a lot (they added bot detection too), so raw requests to the authentication endpoint are easily detected by PostNL and are bot-flagged.

There are two solutions I've came up with;

  1. Use selenium (needs to be non-headless: they can detect it too), and then extract the postnl://code=x to get the access and refresh token.
  2. Use send login mail, in this solution there is no need to use selenium, running GET request on the magic link, returns the postnl://code=x without any problems.

First solution sucks, it requires the computer where this library runs, to have graphics capabilities, the second one needs to be able to access your mail inbox to get the magic link.

I will try to find something better but for now, that's all I've got.

Both methods are PITA, but after the initial login, you get the refresh token so when your access token expires, you only need to obtain a new one using refresh token (so, there is no need to do selenium/email thing again)

jimz011 commented 1 year ago

I can't believe that after years of complaining at PostNL that they still haven't opened up this API. We should probably just create some bot that throws requests at them like crazy until they decide that opening up the API is probably a better option.

peternijssen commented 9 months ago

FYI, there is a new home assistant component: https://github.com/arjenbos/ha-postnl