Doesn't work, did TGTG change something?

[DavyRoswinkel]

Add-on version: 1.8.0 You are running the latest version of this add-on. System: Home Assistant OS 6.6 (aarch64 / raspberrypi4-64) Home Assistant Core: 2021.11.2 Home Assistant Supervisor: 2021.10.8

Log: [cont-init.d] 00-banner.sh: exited 0. [cont-init.d] 01-log-level.sh: executing... [cont-init.d] 01-log-level.sh: exited 0. [cont-init.d] done. [services.d] starting services [services.d] done. [17:08:48] INFO: Copy setings file [17:08:48] INFO: Strating Bridge 2021-11-09 17:08:56 1e14b3fb-tgtg-ha-mqtt-bridge main[308] INFO Connecting mqtt 2021-11-09 17:08:56 1e14b3fb-tgtg-ha-mqtt-bridge main[308] INFO Starting loop 2021-11-09 17:08:56 1e14b3fb-tgtg-ha-mqtt-bridge main[308] DEBUG Loop run started Exception in thread Thread-3: Traceback (most recent call last): File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/usr/local/lib/python3.8/threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "toogoodtogo_ha_mqtt_bridge/main.py", line 158, in loop if not check(): File "toogoodtogo_ha_mqtt_bridge/main.py", line 33, in check shops = tgtg_client.get_items(page_size=400) File "/usr/local/lib/python3.8/site-packages/tgtg/init.py", line 143, in get_items self._login() File "/usr/local/lib/python3.8/site-packages/tgtg/init.py", line 122, in _login raise TgtgLoginError(response.status_code, response.content) tgtg.exceptions.TgtgLoginError: (403, b'

Please enable JS and disable any ad blocker

\n')

Didn't they change the way you login? In their app you don't need a password, you will get a login link. Maybe that is what caused this issue, but I could be wrong.

[MaxWinterstein]

Yeah, seems like to only allow the email flow with this "two factor" style :(

Already started a discussion at the package I use: https://github.com/ahivert/tgtg-python/issues/119

[DavyRoswinkel]

Yeah, seems like to only allow the email flow with this "two factor" style :(

Already started a discussion at the package I use: https://github.com/ahivert/tgtg-python/issues/119

A difficult puzzle...😲would love to see a solution..but fear...the fun is over. Like this addon very much!

[MaxWinterstein]

Yeah, seems like to only allow the email flow with this "two factor" style :( Already started a discussion at the package I use: ahivert/tgtg-python#119

A difficult puzzle...😲would love to see a solution..but fear...the fun is over. Like this addon very much!

Hopefully, someone is able to reverse engineer the authentification flow. Sadly not my main field of knowledge. But as this hurts all projects that used the api, regardless the language used, I hope there is someone out there with enough power and will to solve this 😉

[poudenes]

https://github.com/Chouffy/home_assistant_tgtg/issues/1

other GitHub user with same issue. there was a node-js module posted with a fix. Maybe you can look to it?

[MaxWinterstein]

Chouffy/home_assistant_tgtg#1

other GitHub user with same issue. there was a node-js module posted with a fix. Maybe you can look to it?

as far as i see this only fixes the fetch of items, not the login process.

[poudenes]

and this?

https://github.com/marklagendijk/node-toogoodtogo-watcher/issues/112

[MaxWinterstein]

This was closed by the mentioned fix of the node-js module, afaik no fix for login, only for fetching items by adding some header.

[poudenes]

:(

[poudenes]

Then my last option:

https://github.com/custom-components/alexa_media_player/wiki

Those guys use 2FA also to connect to Amazon so your Alexa will work in HA. Maybe you can check their code, or contact them how they did it?

[liudger]

https://github.com/ahivert/tgtg-python/releases/tag/0.7.0 has a fix

[MaxWinterstein]

https://github.com/ahivert/tgtg-python/releases/tag/0.7.0 has a fix

Yes, yay 🥳

Currently there is an PR by @dielee in progress (https://github.com/MaxWinterstein/toogoodtogo-ha-mqtt-bridge/pull/19), seems like this will be solved soon.

[MaxWinterstein]

I updated the bridge, and also the addon.

As i am still blocked at TooGoodToGo, pls have a test :)

[jablonski-pawel]

Hi! Many thanks for @MaxWinterstein for an fast update! I've updated to the 1.9.0, but still I have this same problem with captcha 😭 What am I doing wrong? I had reinstall add-on, and nothing changed 😢 On phone TGTG app it is working fine after check Captcha.

[DavyRoswinkel]

I updated the bridge, and also the addon.

As i am still blocked at TooGoodToGo, pls have a test :)

Thanks! The first run, showed unknown for the stock of all shops. Te second run...it shows the correct numbers of stock. It looks like it works again. Will watch it closely this morning.

Thanks again for solving this issue🥳🍾🙋‍♂️

[poudenes]

Here updated as well. Log looks Good.

Within first start I see the portions and pickup time coming back. Didn't start twice and no block.

Thanks for the fast and great fix!!

[Dielee]

I updated the bridge, and also the addon.

As i am still blocked at TooGoodToGo, pls have a test :)

Maybe it is possible to fetch the captcha file ? I would have a look into this, but I don't have this type of error...

[poudenes]

I updated only the Addon and start it. Update bridge? That's inside the the Addon right?

Here its working

[Dielee]

Yes, for most of us, also me, its working in 1.8.0. But some people are getting an captcha request.. From app or in the email, I don't now at this time.

[MaxWinterstein]

Maybe worth mentioning, as I guess I am not the only one:

My dev machine, and also my iPhone, is ipv6 compatible and uses it to access TGTG and their services.

The Addon, or any docker container without explicit config I guess, uses ipv4 to communitcate.

---

@Dielee I think solving the captcha and passing the response cookie around should be the solution for this, but I am not sure how to archive this.

[unixfox]

The Addon, or any docker container without explicit config I guess, uses ipv4 to communitcate.

NodeJS use ipv4 by default even if IPv6 is available.

[MaxWinterstein]

The Addon, or any docker container without explicit config I guess, uses ipv4 to communitcate.

NodeJS use ipv4 by default even if IPv6 is available.

oh, at least that explains why I wasn't able to get the nodejs version to run also

[DavidDeSloovere]

Version 1.9.0 of the add-on works for me! Thanks. Already have dinner for tonight.

[unixfox]

The Addon, or any docker container without explicit config I guess, uses ipv4 to communitcate.

NodeJS use ipv4 by default even if IPv6 is available.

oh, at least that explains why I wasn't able to get the nodejs version to run also

If you are interested in making nodejs working with ipv6 then take a look at this github issue: https://github.com/nodejs/node/issues/6307

[Dielee]

@MaxWinterstein is there also an link inside the mail ? Maybe the using an redirect url, like the most oauth. Then, there is an way to archive this.

[MaxWinterstein]

The Addon, or any docker container without explicit config I guess, uses ipv4 to communitcate.

NodeJS use ipv4 by default even if IPv6 is available.

oh, at least that explains why I wasn't able to get the nodejs version to run also

If you are interested in making nodejs working with ipv6 then take a look at this github issue: nodejs/node#6307

oh no way, neither do I speak nodejs, nor do I have any interest in ipv6 😄

---

@MaxWinterstein is there also an link inside the mail ? Maybe the using an redirect url, like the most oauth. Then, there is an way to archive this.

I am not sure if we talk about the same thing? The Captcha is not linked to any mail?

I played around with a flow already, see (see https://github.com/marklagendijk/node-toogoodtogo-watcher/issues/112#issuecomment-966694994)

[Dielee]

Maybe let's talk about this in a chat ? Gitter or Discord ?

[DavyRoswinkel]

And it stopped working for me again. Had to reboot HA. And then it stopped working with: [cont-init.d] 00-banner.sh: exited 0. [cont-init.d] 01-log-level.sh: executing... [cont-init.d] 01-log-level.sh: exited 0. [cont-init.d] done. [services.d] starting services [services.d] done. [13:36:20] INFO: Copy setings file [13:36:20] INFO: Strating Bridge 2021-11-17 13:36:21 1e14b3fb-tgtg-ha-mqtt-bridge main[308] INFO Connecting mqtt 2021-11-17 13:36:21 1e14b3fb-tgtg-ha-mqtt-bridge main[308] INFO Starting loop 2021-11-17 13:36:21 1e14b3fb-tgtg-ha-mqtt-bridge main[308] DEBUG Logging in with credentials Exception in thread Thread-3: Traceback (most recent call last): File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/usr/local/lib/python3.8/threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "toogoodtogo_ha_mqtt_bridge/main.py", line 202, in loop tgtg_client.login() File "/usr/local/lib/python3.8/site-packages/tgtg/init.py", line 126, in login raise TgtgLoginError(response.status_code, response.content) tgtg.exceptions.TgtgLoginError: (404, b'')

I think the end is close...sad🥺

[MaxWinterstein]

tgtg.exceptions.TgtgLoginError: (404, b'')

encouraged the 404 here, too. Very strange.

[Dielee]

Maybe it is possible to use an Proxy to connect to tgtg servers ? I would like to open an new branch for this.

[MaxWinterstein]

Maybe it is possible to use an Proxy to connect to tgtg servers ? I would like to open an new branch for this.

Had some play with https://github.com/jundymek/free-proxy - that went so so... Most of those IPs are already 'blocked'.

As the 404 issue now seems to persist I might have a try with the email verification flow (already build a tgtg-python here: https://github.com/MaxWinterstein/tgtg-python ), will see if I find some time

[DarKOrange75]

Hi all, I have updated to 1.9.0 but I still get the captcha error :-(

2021-11-19 19:44:42 f1c878cb-tgtg-ha-mqtt-bridge main[309] DEBUG Logging in with credentials Exception in thread Thread-3: Traceback (most recent call last): File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/usr/local/lib/python3.8/threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "toogoodtogo_ha_mqtt_bridge/main.py", line 202, in loop tgtg_client.login() File "/usr/local/lib/python3.8/site-packages/tgtg/init.py", line 126, in login raise TgtgLoginError(response.status_code, response.content) tgtg.exceptions.TgtgLoginError: (403, b'

Please enable JS and disable any ad blocker

\n')

[Dielee]

@MaxWinterstein whats the response from the tgtg servers after you recive the email with code and link ? Is there an captcha url inside ? I would like to inspeckt this, but my tgtg polling works fine without captcha...

[MaxWinterstein]

@MaxWinterstein whats the response from the tgtg servers after you recive the email with code and link ? Is there an captcha url inside ? I would like to inspeckt this, but my tgtg polling works fine without captcha...

The login request itself is blocked and responded with the captcha.

If you have rotating IPs, you might try to get captcha required on purpose. In my last test i simply hit the login endpoint pretty hard with a while loop.

[Dielee]

I have no rotating ips, so blacklisting my IP is not an option. Do you tink it is possible to fetch the captcha webseite url and print it in the log ? If this is possible, we can pass the captcha maybe.

[DarKOrange75]

I don't know if it is related but this morning I opened the app and I was asked to reauthenticate and verify my email. The problem is that I never receive the verification email so I am blocked...

Edit:

now I am banned from the app : "too much connection attempts with this email, try again later"

[MaxWinterstein]

I have no rotating ips, so blacklisting my IP is not an option. Do you tink it is possible to fetch the captcha webseite url and print it in the log ? If this is possible, we can pass the captcha maybe.

It already is printed here, check @DarKOrange75 post a little bit up there. The response to the blocked request is some 403 with a minimal HTML website as content. I was not able to get it up by simply pasting it in some foo.html and serving it locally.

---

now I am banned from the app : "too much connection attempts with this email, try again later"

Been there, its some kind of rate limiter. Give it a few hours to cool down.

[Dielee]

I would like to play a bit with this response, but there is no way for me to do this.

[DavidDeSloovere]

Just to confirm @DarKOrange75 - my 1.9.0 integration is also dead again

[liudger]

https://github.com/ahivert/tgtg-python/releases/tag/0.8.0

[liudger]

https://github.com/ahivert/tgtg-python/releases/tag/0.10.0

[MaxWinterstein]

I merged the latest PR from @Dielee, now this plugin uses tgtg==0.10.0.

It even works again for me with ipv4 and ipv6!

@DavyRoswinkel can you have a try? (docker image 1.9.1 should be available in a few minutes)

[DavyRoswinkel]

In the addon store I keep seeing version 1.9.0.

Can you tell me how to update...to docker image 1.9.1?

[MaxWinterstein]

Ah, did not see you are using the add-on, my fault.

Need to adjust it a little bit so the now saved tokens will persist across restarts.

Sadly @Dielee did not provide a function to declare the storage position... 😙🎶

[MaxWinterstein]

I quickly thew out version 1.9.2 - now both add-on and bridge itself use the same versioning.

Please have a try :v:

[DavyRoswinkel]

It logs me in.... It connects to TGTG makes the entities... But it doesn't give the state/attribute info of the entities. State is unknown

The second run of the loop it gives the full info.

Looks like it works again!!!!! Very happy!

Thanks alot!!

[liudger]

Could it be that if you failed the first login that it doesn't update the tokens if you try again? (New mail code is send)

[poudenes]

Skip my comment. I didnt read correct. I received emails :)

[jablonski-pawel]

I started the addon received the email from TGTG I clicked the link in the email

and it worked

I had this same. It have to click this button on email, when on LOGs you see "DEBUG Logging in with credentials". Time is a little short - about 1 minute. After that everything works!! It don't need to do this again after reboot HA machine. @MaxWinterstein Thank you!! Great Job!

[poudenes]

Installed a clear addon. Did action on email. And then i get a error:

s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 00-banner.sh: executing... 
-----------------------------------------------------------
 Add-on: TooGoodToGo Home Assistant MQTT Bridge
 Publish TooGoodToGo stock as MQTT messages
-----------------------------------------------------------
 Add-on version: 1.9.2
 You are running the latest version of this add-on.
 System: Home Assistant OS 6.6  (aarch64 / raspberrypi4-64)
 Home Assistant Core: 2021.11.5
 Home Assistant Supervisor: 2021.10.8
-----------------------------------------------------------
 Please, share the above information when looking for help
 or support in, e.g., GitHub, forums or the Discord chat.
-----------------------------------------------------------
[cont-init.d] 00-banner.sh: exited 0.
[cont-init.d] 01-log-level.sh: executing... 
[cont-init.d] 01-log-level.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[13:31:35] INFO: Copy setings file
[13:31:35] INFO: Strating Bridge
2021-12-03 13:31:37 f1c878cb-tgtg-ha-mqtt-bridge __main__[309] INFO Connecting mqtt
2021-12-03 13:31:37 f1c878cb-tgtg-ha-mqtt-bridge __main__[309] INFO Starting loop
2021-12-03 13:31:37 f1c878cb-tgtg-ha-mqtt-bridge __main__[309] DEBUG Logging in with credentials
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "toogoodtogo_ha_mqtt_bridge/main.py", line 200, in loop
    tgtg_client.login()
  File "/usr/local/lib/python3.8/site-packages/tgtg/__init__.py", line 139, in login
    self.start_polling(first_login_response["polling_id"])
  File "/usr/local/lib/python3.8/site-packages/tgtg/__init__.py", line 179, in start_polling
    raise TgtgPollingError(
tgtg.exceptions.TgtgPollingError: Max retries (120 seconds) reached. Try again

[MaxWinterstein]

Think you must be fast clicking that email, like @jablonski-pawel said. Saw that message too at one point.
Did you click via brower/mail client or at the smartphone were TGTG is also installed?

I might drop some big note and that part.