zachowj / xfinity-data-usage

Fetch Xfinity data usage and serve it via an HTTP endpoint, publish it to MQTT or post it to an URL.
https://hub.docker.com/r/zachowj/xfinity-data-usage
MIT License
53 stars 15 forks source link

Access Denied when trying to login #337

Closed yuejon closed 2 weeks ago

yuejon commented 2 months ago

The latest versions have been working flawlessly for me since the release of version 0.23.0 but it started failing yesterday. Looks like Comcast is doing something new to prevent scraping:

2024-08-24 14:10:55 [debug] : Navigating to starting page
2024-08-24 14:11:01 [debug] : Current Page / Title: XFINITY | My Account | EcoBill® Online Bill Pay / URL: https://customer.xfinity.com/#/devices#usage
2024-08-24 14:11:01 [debug] : Filling in username
2024-08-24 14:11:08 [debug] : Current Page / Title: Access Denied / URL: https://login.xfinity.com/login
2024-08-24 14:11:08 [debug] : Filling in password

I would also get the same message when trying to manually login through an incognito session and it would show the access denied message right after entering the username. Is anyone else facing the same problem?

tfer6 commented 2 months ago

Same thing here. Worked perfectly until a day or two ago. Same error message and same experience with incognito mode.

pmetzke commented 2 months ago

Same. Mine stopped on the 23rd.

zachowj commented 2 months ago

Yes, they are blocking bots with Akamai based on the error page.

yuejon commented 2 months ago

There seems to be a cooldown period. I left the container off the last few days and would start it up occasionally to test it. Anyway, I just tried it again and it worked. I'll probably adjust my refresh from 30 minutes to something longer and hopefully that will be enough to prevent getting block again.

yuejon commented 2 months ago

Welp... I set a 3 hour refresh interval but got blocked again.

zachowj commented 2 months ago

Please try the latest release and let me know how it performs. At this stage, it's becoming a bit of a cat-and-mouse game.

tfer6 commented 2 months ago

This one worked for me! Thank you!

yuejon commented 2 months ago

Latest version so far has pulled data successfully twice but I do have it set at a 6 hour interval. Understand it's a cat-and-mouse game and we appreciate your efforts at evading the fat cat.

naterator commented 2 months ago

Mine is also set to a six-hour interval, and has been pulling successfully for ~24 hours. Thanks @zachowj for your work on this!

tfer6 commented 2 months ago

Spoke too soon. Worked for couple of days but then ran into the access denied page. I let it run another day to no avail. Going to try leaving it off for a few days before trying again.

Appreciate your efforts. It's infuriating that this much has to be done just to track our own data usage.

naterator commented 1 month ago

Unfortunately, my instance has been failing to grab any Xfinity data at all for the last ~7 days (since 9/15). What is everybody else seeing?

pmetzke commented 1 month ago

Unfortunately, my instance has been failing to grab any Xfinity data at all for the last ~7 days (since 9/15). What is everybody else seeing?

Same experience :(

yuejon commented 1 month ago

Yeap. Mine hasn't updated in the last 10 days and I'm resigned that there may be no way to get around Comcast's blocks. Since I have an Asus router, I've resorted to using the Asus Router integration to track WAN traffic along combined with the Utility Meter helper in Home Assistant to get estimates on monthly usage.

zachowj commented 1 month ago

I’m currently unable to address the issues with this repository, but I recommend checking out the Xfinity Hassio Add-On, which has been updated to run as a standalone Docker container and continues to fetch usage data.

I also requested support for dumping raw usage data to an MQTT topic for more detailed insights beyond the current month. This functionality has been added in the development branch.

Below is the Docker Compose configuration I’m using to run the container:

xfinity:
  image: ghcr.io/thor0215/hassio-xfinity-usage-amd64:latest
  container_name: xfinity
  entrypoint: /bin/sh -c "python3 -Wignore /xfinity_usage_addon.py"
  restart: unless-stopped
  environment:
    TZ: America/Los_Angeles
    XFINITY_USERNAME: <username>
    XFINITY_PASSWORD: <password>
    MQTT_SERVICE: true
    MQTT_HOST: <mqtt_ip>
    MQTT_PORT: 1883
    MQTT_RAW_USAGE: true
    LOG_LEVEL: info
    POLLING_RATE: 3600
  shm_size: 1gb