Not returning data due to error

[thiagobruch]

Hi,

First I would like to thank you for this project. I was looking for something like this for a while and I had to always resort to look at the Xfinity App in my phone.

I was not able to get it work on my end, for some reason, it appears that is not able to pull the data and generates the following error:

2024-07-24T23:21:19.422 DEBUG: Updating Session Details https://api.sc.xfinity.com/session
2024-07-24T23:21:19.422 DEBUG: Updating Session Details is_session_active: True
2024-07-24T23:21:19.422 DEBUG: Updating Session Details session time left: 899 seconds
2024-07-24T23:21:19.422 DEBUG: Updating Session Details {"sessionType": "FULL", "exp": 1721878578, "tokenExp": 1721881280, "sessionId": "49409e18-745a-4607-be0d-874680aedc2e"}
2024-07-24T23:21:20.476 DEBUG: Loading page (URL: https://www.xfinity.com/learn/internet-service/auth)
2024-07-24T23:21:22.924 INFO: Updating Plan Details
2024-07-24T23:21:22.924 DEBUG: Updating Plan Details {"InternetDownloadSpeed": 1000, "InternetUploadSpeed": 20}
2024-07-24T23:21:25.351 INFO: Updating Usage Details
2024-07-24T23:21:25.359 DEBUG: Updating Usage Details {"accountNumber": "
{{{{{ I edited and remove some of the usage details here but left part of it below }}}}}
{"policyName": "Unlimited Data Plan", "startDate": "07/01/2024", "endDate": "07/31/2024", "homeUsage": 2249, "wifiUsage":
 0, "totalUsage": 2249, "allowableUsage": 100000, "unitOfMeasure": "GB", "displayUsage": false, "devices": [], 
"additionalBlocksUsed": 0, "additionalCostPerBlock": 0, "additionalUnitsPerBlock": 0, "additionalBlockSize": 0, 
"additionalIncluded": 0, "additionalUsed": 0, "additionalPercentUsed": 0.0, "additionalRemaining": 0, "billableOverage": 0, 
"overageCharges": 0.0, "overageUsed": 0, "currentCreditAmount": 0, "maxCreditAmount": 0, "maximumOverageCharge": 0, 
"policy": "unlimited"}]}
2024-07-24T23:22:12.040 WARNING: Retrying __main__.run_playwright in 1.0 seconds as it raised AssertionError: Locator expected to be visible
Actual value: <element(s) not found> 
Call log:
LocatorAssertions.to_be_visible with timeout 45000ms
  - waiting for get_by_test_id("planRowDetail").filter(has=locator("prism-button[href=\"https://customer.xfinity.com/#/devices#usage\"]"))

It appears that the error above happens just before the sensor update would happen. And because of that, I have no data being saved.

I also attempted that with MQTT but have the same result.

One interesting thing is that when I access the web page (https://customer.xfinity.com/#/devices#usage) I cannot see any information about Data Usage (screenshot below)

However, when I use the app, I can see and it matches the json data from "2024-07-24T23:21:25.359 DEBUG: Updating Usage Details" (screenshot below)

Do you know what could be the cause?

Thank you.

[thor0215]

Can you go to this URL and grab a screenshot of the page once it's fully loaded? https://www.xfinity.com/learn/internet-service/auth

I need to see how it compares to what I see for my account. The script is looking for the View usage and it's failing for you:

[thiagobruch]

Hi,

Thank you for the quick response. Here is the screenshot. It doesn't show the Usage.

The interesting part is that if I grab the JSON, I can find this:

{
      "policyName": "Unlimited Data Plan",
      "startDate": "07/01/2024",
      "endDate": "07/31/2024",
      "homeUsage": 2245,
      "wifiUsage": 0,
      "totalUsage": 2245,
      "allowableUsage": 100000,
      "unitOfMeasure": "GB",
      "displayUsage": false,

And the "totalUsage": 2245 matches what I have in the phone app

[thor0215]

The issue is not gathering the usage data. The issue is how the script tries to verify the usage data has enough time to load. Try updating to the latest 0.0.11.0.1 version I just posted. It will still fail but send the the ERROR: planRowDetail output when in INFO logging mode.

[thiagobruch]

This is what I got as an error:

2024-07-25T12:00:34.616 INFO: Entering username (URL: https://login.xfinity.com/login)
2024-07-25T12:00:40.290 INFO: Entering password (URL: https://login.xfinity.com/login)
2024-07-25T12:00:54.664 INFO: Updating Session Details
2024-07-25T12:00:58.398 INFO: Updating Usage Details
2024-07-25T12:00:59.604 INFO: Updating Plan Details
2024-07-25T12:01:44.827 ERROR: planRowDetail Count: 3
2024-07-25T12:01:44.898 ERROR: planRowDetail Row 3 inner html: <p class="text-headline1 plan-row-item-headline text-neutral-base lg:justify-start lg:grid lg:place-items-center lg:col-span-2">WiFi Name</p><div class="mt-2 lg:mt-0 lg:flex lg:justify-start lg:col-start-4 lg:col-span-5"><div class="text-body1 text-neutral-base opacity-60 lg:grid lg:grid-flow-col lg:place-items-center"><strong>CHANGED1, CHANGED2</strong></div></div><div class="mt-6 lg:mt-0 lg:col-start-9 lg:col-span-4 lg:justify-end lg:grid lg:place-items-center"><prism-button tone="theme1" display="ghost" size="sm" href="https://customer.xfinity.com/settings/wifi" class="sc-prism-button-h sc-prism-button-s hydrated"><!----><a part="native" href="https://customer.xfinity.com/settings/wifi" class="btn theme1 ghost dir-row horizontal default standard sc-prism-button sc-prism-button-s" id="prism-button-c16b4e7c-2abe-4eb2-a0b7-e9b7d344e6de"><prism-text class="text sc-prism-button sc-prism-button-s button hydrated">View or edit WiFi settings</prism-text></a></prism-button></div>
2024-07-25T12:01:44.933 ERROR: planRowDetail Row 3 text content: WiFi NameCHANGED1, CHANGED2View or edit WiFi settings
2024-07-25T12:01:48.114 INFO: Loading Device Data (URL: https://www.xfinity.com/support/status)
2024-07-25T12:01:50.053 INFO: Updating Device Details
2024-07-25T12:01:54.890 WARNING: Retrying __main__.run_playwright in 1.0 seconds as it raised Error: LocatorAssertions.to_contain_text: Error: strict mode violation: locator("div#app p[class^=\"connection-\"]") resolved to 4 elements:
    1) <p class="connection-fd8e6 status-a6e7b">…</p> aka get_by_text("Connected").first
    2) <p class="connection-fd8e6 status-a6e7b">…</p> aka get_by_text("Connected").nth(1)
    3) <p class="connection-fd8e6 status-a6e7b">…</p> aka get_by_text("Status Unavailable").first
    4) <p class="connection-fd8e6 status-a6e7b">…</p> aka get_by_text("Status Unavailable").nth(1)

Call log:
LocatorAssertions.to_contain_text with timeout 45000ms
  - waiting for locator("div#app p[class^=\"connection-\"]")
.
2024-07-25T12:02:05.122 INFO: Loading Internet Usage (URL: https://login.xfinity.com/login)
2024-07-25T12:02:05.696 INFO: Entering username (URL: https://login.xfinity.com/login)
2024-07-25T12:02:10.752 INFO: Entering password (URL: https://login.xfinity.com/login)

[thor0215]

Ok now it's failing as it tries to gather your modem details. Try again with 0.0.11.0.2 and send me the info output

[thiagobruch]

I don't see the version 0.0.11.0.2 Is that already published as release?

Thank you

[thor0215]

I've bumped the version now to 0.0.11.0.4. Just go to the HA addon and check for updates, then update the addon and retest. Thanks!

[thiagobruch]

It looks like it is working now (at least I got all the data in a sensor now). However, there is still an "error" in Debug Mode. Not sure if it is relevant, here it is:

"status": {"status": null, "statusCode": null, "activationStatus": null, "uptime": null, "regState": null, "state": "ONLINE", "color": "GREEN"}}]}, "home": {"devices": []}, "unknown": {"devices": []}}}
2024-07-25T15:00:27.835 ERROR: div#app p Count: 0
2024-07-25T15:00:27.850 INFO: Usage data retrieved and processed
2024-07-25T15:00:27.850 DEBUG: Usage Data JSON: {"attributes": {"policy_name": "Unlimited Data Plan", "start_date": "07/01/2024", "end_date": "07/31/2024", "home_usage": 2305

Thank you very much for the quick response and fixing the issue :)

[thor0215]

Just curious do you have MQTT enabled or not?

[thiagobruch]

Yes I do.

[thor0215]

Could you send me a screenshot of your MQTT usage device, similar to the one below?

[thiagobruch]

Hi,

Please find it below. tank you.

[thor0215]

Thanks, I wanted to confirm the script detected your cable modem details correctly.