Virgin Media (GB) recording capacity

[callumeveratt]

I'm not sure this ever worked on the original Arris integration (certainly for me the value was always unavailable for recording_capacity) but the following appears in the log for getting recording capacity in GB:

Logger: lghorizon.lghorizon_api Source: /usr/local/lib/python3.10/site-packages/backoff/_common.py:120 First occurred: 10:39:10 (1 occurrences) Last logged: 10:39:10 Giving up _do_api_call(...) after 3 tries (lghorizon.exceptions.LGHorizonApiConnectionError: Unable to call https://prod.spark.virginmedia.com/eng/web/recording-service/customers/xxx_gb/quota. Error:501 Server Error: Not Implemented for url: https://prod.spark.virginmedia.com/eng/web/recording-service/customers/xxx_gb/quota)

[Sholofly]

@callumeveratt I think this works as intended. For me there's no way to determine if cloud recording is available but to query that url. If it gives me a 501 i know it isn't avaiable. I presume you don't have a sensor at all anymore for recording capacity?

[caraar12345]

It looks like VM sends this over MQTT as a percentage --

Topic: {household_id}/{box_id}/localRecordings/capacity Payload: {"messageType":"CPE.capacity","version":"1.0.7","used":97}

[Sholofly]

Ah m8 thats brilliant information. Until now i was only supporting cloud recordings. In this way I can add a sensor for local recordings. How did you find out?

[caraar12345]

All the investigation I've been doing has been just by logging into https://virgintvgo.virginmedia.com with the Chrome dev tools open and watching the calls it makes.

In particular, in the call to mqtt, which is the websocket, you can see the MQTT messages either way!

[Sholofly]

Cool, I'm going to try to add this to the api. Thanks!

[Sholofly]

BTW, can you find out of anything in your account data states that recording is enabled and if this is network or local?

[callumeveratt]

Ignore that - I imagine it's not specified anywhere for Virgin GB as channel owners have cold feet over cloud recording - so it's never been enabled in the UK from what I gather

[callumeveratt]

Would definitely be cool to use the local recording data where applicable though :-)

[caraar12345]

Will also have a dig around -- but with Virgin, I can confidently say it'll always be local as we don't have Cloud DVR!

@callumeveratt I think remote recording in that context is the ability to remotely instruct your box to record something rather than recording to VM's servers

[callumeveratt]

@caraar12345 - Yeah you're right, I got ahead of myself for a second. It is the ability to remote record, not cloud record.

[caraar12345]

Ha! Impressive timing from both of us 😁

Does look like there's something about local DVR though -- localDvrAvailable. I believe networkDvrAvailable refers to being able to play back local recordings on a mobile device.

Unrelated tidbit, I do wonder where they get 'gender' from, as I may be somewhat effeminate, but I've definitely never identified myself as female to anyone 😆

{
    "oespToken": "",
    "accessToken": "",
    "householdId": "{customer_id}_gb",
    "username": "{customer_id}",
    "locationId": "18",
    "originalLocationId": "18",
    "timeToIdleSeconds": 7200,
    "customer": {
        "givenName": "Aaron",
        "countryCode": "GB",
        "gender": "female",
        "status": [],
        "smartCardId": "",
        "physicalDeviceId": "{box_id}",
        "stbType": "EOS",
        "networkDvrAvailable": true,
        "replayOTTOnlyAvailable": false,
        "localDvrAvailable": true,
        "householdId": "{customer_id}_gb",
        "sharedProfileId": "",
        "replayTvEntitled": true,
        "replayTvAvailable": true,
        "replayTvMinimumBroadcastTime": 0,
        "replayTvOptedIn": true,
        "offlineViewingEntitled": true,
        "profileIds": [
            ""
        ],
        "oboProvisioned": true,
        "customerId": "{customer_id}",
        "purchaseEntitlementsToken": "",
        "isSkLite": false
    },

[Sholofly]

From what call is this data coming from?

[caraar12345]

Oops - https://prod.oesp.virginmedia.com/oesp/v4/GB/eng/web/session

[Sholofly]

Are you guys (or guy and lady) able to update and can you check if every box have an attribute with recording capacity?

[caraar12345]

Will give it a shot; unfortunately I'm still contending with the config flow issue (hopefully this restart will sort it!?)

[ChrisThomas42]

@Sholofly On VirginMedia GB, I have 1 device and 1 entity. No sign of recording capacity anywhere.

[caraar12345]

If you click the box entity, does it show in attributes?

[Sholofly]

No you should check the developer tools

[callumeveratt]

Yep - there for me. Null though - is that expected at the mo?

[Sholofly]

Depends, do you have local recordings?

[callumeveratt]

Yep - have recordings on the box and on TVGo I can see the MQTT request @caraar12345 gave

{"messageType":"CPE.capacity","version":"1.0.7","used":72}

[Sholofly]

ok, i that case it doesn't work yet. I have to add more logging to find out where the problem is. Do you have debugging enabled in your component?

[callumeveratt]

Nope - but if you need to add some more logging I can get it turned on in the meantime

[Sholofly]

OK stupid. I didnt't commit the part to subscribe to the actual mqtt topic :( Can you manually update your manifest.json to

"requirements": [
    "lghorizon>=0.5.13"
  ]

And then reboot of course ;)

[Sholofly]

@Sholofly On VirginMedia GB, I have 1 device and 1 entity. No sign of recording capacity anywhere.

The local recordings are an attribute of the box. It's not an account thing like the cloud recordings.

[ChrisThomas42]

After changing the entry 0.5.12 to 0.5.13 and restarting, I no longer have an entry for recording capacity listed in states. I can see all my local recodings in the integration.

[Sholofly]

Do you see recording capacity of each box in the box attributes?

[callumeveratt]

I updated the manifest also - the recording_capacity did disappear originally but it came back but sadly still null

[Sholofly]

Can you check the logs for mqtt messages if you set it to debug?

[callumeveratt]

Just trawling though, so far found these

2022-12-21 15:05:41.002 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Sending SUBSCRIBE (d0, m7) [(b'201434117_gb/+/localRecordings', 0)]

2022-12-21 15:05:41.003 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Sending SUBSCRIBE (d0, m8) [(b'201434117_gb/+/localRecordings/capacity', 0)]

[callumeveratt]

I see a SUBACK shortly after - but not much else

[Sholofly]

If you visit the web page now and force the webpage to retrieve your capacity. Is your HA instance getting an update too?

[ChrisThomas42]

I've just re-enabled debug and restarted, SOOO much stuff in the logs I need to go through more fully, but this is a start

2022-12-21 15:16:54.468 INFO (SyncWorker_2) [lghorizon.lghorizon_api] Executing API call to https://prod.spark.virginmedia.com/eng/web/recording-service/customers/105210861_gb/quota 2022-12-21 15:16:54.567 DEBUG (SyncWorker_2) [lghorizon.lghorizon_api] Step 1 - Get Authorization data 2022-12-21 15:16:54.739 DEBUG (SyncWorker_2) [lghorizon.lghorizon_api] Step 2 - Get Authorization cookie 2022-12-21 15:16:54.961 DEBUG (SyncWorker_2) [lghorizon.lghorizon_api] Step 3 - Login 2022-12-21 15:16:55.194 DEBUG (SyncWorker_2) [lghorizon.lghorizon_api] Step 4 - Follow redirect 2022-12-21 15:16:55.305 DEBUG (SyncWorker_2) [lghorizon.lghorizon_api] Step 5 - Extract auth code 2022-12-21 15:16:55.305 DEBUG (SyncWorker_2) [lghorizon.lghorizon_api] Auth code: UBoZFR, Auth state: f00a129d-1974-44e4-842c-3f1b478c5ab4 2022-12-21 15:16:55.305 DEBUG (SyncWorker_2) [lghorizon.lghorizon_api] Step 6 - Post auth data with valid code 2022-12-21 15:16:56.001 INFO (SyncWorker_2) [lghorizon.lghorizon_api] Backing off _do_api_call(...) for 1.1s (lghorizon.exceptions.LGHorizonApiConnectionError: Unable to call https://prod.spark.virginmedia.com/eng/web/recording-service/customers/105210861_gb/quota. Error:501 Server Error: Not Implemented for url: https://prod.spark.virginmedia.com/eng/web/recording-service/customers/105210861_gb/quota)

[callumeveratt]

@ChrisThomas42 I believe that's the cloud recording rather than local - and that error is expected with GB as no cloud recording

@Sholofly - I forced a refresh of my TV Go (if that's what you meant) and I couldn't see anything in the logs around MQTT and the localCapacity topic

[Sholofly]

Ok just one last try for today: can you update your manifest to v0.5.14?

[callumeveratt]

Tried - didn't work originally but thought I'd relink as a test also but actually seems the integration can't connect now:

Logger: custom_components.lghorizon.config_flow Source: custom_components/lghorizon/config_flow.py:81 Integration: LG Horizon (documentation, issues) First occurred: 15:38:17 (4 occurrences) Last logged: 15:38:47 'LGHorizonAuth' object has no attribute 'accessToken'

[callumeveratt]

2022-12-21 15:40:24.721 DEBUG (SyncWorker_11) [lghorizon.lghorizon_api] Obtain Virgin GB mqtt token... 2022-12-21 15:40:24.725 ERROR (MainThread) [custom_components.lghorizon.config_flow] 'LGHorizonAuth' object has no attribute 'accessToken'

[caraar12345]

Oh crap, I’d bet you’ve maxed your session count. I had to leave it an hour and some started clearing out so I could log in again 😅

On Wed, 21 Dec 2022 at 15:40, callumeveratt @.***> wrote:

Tried - didn't work originally but thought I'd relink as a test also but actually seems the integration can't connect now:

Logger: custom_components.lghorizon.config_flow Source: custom_components/lghorizon/config_flow.py:81 Integration: LG Horizon (documentation https://github.com/Sholofly/lghorizon, issues https://github.com/Sholofly/lghorizon/issues) First occurred: 15:38:17 (4 occurrences) Last logged: 15:38:47 'LGHorizonAuth' object has no attribute 'accessToken'

— Reply to this email directly, view it on GitHub https://github.com/Sholofly/lghorizon/issues/31#issuecomment-1361517485, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOYOFGR6K6NMMUXP2VJBQDWOMQFZANCNFSM6AAAAAATFPNJ4A . You are receiving this because you were mentioned.Message ID: @.***>

[callumeveratt]

Aha of course - I'll give it a little

[ChrisThomas42]

so this is a small extract from my log,

2022-12-21 15:38:22.873 INFO (SyncWorker_0) [lghorizon.lghorizon_api] Retrieving recordingcapacity... 2022-12-21 15:38:22.873 INFO (SyncWorker_0) [lghorizon.lghorizon_api] Executing API call to https://prod.spark.virginmedia.com/eng/web/recording-service/customers/*****_gb/quota 2022-12-21 15:38:22.945 DEBUG (SyncWorker_0) [lghorizon.lghorizon_api] Step 1 - Get Authorization data 2022-12-21 15:38:23.119 DEBUG (SyncWorker_0) [lghorizon.lghorizon_api] Step 2 - Get Authorization cookie 2022-12-21 15:38:23.142 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received SUBACK 2022-12-21 15:38:23.146 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received SUBACK 2022-12-21 15:38:23.399 DEBUG (SyncWorker_0) [lghorizon.lghorizon_api] Step 3 - Login 2022-12-21 15:38:23.579 DEBUG (SyncWorker_0) [lghorizon.lghorizon_api] Step 4 - Follow redirect 2022-12-21 15:38:23.699 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received SUBACK 2022-12-21 15:38:23.707 DEBUG (SyncWorker_0) [lghorizon.lghorizon_api] Step 5 - Extract auth code 2022-12-21 15:38:23.707 DEBUG (SyncWorker_0) [lghorizon.lghorizon_api] Auth code: PJ363e, Auth state: 3fed465e-20e8-44a1-86ee-f190c6f72618 2022-12-21 15:38:23.707 DEBUG (SyncWorker_0) [lghorizon.lghorizon_api] Step 6 - Post auth data with valid code 2022-12-21 15:38:23.792 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received SUBACK 2022-12-21 15:38:23.793 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received SUBACK 2022-12-21 15:38:23.793 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received SUBACK 2022-12-21 15:38:24.097 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received SUBACK 2022-12-21 15:38:24.115 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received SUBACK 2022-12-21 15:38:24.116 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received SUBACK 2022-12-21 15:38:24.116 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received SUBACK 2022-12-21 15:38:24.117 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received SUBACK 2022-12-21 15:38:24.118 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received SUBACK 2022-12-21 15:38:24.118 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received SUBACK 2022-12-21 15:38:24.266 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received PUBLISH (d0, q0, r0, m0), '******_gb/0QhZJ30DAj/status', ... (72 bytes) 2022-12-21 15:38:24.266 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received MQTT message. Topic: ******_gb/0QhZJ30DAj/status 2022-12-21 15:38:24.266 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Message: {'source': '0QhZJ30DAj', 'state': 'ONLINE_RUNNING', 'deviceType': 'HGO'} 2022-12-21 15:38:24.285 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received PUBREC (Mid: 14) 2022-12-21 15:38:24.286 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Sending PUBREL (Mid: 14) 2022-12-21 15:38:24.322 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received PUBCOMP (Mid: 14) 2022-12-21 15:38:24.402 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received PUBLISH (d0, q0, r1, m0), '******_gb/6d3aaf2f-721a-4163-a638-ffcbc46e3cbc/status', ... (110 bytes) 2022-12-21 15:38:24.403 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received MQTT message. Topic: ******_gb/6d3aaf2f-721a-4163-a638-ffcbc46e3cbc/status 2022-12-21 15:38:24.403 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Message: {'mac': '', 'ipAddress': '', 'state': 'OFFLINE', 'source': '6d3aaf2f-721a-4163-a638-ffcbc46e3cbc', 'deviceType': 'HGO'} 2022-12-21 15:38:24.421 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received PUBLISH (d0, q0, r1, m0), '******_gb/3C36E4-EOSSTB-003469169803/status', ... (152 bytes) 2022-12-21 15:38:24.421 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received MQTT message. Topic: ******_gb/3C36E4-EOSSTB-003469169803/status 2022-12-21 15:38:24.422 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Message: {'deviceType': 'STB', 'source': '3C36E4-EOSSTB-003469169803', 'state': 'ONLINE_STANDBY', 'mac': '**:**:', 'ipAddress': '192.168.*.*/255.255.255.0'} 2022-12-21 15:38:24.428 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Sending PUBLISH (d0, q2, r0, m15), 'b'******_gb/3C36E4-EOSSTB-003469169803'', ... (66 bytes) 2022-12-21 15:38:24.487 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received PUBREC (Mid: 15) 2022-12-21 15:38:24.488 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Sending PUBREL (Mid: 15) 2022-12-21 15:38:24.584 DEBUG (Thread-3 (_thread_main)) [lghorizon.models] Received PUBCOMP (Mid: 15) 2022-12-21 15:38:24.890 INFO (SyncWorker_0) [lghorizon.lghorizon_api] Backing off _do_api_call(...) for 0.8s (lghorizon.exceptions.LGHorizonApiConnectionError: Unable to call https://prod.spark.virginmedia.com/eng/web/recording-service/customers/******_gb/quota. Error:501 Server Error: Not Implemented for url: https://prod.spark.virginmedia.com/eng/web/recording-service/customers/******_gb/quota) 2022-12-21 15:38:25.686 INFO (SyncWorker_0) [lghorizon.lghorizon_api] Executing API call to https://prod.spark.virginmedia.com/eng/web/recording-service/customers/******_gb/quota

I think(hope) I've removed any potentially sensitive info???? There's more before and after.

[caraar12345]

Woohoo - I finally managed to get the config flow to work so I can chime in!

source_list: 
  - <many_channels>
media_content_type: episode
media_title: BBC Launcher
source: BBC Launcher
entity_picture_local: >-
  /api/media_player_proxy/media_player.virgin_tv?token=
play_mode: app
channel: BBC Launcher
title: BBC Launcher
image: >-
  https://cdn.metrological.com/lightning/apps/com.bbc.app.launcher/1.0.0-a646d9e/static/icon.png
recording_capacity: null
device_class: tv
entity_picture: >-
  https://cdn.metrological.com/lightning/apps/com.bbc.app.launcher/1.0.0-a646d9e/static/icon.png
friendly_name: Virgin TV
supported_features: 154497

No MQTT message is coming through with the recording_capacity which is just... weird.

Interesting note: https://prod.spark.virginmedia.com/eng/web/recording-service/customers/{household_id}/recordings?sort=time&sortOrder=desc&language=en does work. And the data there contains diskSpace for every item. And for me, adding them together gets 96.05% which is reasonable based on the MQTT message reporting 97% (always round up?).

So worst case: iterate through the recordings and add the disk spaces!???

[callumeveratt]

Apologies as been out this evening, just got back and managed to readd. Still no luck and either no value reported for recording or a null value

[Sholofly]

https://prod.spark.virginmedia.com/eng/web/recording-service/customers/{household_id}/recordings?sort=time&sortOrder=desc&language=en

@caraar12345 We have this call too. I results in a list AND a summary of quota at the end. Isn't that the case here?

[callumeveratt]

@Sholofly - just realized we all went quiet on this. I have a bit of time this evening so I can have a look into some of the calls ect and see if I can tweak it to work on my HA if you want :)

[Sholofly]

@callumeveratt Sorry, I'm very busy lately. Please let me know when you have time. Maybe we can figure this out soon :)