Youtube channel subscriptions are partially missing.

[macruspareto]

Kodi 18 Leia on Raspberry OS. YT addon 6.8.10. Custom-generated API keys.

I'm subscribed to maybe 80 channels on YT and to my surprise, about 2 weeks ago, some have dissappeared.

When I manually search for a particular channel and hit "Subscribe", the plugin says "Already subscribed".

I've googled around but couldn't find any reasonable solution. Can you be of any help? I use RaspberryOS Kodi and also LibreELEC, same problem on both.

[jmbreuer]

I have the same issue with a small number of channels; these are two that are reproducibly affected and that I can identify right now:

https://www.youtube.com/c/KeithRuckerVintageMachineryorg https://www.youtube.com/channel/UCEn3fRj2e0mpqYsijxnzayg

There is absolutely nothing in the logs that stands out.

... I was in a digging mood and tracked down the affected channel ids / how the YouTube v3 API reacts to them. Both of those are not returned by youtube.subscriptions.list(mine=true, ...).

They do show up on my https://www.youtube.com/feed/channels; that's where I noticed that the bell icon next to those particular channels is greyed out. Apparently those channels are configured as "content for kids" with YouTube, which apparently has the effect of both disabling the notification bell and excluding it from the channel list API.

I consider this a bug in the YouTube API. I have a suspicion it's intentional, but it's not documented in any case. Anyway: https://issuetracker.google.com/issues/205813555

[benyamin-codez]

I'm pretty sure this is still an issue as reported. Can it be reopened? An example: https://www.youtube.com/colinfurze

The Google issue @jmbreuer mentioned above was closed as Intended Behavior. Therefore the root cause is the Data API will not return the channel_id for any channel marked as MadeForKids.

As scraping for these odd channels is problematic, perhaps we could add a setting for "Permanent Channel IDs" (or similar) with e.g. a comma-separated list of IDs. These could then be used with the various functions in the /resources/lib/youtube_plugin/youtube/client/youtube.py file.

At the time of writing (release 7.0.5+beta.4):

The entry point in the get_my_subscriptions() function would be after the while loop (while sub_page_token:) at line 1480, and before headers = {..} at line 1515. If I add, e.g. sub_channel_ids.append('UCp68_FLety0O-n9QU6phsgw') at this point, the colinfurze channel is added to "My Subscriptions". This is what I have in place for the three channels in my feed that are dropped by the API.

A more complicated solution would need to be used for "Subscriptions" using get_subscriptions() (at line 403 in the same file), as the raw API data is returned.

[benyamin-codez]

For "Subscriptions", an easy solution might be to create a new main page (or maybe a sub-page).

Perhaps named "MadeForKids Subscriptions", it could be populated only with the channel IDs provided in the relevant Settings item, perhaps under "Permanent Channel IDs" as suggested above, or maybe "MadeForKids Subs Channel IDs" would be better. The page could replicate the view of the "Subscriptions" page.

If the setting is NOT configured, perhaps then don't show the page...

[MoojMidge]

I won't be adding a separate setting for specific channel ids, too niche and also not necessarily straightforward to get the channel ids in the first place.

This is another issue that will be resolved by using the V1 innertube API.

As an alternative, I do also have plans to make as much functionality as possible have local only equivalents. Currently watch history and watch later are implemented locally even if not logged in.

There is also an internal favourites function that was made functional, but which is not currently exposed in the plugin UI. My plan is to use the plugin internal favourites as a substitute for subscriptions and saved playlists, if not logged in. Any channel stored in the favourites can then be retrieved for use within My Subscriptions.

[MoojMidge]

There is also an internal favourites function that was made functional, but which is not currently exposed in the plugin UI. My plan is to use the plugin internal favourites as a substitute for subscriptions and saved playlists, if not logged in. Any channel stored in the favourites can then be retrieved for use within My Subscriptions.

This has been implemented as Bookmarks in https://github.com/anxdpanic/plugin.video.youtube/pull/691

[benyamin-codez]

This has been implemented as Bookmarks in #691

Looking forward to giving #691 a try when it hits a repo.

[DjDiabolik]

@MoojMidge and @anxdpanic

On my system (PC Ubuntu 22.04 and current youtube 7.0.6 unofficial) i have the same EXACT issue.

EDIT I have to add one things.. i can reproduce this issue whit two different kodi version: the old ubuntu ppa 20.2 and the recent flatpack kodi 21....

Open "My Subscription" and apparently i can see the video only from one CHANNEL i subscripted. It's very strange issue because about 10/20 minutes ago apparently all works correctly.

How i can fix this issue ? This similar issue it's appears also if i open the "normal" "My subscription" and also if i use the "filtered" variant.

Ok kodi.log apparently nothings appears like errors or similar......

[MoojMidge]

@DjDiabolik - if it works sometimes but not other times then it is not this problem.

This problem can be worked around by adding the channels to your bookmarks from the context menu after searching for them.

For anything else, as has been said before, create a new issue, provide all the required details and provide a debug log from when it doesnt work. Don't have time to be playing guessing games.

[DjDiabolik]

@MoojMidge mmmm.. ok. When this issue it's come i have simply try to found a similar issue here. Apparently on this specific case probably as soon as the problem appeared it disappeared...... I also believe that it is a coincidence that updating the same addons solved/caused the problem.

[MoojMidge]

There are a few issues related to how the subscriptions are fetched that could result in what you are seeing. They are on my list of things to fix, but without a debug log from when you are seeing this problem (even a normal log is better than nothing) it is impossible to determine if it is an already known issue or something new.

[benyamin-codez]

This has been implemented as Bookmarks in #691

@MoojMidge, I can confirm this certainly works for content in the "My Subscriptions" feed.

I presume the performance hit and additional complexity to consolidate bookmarked channels into "Subscriptions" is unwarranted given the purity and simplicity of the current APIv3 solution, especially when "Bookmarks" is right there next to "Subscriptions" in the menu.

The only issue I guess is how do users know to use "Bookmarks" to resolve the OP issue, i.e. to bookmark channels marked as MadeForKids. Is a search result referencing this issue perhaps the only way?

Otherwise, I am happy for the issue to be closed again.

[MoojMidge]

Agree that it is a bit messy at the moment, with two different things performing largely the same function, especially in the context menu.

Don't have any good ideas to resolve that, as the overlap is not complete enough such that it warrants combining Bookmarks and Subscriptions in the background (you can't Subscribe to a single video or a playlist, and you can't open your Bookmarks in another logged in client).

Open to suggestions, but at the moment not planning on doing anything to change that.

Will keep the issue open until the V1 API is implemented for subscriptions.

[benyamin-codez]

The only issue I guess is how do users know to use "Bookmarks" to resolve the OP issue, i.e. to bookmark channels marked as MadeForKids. Is a search result referencing this issue perhaps the only way?

@MoojMidge, I presume when V1 API is implemented for "Subscriptions" this will resolve this issue anyway. Is the plan to end the use of the V3 Data API itself, and thus the need for keys, OAuth, etc. ?

If the V3 Data API is being retained, would there be any advantage to:

1. Populating a cache on first run (or expired TTL) with V1 API output;
2. Detecting any MadeForKids channels;
3. Adding MadeForKids channels to "Bookmarks" (perhaps with a MadeForKids icon);
4. Create the "Subscriptions" view using the cache as model;
5. Periodically refresh the cache using V3 Data API + bookmarked channel lookups; &
6. Use manual refreshes to clear and recreate the cache.

There might also be some value to populating a central cache with data as a model that "Bookmarks", "Subscriptions" and "My Subscriptions" can all draw from. Perhaps it could be checked for staleness on plugin launch and populated as necessary. It might delay startup slightly (depending on the number of subscribed channels) but improve run times for various menu items.

[MoojMidge]

Is the plan to end the use of the V3 Data API itself

No plans from me to do this. The V1 API is more difficult to use than the V3 API, as it mixes presentational elements with the data/meta-data.

Would only be used for certain functions, e.g. related videos (where it is already currently used), where the V3 API too limited or has removed support for existing functionality.

thus the need for keys, OAuth, etc. ?

This wouldn't change. The requests still need to be authorised, and OAuth is the best way to do this. One alternative is to copy cookies from a logged in browser session, but that is a worse alternative in almost all ways.

Bookmarks, and other local only functions, are the only other alternatives for this that won't require user account authorisation.

If the V3 Data API is being retained

It will be, but not for getting the list of subscriptions. Once it is replaced for this function, there is no need to sync it with Bookmarks just for the purposes of listing the missing channels. Bookmarks will continue to be used in My Subscriptions, but currently also serves as a workaround for the missing channels. It just won't be necessary when the V1 API is used.

There might also be some value to populating a central cache with data

This is already done, just not in a consistent manner.

One of the problems has been how to check whether the cache is still fresh and manage the updates.

Refreshing of cached listings is now possible via the context menu, but it is implemented inconsistently in different listings. Trying to fix them as I go. The next v7.0.7 release will enable this for My Subscriptions and Related Videos https://github.com/anxdpanic/plugin.video.youtube/pull/774/commits/05b900b1cb3d829544fb4351c1f2d0c16f2ecf51

One of the intended aims of the threaded fetch implementation is to be able to always use the cached content, but also make the API requests in the background and check whether the server responds with a 304 status when provided the ETag of the previously cached request. If it doesn't, then the new response will be requested and processed in separate threads.

That's the intent anyway, but no eta for any of this.