Kodi only checks for add-on updates every 24 hours

[dagwieers]

Describe the feature

In a VOD-centric world, Kodi add-ons are living by the grace of the VOD content provider and the stability of the VOD infrastructure. Unfortunately, since Kodi add-ons are not first-class citizens, they are not prepared or made aware of coming changes to the VOD infrastructure.

This means that "suddenly" the Kodi add-on stops working when the VOD infrastructure undergoes changes that the Kodi add-on was not prepared for. This immediately impacts users. And no matter how fast add-on developers react, or how well they are improving their processes, this will always be reactive.

Many add-ons pro-actively test their add-ons at full length and are made aware of such issues as soon as they arise, and a fix is out as fast as possible. We still need to wait until the add-on is being accepted and merged in the Kodi repository, And additionally it takes up to 24 hours to get these add-ons installed on end-user systems.

24 hours mean that daily users are always affected, which is a real problem for Kodi as a reliable platform.

• Mean time to detection (MTTD) is now close to 0
• Mean time to acknowledge (MTTA) depends on add-on development team, could be within hours
• Mean time to recovery (MTTR) is a problem
  • Fixing depends on add-on development team
  • Getting it merged into Kodi repository depends on Team Kodi
  • Getting it on user systems depends on this 24h check in Kodi

We would like to see this 24 hour check in Kodi to be reduced to 4 or 6 hours so that fixes can be delivered before they impact end-users.

In most cases being able to get the fix on end-users systems by 18:00 would avoid impacting the majority of users.

[dagwieers]

For international add-ons

(Think: Netflix, HBO GO or Disney+) With a 24h repository update frequency, it takes 24 hours to get a fix to all end-users. Since primetime moves from timezone to timezone, getting a fix out more quickly is paramount. If we miss European primetime due to unavailability of Add-on developers, we could still save US primetime, but not if the frequency is 24 hours.

For regional add-ons

(Think: European broadcasting) If we would like to get a fix deployed to all our users before primetime (18:00?), you have to get a newly released add-on pushed and merged in the Kodi repository by 12:00 if the update frequency is 6h. Which is highly unlikely IMO.

In the incidents we have experienced, new VOD infra releases happened in the morning (we have seen them at 06:00 as well as at 09:00), that gives about 3 to 5 hours to get a fix and push it out to users. We have been in this situation and we have been able to fix, but not able to push to users because of the 24h frequency.

So a repository update frequency of 4h makes more sense overall IMO.

[dagwieers]

I opened https://github.com/xbmc/xbmc/pull/17955 for this upstream.

[dagwieers]

So @matthuisman has an interesting suggestion at: https://github.com/xbmc/xbmc/pull/17955#issuecomment-634964489

We could offer an easier path for users to update the add-on from the add-on settings. We could put this under the Expert category at the top, or a separate "Update" category. Even if we can increase the frequency of the Kodi add-ons repository, having the ability to update the repositories manually is a useful addition in any case.

This is not an alternative to increasing the frequency, we still need this for a better user experience.

[dagwieers]

So I tested this:

• UpdateAddonRepos only works if the add-on has Auto-update enabled
• InstallAddon(plugin.video.vrt.nu) does not seem to work if the add-on is already installed
• I don't see how we can check if an add-on has Auto-update disabled
• I don't see how we can ensure that an add-on has Auto-update enabled

So only running UpdateAddonRepos is a possibility at this time.

Update: The upcoming VRT NU add-on will include an option in the settings to update your repositories and add-ons.

[matthuisman]

I found UpdateAddonRepos does fetch available updates even with Auto updates disabled.

However, yes - it won't make the add-ons actually update. If they have disabled auto updates - then probably best not to update it automatically anyway? But at least kodi will know of the update quicker.

Your right - InstallAddon won't update an add-on. A work around would be to remove the addon first. It's pretty hacky though.

pseudo code

xbmc.executebuiltin('UpdateAddonRepos')
for addon in to_update:
     shutil.rmdir(addon.dir) #or rename
xbmc.executebuiltin('UpdateLocalAddons')
for addon in to_update:
    xbmc.executebuiltin('InstallAddon(addon.id )', True)
    Addons.SetAddonEnabled(addon.id)

I just tested and kodi doesn't wait for UpdateAddonRepos to complete... Which makes it ever trickier and above won't even work.

You should be able to detect if auto updates are enabled with the JSON RPC call

{'method': 'Settings.GetSettingValue', 'params': {'setting': 'general.addonupdates'}}
0 = Install automatically
1= Notify but dont install
2 = Never check

You can probably even change it with {'method': 'Settings.SetSettingValue', 'params': {'setting': 'general.addonupdates', 'value': 0}}

[dagwieers]

@matthuisman I am not interested in adding this as a workaround. But my main concern was that the add-on may have Auto update disabled. Which we cannot detect and we cannot change.

[matthuisman]

You can detect it

{'method': 'Settings.GetSettingValue', 'params': {'setting': 'general.addonupdates'}} 0 = Install automatically 1= Notify but dont install 2 = Never check

and you can change it (but maybe shouldnt) {'method': 'Settings.SetSettingValue', 'params': {'setting': 'general.addonupdates', 'value': '0'}

[dagwieers]

@matthuisman AFAICT general.addonupdates is a global setting, not a per-addon setting.

[matthuisman]

yes, your wanting to check if Users have Auto Update addons disabled in kodi settings right? Can you even disable per add-on updates?

[matthuisman]

{'method': 'Settings.GetSettingValue', 'params': {'setting': 'general.addonupdates'}} will return the kodi setting for this

I assumed you wanted to detect if this was disabled so you could advise the user that the latest version needs to be manually installed after you call UpdateAddonRepos

1) Detect new version 2) UpdateAddonRepos 3) If general.addonupdates != 0: "You have updates disabled, you need to manually update" (you could even open the add-ons update window for them) else: "Latest version will be installed shortly"

You could even do 1) get general.addonupdates value 1) set general.addonupdates to 0 2) UpdateAddonRepos 3) general.addonupdates set to old value

That would force install the update - but I think if the user has explicitly disabled automatic updates - you should respect that?

[dagwieers]

yes, your wanting to check if Users have Auto Update addons disabled in kodi settings right? Can you even disable per add-on updates?

Yes, you can enable/disable updates per add-on. In fact auto-updates is disabled if you install an add-on (first) using a ZIP file AFAICR.

But again, I don't think this workaround is a good general solution, and I would prefer to look at a solution that fix the problem for all add-ons and all Kodi users, rather than focus on a single add-on and a single use-case.

[matthuisman]

oh, I see in the add-on info menu. I've gone through that menu so much and never took notice of the auto-update toggle. (Haha)

Even kodi updating repos more often won't help if they have updates disabled. So what are you thinking to fix that?

[dagwieers]

It's a marginal problem, the majority of users do not have this disabled. I would even say, someone that has this disabled is a technical user. And if they do, they live with the consequences, at least they can tell they do not run the latest version. My concern is mostly stated at the description of this bug.

1. Infrastructure change breaks add-on
2. We know this has happened (within 30 minutes)
3. We fix it as fast as we can (within hours)
4. Our fix is sitting idle in the Kodi repository (for days) ← we could offer our own repository
5. Our fix needs 24 hours to be installed for all users ← this is our main concern

This has happened to us now a dozen times. Not just with this add-on, with various add-ons and even with inputstreamhelper (affecting almost every Kodi user at once). If a fix takes 24 hours to get distributed, (and we are not able to fix it before primetime) it means every single daily user is impacted. This is a very disruptive event.

If Kodi is not able to offer a reliable viewing experience, people will not rely on it and will not recommend it to others. I have a hard time recommending Kodi to anyone non-technical because it may fail at any given time and may require manual intervention to get something fixed timely. (Even though we do our best to fix issues within hours, we cannot get it to users)

BTW for an add-on that is used internationally this dynamic is different, there is no single primetime. So a solution for Netflix, HBO GO, Disney+ is even more important, both in scope and impact.

[Cpainy]

I'm using kodi for my daily TV viewing via addons. Don't have cable any more. Downtime is annoying. I really support any development to decrease downtime!

[dagwieers]

@mediaminister mentioned on Slack that we could monitor the VRT NU app on the Google Play Store and Apple App Store to get an advance warning of (potential) upcoming infrastructure changes. It may not help us fix proactively, but could help to plan fixing reactively.

• https://play.google.com/store/apps/details?id=be.vrt.vrtnu
• https://apps.apple.com/be/app/vrt-nu/id1337574835

Especially the Apple Store app history link is useful to look at what has changed. The changelog actually hints which updates are backend related !!

This would require a GitHub CI workflow to detect and report changes.

[mediaminister]

I already have some code for Google Play:

# -*- coding: utf-8 -*-

from __future__ import absolute_import, division, print_function, unicode_literals
import re
import json
from datetime import datetime

try:  # Python 3
    from urllib.request import urlopen, Request
except ImportError:  # Python 2
    from urllib2 import urlopen, Request

url = "https://play.google.com/store/apps/details?id=be.vrt.vrtnu"
req = Request(url)
response = urlopen(req)
regex = re.compile(r'AF_initDataCallback\(([\s\S]*?)\);<\/')
match = re.findall(regex, response.read().decode('utf-8'))
key = re.compile(r"key: 'ds:(.*?)',")
value = re.compile(r'return ([\s\S]*?)}}')
info =  next((json.loads(re.search(value, prop).group(1)) for prop in match if re.search(key, prop).group(1) == '8'), None)
print(info[1])
info =  next((json.loads(re.search(value, prop).group(1)) for prop in match if re.search(key, prop).group(1) == '5'), None)
print(info[0][12][6][1])
published = info[0][12][8][0]
print(published)
print(datetime.fromtimestamp(published).isoformat())

[dagwieers]

@mediaminister Perfect, I think one of them is sufficient.

[dagwieers]

Kodi has added a new feature upstream that allows to control the update interval from the server as was suggested: https://github.com/xbmc/xbmc/commit/4396fdcb29df00147f68b3ced795a0bfcaf83b81