Closed vanessa-chang closed 9 months ago
Tested an es preload image: https://images.endlessos.org/builds/12693/
There are findings:
I think this is going to be fairly straightforward, at least for the case where we've properly preloaded the expected data. For content nodes, I believe that Kolibri will find that everything is present and the scheduled tasks will become no-ops.
The issue is channel database imports. Since channel databases are versioned, Kolibri always tries to download them to see if there's a new version available. However, the collections packs already contain the desired version of the databases. We can check that we already have that version of the database available and skip creating a task for it.
Skipping content task creation would be harder since we'd have to duplicate all the Kolibri code that determines what content files are needed. It's all pretty hairy as it has to descend the content tree in the channel and apply some heuristics to decide what nodes to include and which files from those nodes are needed. I'm hoping we don't need that.
Now, if the preload is missing data that the pack expects, then I don't know how to handle that nicely. We'd need to decide what to do in the case that initial download tasks are failing because of missing network access. Currently it just loops forever trying to restart the tasks over and over. I'm going to consider that handling out of scope for this task.
It's actually news to me that we're scheduling tasks here in the first place. Looking at 53-ek-content-preload, my understanding is the image builder leaves us with a fully populated database and all those content nodes marked available
. So, why is the explore plugin trying to download stuff at all? I could have sworn it didn't do that before, but maybe I just did a bad job testing things with https://github.com/endlessm/endless-key-content-private/issues/97 :b Sorry for my confusion around this. I had assumed it was working a certain way.
Anyway, um, I kind of feel like the explore plugin should not direct the user to the welcome screen if there is content available in the database. That is, if we're creating that whole download state thing for the first time (somewhere inside kolibri_explore_plugin/collectionviews.py), check if there is any content available, and if there is, exit early with "yep, all's good [everything_is_fine.gif]".
Separately, I think it would be nice to consider a separate general-purpose "bulk import from kolibri/content/bulk-import" plugin here, which should be easier to test (and to point upstream eventually). I described that in https://github.com/endlessm/endless-key-content-private/issues/102. The bit we would need for the image builder to EOS route is the "code that runs when Kolibri starts up" in this document: https://docs.google.com/document/d/15Yc0Tc4oB9hAO-nFCJ_gNDIuAvsBLKsE5UVhzarNWtc/edit?skip_itp2_check=true&pli=1.
One nice thing with that approach is it gives us a way (outside of generating a db.sqlite3 file) for the image builder itself to signal to Kolibri that it has some content to start with, so we aren't doing any guessing and we also know exactly what content pack the user is supposed to be seeing.
Anyway, um, I kind of feel like the explore plugin should not direct the user to the welcome screen if there is content available in the database. That is, if we're creating that whole download state thing for the first time (somewhere inside kolibri_explore_plugin/collectionviews.py), check if there is any content available, and if there is, exit early with "yep, all's good [everything_is_fine.gif]".
That's the way it did work until #863, but that's too simplistic. We need to apply tags during initial setup and the image builder can't do that since the pack selection needs to be delayed to runtime. If we just checked if there was content when the tags hadn't been applied, then the discovery page would be empty.
You could maybe optimize to skip to the tagging if any content exists, but you still need to validate that all the necessary nodes are available or the tagging might fail. So, I think it is appropriate that the downloader is run to ensure that you have all the necessary content. We just have to make it smarter to not actually perform any downloads unless really needed.
Okay, in a separate thread (https://github.com/endlessm/kolibri-explore-plugin/issues/898) I'm going to try my hand at the bulk-import stuff that's in my head :) I'm not 100% sure if it'll work out in time so it shouldn't block what you're working on (and those issues you have in mind are 100% worth solving either way). But I'm hoping it'll give us a way to kind of circumvent what we're running into here.
I'm going to make a release so we can consume this in the flatpak and test it out.
This is available to test in 7.11.0 and in the latest flatpak.
@dbnicholson is there a preloaded image available for the test?
I tested the image (eosimpact-eos5.0-amd64-amd64.231027-160050.es_GT.img) today, the issue is still there as:
Flatpak info: version: 0.7 build date: 2023-10-26
eos-diagnostic-231101_144021_UTC+0800.txt
I will re-open this issue
In the diagnostics:
nov 01 14:37:26 endless endless-key-daemon[3228]: INFO 2023-11-01 14:37:26,886 Enqueuing task {'task': 'kolibri.core.content.tasks.remotechannelimport', 'params': {'channel_id': '359e048230974c8f80db1a95dc80d544', 'channel_name': 'EiE Familias'}} at 1698820646.886678
Hmm, why is it trying to fetch that channel? Shouldn't it already be in the image? Also:
nov 01 14:37:26 endless endless-key-daemon[3228]: INFO 2023-11-01 14:37:26,887 Attempting connections to variations of the URL: https://kolibri-content.endlessos.org
Uh, that should only be used in the image builder. At runtime studio.learningequality.org should be used. Something must be wrong in the image builder code.
- "unable to connect" message is show (I guess it needs a translation too?)
This message comes from Kolibri and I think it's a side effect of triggering a channel metadata download without the content server being available. If we need to translate it, it would have to happen upstream.
Oh, wow. If you have any Kolibri option environment variables set when the homedir is first created, they get persisted in the options.ini
file. That's... unexpected. I guess the image builder hook should kolibri configure setup
first before setting KOLIBRI_CENTRAL_CONTENT_BASE_URL
.
Oh, wow. If you have any Kolibri option environment variables set when the homedir is first created, they get persisted in the
options.ini
file. That's... unexpected. I guess the image builder hook shouldkolibri configure setup
first before settingKOLIBRI_CENTRAL_CONTENT_BASE_URL
.
That sounds like a reasonable solution. Alternatively, it should be fine to delete options.ini
at the same time we run kolibri manage deprovision
. I don't think we need anything other than defaults in there, and Kolibri will happily make a new one whenever it starts.
Although, indeed, I never noticed that it persists environment variables there. Out of curiousity (and hoping that it would be an easy fix), I checked and it's the same behaviour with Kolibri 0.15.
I tried to debug this directly in kolibri-explore-plugin by mocking a Kolibri homedir with the channels imported the way the image builder does it, but I couldn't reproduce it. I'm flashing the massive 120 GB test image to a USB drive and will try on the real thing.
Ooops... I just realized that https://github.com/endlessm/endless-image-config/pull/1158 has the 3 packs to download as spanish
, extras
and spanish-extras-preload
. That middle one should be spanish-extras
. No wonder it was trying to download a channel.
I tried to flash the massive image to a USB drive I had lying around, but it was too small. So today I was going to build a custom image locally and immediately noticed the list of packs was wrong. I'm going to continue with the local build so I can debug it live.
Ooops... I just realized that endlessm/endless-image-config#1158 has the 3 packs to download as
spanish
,extras
andspanish-extras-preload
. That middle one should bespanish-extras
. No wonder it was trying to download a channel.
No, that's wrong as it's supposed to be the English extras pack.
Argh. The image build has the Endless Key flatpak with kolibri-explore-plugin 7.10.0. The fixes from #899 are in 7.11.0. I think after I merged https://github.com/flathub/org.endlessos.Key/pull/20 I must not have waited enough time for it to be published on flathub.
This build has some of the image builder changes. Even though there's one more useful change to the flatpak I'd like to have, I'm going to download that image and test it.
QA fail. With the updated image, the channel metadata tasks were being skipped. However, my previous assumption that the content tasks would become essentially no-ops was faulty. In the test suite, a couple Kolibri methods were being mocked, and that hid the fact that Kolibri probes the remote server in this scenario even if it already has all the content objects. So, all tasks really need to be skipped.
I hope https://github.com/endlessm/kolibri-explore-plugin/pull/900 fixes this issue. I've triggered a release of kolibri-explore-plugin, then will get that into the flatpak, then to rebuild the test image.
I'm leaving this open until we get actual verification.
@vanessa-chang can you try the image at https://images.endlessos.org/builds/13181/?
Sure. The download is a bit slow here for me. I will try to update the result soon as I can.
@vanessa-chang was having trouble downloading the image, so I ran the test myself. It works! I turned off the network in my VM and started Endless Key. The downloader progress bar jumps immediately to 90% where it runs the (ridiculously slow) tag application. Then it worked as normal with a populated overview and full library.
I think we can go ahead merging and backporting all the things. I'm going to close this kolibri-explore-plugin specific issue and carry on in https://github.com/endlessm/endless-key-content-private/issues/103.
Thanks @dbnicholson for helping the testing. I finally finished the download! I saw the same thing as it ran fast to 90 % and took around 15 seconds to complete the loading. Then everything works well without the internet.
Follow up from https://github.com/endlessm/eos-image-builder/issues/117
There are some customized images the would need the app content preloaded, so that the user can access it offline.
Content: https://github.com/endlessm/endless-key-content-private/issues/103
AC: