blue-build / modules

BlueBuild standard modules used for building your Atomic Images
Apache License 2.0
24 stars 1 forks source link

fix(default-flatpaks): Module does not add flathub remote when not explicitly specified #166

Open Oakleafknight06 opened 6 months ago

Oakleafknight06 commented 6 months ago

I run a custom image, (https://github.com/Oakleafknight06/startingleaf), installed on two systems, my main desktop and a second test "desktop" intel nuc. On the main pc, default-flatpaks works correctly. On the second one, with the current install (from iso made from ublueos/isogenerator) it doesnt seem to have run at all, and on the previous install showed the notification but didnt seem to do anything. The previous install was from fedora silverblue iso, and a rebase to my own image. On both that install and this current one, I found flathub (nor any other flatpak remote) seem to be enabled by default. On the first time installing I manually added flathub, on the second install I have not.

This is the output of systemctl status system-flatpak-setup on my main pc

○ system-flatpak-setup.service - Manage system flatpaks
     Loaded: loaded (/usr/lib/systemd/system/system-flatpak-setup.service; enabled; preset: disabled)
    Drop-In: /usr/lib/systemd/system/service.d
             └─10-timeout-abort.conf
     Active: inactive (dead) since Sat 2024-03-16 08:01:53 CST; 12h ago
    Process: 6094 ExecStart=/usr/bin/system-flatpak-setup (code=exited, status=0/SUCCESS)
   Main PID: 6094 (code=exited, status=0/SUCCESS)
        CPU: 195ms

Mar 16 08:01:53 silverbird system-flatpak-setup[6094]: Adding system-wide remote  from
Mar 16 08:01:53 silverbird system-flatpak-setup[6108]: error: Remote "" not found in the system installation
Mar 16 08:01:53 silverbird system-flatpak-setup[6094]: Setting title  for remote
Mar 16 08:01:53 silverbird sudo[6160]:     root : PWD=/ ; USER=gdm ; ENV=DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/42/bus DISPLAY= ; COMMAND=/usr/bin/notify-send 'Flatpak Installer' 'Started install of system flatpaks' '--app-name=Flatpak Installer' -u NORMAL
Mar 16 08:01:53 silverbird system-flatpak-setup[6170]: F: An error was encountered searching remote ‘flathub’ for ‘’: Unable to load summary from remote flathub: While fetching https://dl.flathub.org/repo/summary.idx: [6] Couldn't resolve host name
Mar 16 08:01:53 silverbird system-flatpak-setup[6170]: F: An error was encountered searching remote ‘gnome-nightly’ for ‘’: Unable to load summary from remote gnome-nightly: While fetching https://nightly.gnome.org/repo/summary.idx: [6] Couldn't resolve host name
Mar 16 08:01:53 silverbird system-flatpak-setup[6170]: error: No remote refs found for ‘’
Mar 16 08:01:53 silverbird sudo[6212]:     root : PWD=/ ; USER=gdm ; ENV=DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/42/bus DISPLAY= ; COMMAND=/usr/bin/notify-send 'Flatpak Installer' 'Finished install of system flatpaks:\\norg.gnome.TextEditor#012org.gnome.World.PikaBackup#012org.mozilla.Firefox' '--app-name=Flatpak Installer' -u NORMAL
Mar 16 08:01:53 silverbird systemd[1]: system-flatpak-setup.service: Deactivated successfully.
Mar 16 08:01:53 silverbird systemd[1]: Finished system-flatpak-setup.service - Manage system flatpaks.

versus on the second pc

○ system-flatpak-setup.service - Manage system flatpaks
     Loaded: loaded (/usr/lib/systemd/system/system-flatpak-setup.service; enabled; preset: disabled)
    Drop-In: /usr/lib/systemd/system/service.d
             └─10-timeout-abort.conf
     Active: inactive (dead) since Sat 2024-03-16 19:28:25 CST; 1h 0min ago
    Process: 4725 ExecStart=/usr/bin/system-flatpak-setup (code=exited, status=0/SUCCESS)
   Main PID: 4725 (code=exited, status=0/SUCCESS)
        CPU: 424ms

Mar 16 19:28:25 butterfly system-flatpak-setup[4805]: sudo: unknown user darin
Mar 16 19:28:25 butterfly system-flatpak-setup[4805]: gdm
Mar 16 19:28:25 butterfly system-flatpak-setup[4805]: sudo: error initializing audit plugin sudoers_audit
Mar 16 19:28:25 butterfly system-flatpak-setup[4812]: error: No remote refs found for ‘’
Mar 16 19:28:25 butterfly sudo[4836]:     root : unknown user darin#012gdm ; PWD=/ ; USER=darin#012gdm ;
Mar 16 19:28:25 butterfly system-flatpak-setup[4836]: sudo: unknown user darin
Mar 16 19:28:25 butterfly system-flatpak-setup[4836]: gdm
Mar 16 19:28:25 butterfly system-flatpak-setup[4836]: sudo: error initializing audit plugin sudoers_audit
Mar 16 19:28:25 butterfly systemd[1]: system-flatpak-setup.service: Deactivated successfully.
Mar 16 19:28:25 butterfly systemd[1]: Finished system-flatpak-setup.service - Manage system flatpaks.
fiftydinar commented 6 months ago

I wonder if manually adding flathub remote to recipe fixes this error. It is indeed noted in docs that flathub will be used by default if not specified, but maybe that function is broken? I remember 1 user that also had this issue.

Sudo errors are for notify-send, which can't function on login screen (gdm), so that's normal.

Oakleafknight06 commented 6 months ago

How would I manually add flathub remote to the recipe?

fiftydinar commented 6 months ago

How would I manually add flathub remote to the recipe?

You would add remote like this in recipe:

repo-url: https://dl.flathub.org/repo/flathub.flatpakrepo
repo-name: flathub
Oakleafknight06 commented 6 months ago

And this would add it as a system remote, yes? EDIT ah yes as it's under the system: header not the main one

fiftydinar commented 6 months ago

And this would add it as a system remote, yes?

You add it in system section in your recipe, yes.

xynydev commented 6 months ago

How would I manually add flathub remote to the recipe?

You would add remote like this in recipe:

repo-url: https://dl.flathub.org/repo/flathub.flatpakrepo
repo-name: flathub

Also add repo-title: Flathub (system).

Oakleafknight06 commented 6 months ago

Added that into the image now. First without the repo-title, then with. (though it shouldn't change things as far as i'm aware). Now (after each of those) the output of systemctl status system-flatpak-setup looks the same as it does on the main pc.

(ie, this)

○ system-flatpak-setup.service - Manage system flatpaks
     Loaded: loaded (/usr/lib/systemd/system/system-flatpak-setup.service; enabled; preset: disabled)
    Drop-In: /usr/lib/systemd/system/service.d
             └─10-timeout-abort.conf
     Active: inactive (dead) since Sun 2024-03-17 20:54:53 CST; 3min 34s ago
    Process: 4615 ExecStart=/usr/bin/system-flatpak-setup (code=exited, status=0/SUCCESS)
   Main PID: 4615 (code=exited, status=0/SUCCESS)
        CPU: 379ms

Mar 17 20:54:53 butterfly system-flatpak-setup[4626]: /usr/bin/system-flatpak-setup: line 23: yq: command not found
Mar 17 20:54:53 butterfly system-flatpak-setup[4627]: error: Invalid remote name
Mar 17 20:54:53 butterfly system-flatpak-setup[4615]: Adding system-wide remote  from
Mar 17 20:54:53 butterfly system-flatpak-setup[4631]: error: Remote "" not found in the system installation
Mar 17 20:54:53 butterfly system-flatpak-setup[4615]: Setting title  for remote
Mar 17 20:54:53 butterfly sudo[4676]:     root : PWD=/ ; USER=gdm ; ENV=DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/42/bus DISPLAY= ; COMMAND=/usr/bin/notify-send 'Flatpak Installer' 'Started install of system flatpaks' '--app-name=Flatpak Installer' -u NORMAL
Mar 17 20:54:53 butterfly system-flatpak-setup[4696]: error: No remote refs found for ‘’
Mar 17 20:54:53 butterfly sudo[4725]:     root : PWD=/ ; USER=gdm ; ENV=DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/42/bus DISPLAY= ; COMMAND=/usr/bin/notify-send 'Flatpak Installer' 'Finished install of system flatpaks:\\ncom.raggesilver.BlackBox#012org.gnome.TextEditor#012org.gnome.World.PikaBackup#012org.mozilla.Firefox' '--app-name=Flatpak Installer' -u NORMAL
Mar 17 20:54:53 butterfly systemd[1]: system-flatpak-setup.service: Deactivated successfully.
Mar 17 20:54:53 butterfly systemd[1]: Finished system-flatpak-setup.service - Manage system flatpaks.

Nevertheless, flatpak remotes still shows nothing and there are no flatpaks installed.

Oakleafknight06 commented 6 months ago

Not seeing the notifications either btw

fiftydinar commented 6 months ago

Oh, I know what is the issue.

yq was not copied to the system on Build Action on 1.1.0 & below.

Please update Build Action yml to latest 1.2.0 & issue will be solved.

Oakleafknight06 commented 6 months ago

okay. (makes sense theres an error about yq). How would I do that?

Oakleafknight06 commented 6 months ago

Okay so its in .github/workflows/build.yml (I'm still new to this). Should dependabot keep that up to date? Regardless, if I manually edit uses: blue-build/github-action@v1.1.0 to uses: blue-build/github-action@v1.2.0 that will fix this issue right?

fiftydinar commented 6 months ago

Okay so its in .github/workflows/build.yml (I'm still new to this). Should dependabot keep that up to date? Regardless, if I manually edit uses: blue-build/github-action@v1.1.0 to uses: blue-build/github-action@v1.2.0 that will fix this issue right?

It should. If dependabot somehow didn't notify you of the new release, go to repo Insights, Dependency graph, Dependabot, Recent update jobs & check the status of what happened there.

That's correct way to manually edit.

Oakleafknight06 commented 6 months ago

Had already bumped the version number by editing the file, but I'll give that a look and do that next time

Oakleafknight06 commented 6 months ago

Also, with that change, it still doesn't seem to work properly. On reboot the service might not have run? I didn't see a notification. Ran it manually with systemctl and it showed the notification as if it ran and completed successfully but no flatpaks are installed

Oakleafknight06 commented 6 months ago

Okay so its in .github/workflows/build.yml (I'm still new to this). Should dependabot keep that up to date? Regardless, if I manually edit uses: blue-build/github-action@v1.1.0 to uses: blue-build/github-action@v1.2.0 that will fix this issue right?

It should. If dependabot somehow didn't notify you of the new release, go to repo Insights, Dependency graph, Dependabot, Recent update jobs & check the status of what happened there.

That's correct way to manually edit.

Dependabot wasn't enabled, now it is. should work in the future. thanks!

Oakleafknight06 commented 6 months ago

I'll try in a bit adding another default to see if it installs on my main system, as well as reinstall from iso with the new image and see what each of those do

Oakleafknight06 commented 6 months ago

After some testing and typo fixing, it now works when I run it manually. (with systemctl start system-flatpak-setup.service' ) The issue seems to be with network I think. It fails to download the flatpaks and then stops. I suspect this might be related to the vpn autoconnecting at login. Secondly, there doesn't seem to be an obvious place it tells you if you typoed the name of the flatpak. I did, and only realized while troubleshooting by runningsystemctl status system-flatpak-setup`. So perhaps that could be improved somehow

fiftydinar commented 6 months ago

The issue seems to be with network I think. It fails to download the flatpaks and then stops. I suspect this might be related to the vpn autoconnecting at login.

That's possible. I never used VPN, so I didn't test how the module behaves with it. It also depends on how VPN is implemented.

Secondly, there doesn't seem to be an obvious place it tells you if you typoed the name of the flatpak.

Do you mean the check of flatpak names in the list? If so, you can just do

cat /usr/share/bluebuild/default-flatpaks/system/install

which will output the list of flatpaks you inputted. Location of that list is present in the docs.

You can also see the content of the flatpak list in build-time logs before looking inside the flatpak list on booted system.

xynydev commented 6 months ago

Secondly, there doesn't seem to be an obvious place it tells you if you typoed the name of the flatpak.

Automatically checking whether the Flatpak ID exists in the repo would require interfacing with the repository, not sure if that would be hard...

fiftydinar commented 6 months ago

Automatically checking whether the Flatpak ID exists in the repo would require interfacing with the repository, not sure if that would be hard...

If he means the verification of flatpak IDs themself, there was a talk about this a while back in Discord about having this integrated in the build-time, since Bluefin had the mistake in pre-installing Pods due to wrong ID, which caused constant retry of the service & unnecessary CPU usage.

However, it can happen that flathub repo could be down for maintenance reasons sometimes, so builds would falsely fail.

If he means on simplifying flatpak list to use direct names instead of IDs, I'm afraid that wouldn't be possible without making a mess.

Although, this is a separate topic not related to this issue.

Oakleafknight06 commented 6 months ago

That's possible. I never used VPN, so I didn't test how the module behaves with it. It also depends on how VPN is implemented.

Yea I think this is the issue, from further testing. Otherwise it seems it works now

Oakleafknight06 commented 6 months ago

If he means on simplifying flatpak list to use direct names instead of IDs, I'm afraid that wouldn't be possible without making a mess.

I'm noting that it seems easy to make a typo in the id of the flatpak and somewhat harder to notice that was the error, making troubleshooting perhaps more difficult. And as you said it's a separate topic, just something I noticed troubleshooting for this issue

fiftydinar commented 6 months ago

I'm noting that it seems easy to make a typo in the id of the flatpak and somewhat harder to notice that was the error, making troubleshooting perhaps more difficult. And as you said it's a separate topic, just something I noticed troubleshooting for this issue

Thanks for the reminder of that issue anyway, I will open it separately.

Oakleafknight06 commented 6 months ago

That's possible. I never used VPN, so I didn't test how the module behaves with it. It also depends on how VPN is implemented.

Yea I think this is the issue, from further testing. Otherwise it seems it works now

So to fix one could just not have a vpn autostart, or maybe default-flatpaks could start on a longer delay or retry in a minute or two if it fails

fiftydinar commented 6 months ago

So to fix one could just not have a vpn autostart, or maybe default-flatpaks could start on a longer delay or retry in a minute or two if it fails

Maybe systemd service unit could be optimized a bit better in some areas.

If you can test how the default-flatpaks module behaves with different unit settings, it would be good to know the feedback & if there is a potential fix for your VPN scenario.

Oakleafknight06 commented 6 months ago

Maybe systemd service unit could be optimized a bit better in some areas.

If you can test how the default-flatpaks module behaves with different unit settings, it would be good to know the feedback & if there is a potential fix for your VPN scenario.

Sure I could do that. Might be a while between tests as I'm somewhat busy but i'd love to help improve the project if I can

Oakleafknight06 commented 6 months ago

Still being very unreliable with additional testing, even running manually. However, I have run into what seems to be an additional issue. When I add a second instance of the default-flatpaks module, in order to add a second repo (gnome-nightly specifically), that takes precedence over the previous instance and no longer adds flathub or any of the packages to the install list. In the second instance of default-flatpaks I did not list any flatpaks to install, only the repo.

Oakleafknight06 commented 6 months ago

Okay, looking at the systemd unit file now. It says Restart=on-failure . What is the failure condition? This could be near the core of the problem, as it runs the service and reports a success without actually installing the desired flatpaks

fiftydinar commented 6 months ago

When I add a second instance of the default-flatpaks module, in order to add a second repo (gnome-nightly specifically), that takes precedence over the previous instance and no longer adds flathub or any of the packages to the install list.

This is a known issue, where default-flatpaks module would need a refactor: https://github.com/blue-build/modules/issues/146

Okay, looking at the systemd unit file now. It says Restart=on-failure . What is the failure condition?

Failure is usually triggered when repo instance is down or because of some local network issue.

fiftydinar commented 6 months ago

Before refactoring the module, we need to form bash module code guidelines, where all modules would be refactored to conform those. So that would be a chance to address some existing issues like yours & possibly add some better functionality to the modules.

https://github.com/blue-build/modules/issues/157

Oakleafknight06 commented 6 months ago

Failure is usually triggered when repo instance is down or because of some local network issue.

It seems like it might not be doing this quite right. I'll look at it further sometime soon.

Otherwise it seems like everything should be fixable with a refactor when that comes about. 👍🏼

Oakleafknight06 commented 6 months ago

This is a known issue, where default-flatpaks module would need a refactor:

146

Should this be updated in the docs then? As currently the docs imply that multiple repos work

fiftydinar commented 6 months ago

Should this be updated in the docs then? As currently the docs imply that multiple repos work

I updated it as soon as you mentioned it, it's available on website now. https://github.com/blue-build/modules/commit/2f304f0d7133b77048b210fbcd80a6ed85d46dd2

https://blue-build.org/reference/modules/default-flatpaks/#known-issues