Downloading driver files every time the application is opened

[mihalyr]

Hi, it seems some state is broken in my installation, whenever I open the app and connect to a DB (PostgreSQL) it shows me a dialog to download driver files. I click "Download" then it downloads and there are no further issues. But if I restart the app it will ask me to download the same drivers again.

I'm not sure what is happening, but it seems those drivers might be stored at an ephemeral location and destroyed when the app is closed.

DBeaver Community - Universal Database Manager.

          ID: io.dbeaver.DBeaverCommunity
         Ref: app/io.dbeaver.DBeaverCommunity/x86_64/stable
        Arch: x86_64
      Branch: stable
     Version: 22.3.1
     License: Apache-2.0
      Origin: flathub
  Collection: org.flathub.Stable
Installation: system
   Installed: 256.9 MB
     Runtime: org.gnome.Platform/x86_64/43
         Sdk: org.gnome.Sdk/x86_64/43

      Commit: 13e0b0553e6d04b85c02d80072adb015362aa3330fbca45587870030156478a8
      Parent: 02266bab85f7d6fdf560743f1ca525522202e01b750ed55d071ae5f5ec231b2a
     Subject: Fix for changing built-artefects (bd359b69)
        Date: 2022-12-28 07:53:58 +0000

OS: Fedora 37 Silverblue (rpm-ostree) Flatpak: 1.14.1

But other things like my DB configurations, plugins, etc. are not lost, only the drivers for some reason.

In Preferences -> Connections -> Drivers -> Driver location I see /home/mihalyr/.local/share/DBeaverData/drivers and this is what I have at the location:

➜  drivers find -type f -name '*.jar'
./maven/maven-central/net.postgis/postgis-geometry-2.5.0.jar
./maven/maven-central/net.postgis/postgis-jdbc-2.2.1.jar
./maven/maven-central/org.postgresql/postgresql-42.2.5.jar
./maven/maven-central/com.github.waffle/waffle-jna-1.9.1.jar
./maven/maven-central/net.java.dev.jna/jna-4.5.1.jar
./maven/maven-central/net.java.dev.jna/jna-platform-4.5.1.jar
./maven/maven-central/org.slf4j/jcl-over-slf4j-1.7.25.jar
./maven/maven-central/org.slf4j/slf4j-api-1.7.25.jar
./maven/maven-central/com.github.ben-manes.caffeine/caffeine-2.6.2.jar

The folder contents don't change when I close and reopen the app, yet, I'm asked to install the drivers again with the following dialog:

[mihalyr]

I can see that the Download does not really work. locally I have postgresql-42.2.5 but it wants to download 42.5.0 and that does not happen - or it at least is not stored in my .local/share folder.

When I select Force download / overwrite the driver files in .local/share are not overwritten, showing old versions and old dates.

[Eonfge]

Are you sure about this?

I can see the files downloaded in my local folder; ~/.local/share/DBeaverData/drivers/maven/maven-central/org.postgresql

If I then connect to a server, the logging will mention that i use the right version.

[mihalyr]

Are you sure about this?

Yes? That's the reason I opened the issue. Every single time I launch the flatpak app it asks me to install the PostgreSQL driver as described above. Not sure what is happening. I think I'll revert to the regular version for now at least I'll test also https://github.com/flathub/io.dbeaver.DBeaverCommunity/issues/158

[mihalyr]

I'm trying something now:

• delete the flatpak with all data
• backup and delete DBeaverData from .local/share
• reinstall the flatpak again (system)

[mihalyr]

I've noticed another flatpak that did not get installed with the app

runtime/io.dbeaver.DBeaverCommunity.Client.pgsql/x86_64/stable

I'm going to install this manually, seems like the files I needed.

[mihalyr]

This did not help, even after installing that pgsql client flatpak I still get asked to download Postgres drivers on every single run.

I'll try one more thing, remove everything again and try a user flatpak install to see if it helps.

[mihalyr]

I think I've tracked it down to my .local/share/DBeaverData folder.

I had an old installation full of connections and scripts that I wanted to restore. But it somehow breaks the flatpak not sure why. When I remove this folder and do a clean install, I only have to download the Postgres (or any other) drivers once and then on the next run it won't ask me.

But when I copy my configuration from the backup to restore my settings and connections, it will keep downloading the drivers on every run.

I think my config backup is from a regular edition which does not seem to be compatible with the flatpak edition configuration.

[mihalyr]

It seems I found out how to workaround this migration issue

Instead of restoring the full ~/.local/share/DBeaverData folder I only restored the contents of

• workspace6/General/.dbeaver
• workspace6/General/Scripts

Now, I have all of my scripts and connections back and it does not ask me to download drivers on every restart. But I lost my workspace settings, color scheme, non-database configuration, etc. These are not critical, though.

I'm resolving this since the main issue has been fixed.

[mihalyr]

Btw, I narrowed down the problem to DBeaverData/workspace6/.metadata folder. Whenever I restore this, I got back my workspace configuration, colors, etc., but I lose the drivers too.

It is actually the DBeaverData/workspace6/.metadata/.plugins folder that contains my workspace settings and also the one that breaks driver downloads. Not sure yet which plugin is the culprit.

[mihalyr]

@Eonfge Finally, I've figured this out.

In this plugin configuration file ~/.local/share/DBeaverData/workspace6/.metadata/.plugins/org.eclipse.core.runtime/.settings/org.jkiss.dbeaver.core.prefs there is this line I had in my old configuration:

ui.drivers.home=/home/mihalyr/.local/share/DBeaverData/drivers

I've noticed that it is enough to remove this line to make driver downloads working.

The interesting thing on Fedora Silverblue is that the root filesystem is read-only apart from /var so /home is a symlink to /var/home for example.

➜  ~ ls -l /home
lrwxrwxrwx. 3 root root 8 Dec 28 01:54 /home -> var/home

When I don't remove the line above but change it to /var/home then everything works fine:

ui.drivers.home=/var/home/mihalyr/.local/share/DBeaverData/drivers

It seems that either the Flatpak or DBeaver itself is not able to recognize this symlink properly. Could it be that the Flatpak needs some special handling for this to work? Or is it an upstream problem and should report it to DBeaver instead?

[Eonfge]

One questions though; Did you add the ui.drivers.home=/home/mihalyr/.local/share/DBeaverData/drivers line yourself?

If you didnt, that means that there is a serious problem with immutable operating systems.

[mihalyr]

@Eonfge I have never seen this file before that line was there in the config from my previous DBeaver installation (non-flatpak), but when I made a fresh flatpak install that line was not there. Not sure if it was something added by an earlier version of DBeaver that got removed in newer versions, or it was a result of some configuration changes I made, or it is something that came from the RPM I used to have before, but I did not add it myself, for sure.

[mihalyr]

@Eonfge there is indeed something wrong how DBeaver flatpak (don't have the RPM to try installed yet) works with home symlinks. Today I wanted to export SQL results to a CSV in my /home/mihalyr/Downloads folder, the operations succeeded without errors, but file is nowhere to be found - just like with the driver downloads. It took me a while to realize that I had to change the path to /var/home/mihalyr/Downloads in the DBeaver export dialog to make it work.

/home/mihalyr is a symlink to /var/home/mihalyr I expected that to work, now I'm not sure yet if this is only a flatpak issue, but certainly all my other tools flatpak or not are working and only the DBeaver flatpak seems to have problems with it so far (but I don't have too many flatpaks - Slack, Spotify, Zoom, these work, although Zoom is sometimes crashing, but it was crashing for me on Linux before too, so not a good comparison)

I think I should try installing DBeaver from RPM to see if this is flatpak or DBeaver related.

[Eonfge]

@mihalyr Not sure if you're still available to try something out, but I can no longer reproduce the problem you mention. When I start a VM with Fedora Silverblue, all the things you mentioned work:

• I can download drivers
• I can export a CSV to /home/kevin and it correctly appears in /var/home/kevin
• I don't have any weird config in org.jkiss.dbeaver.core.prefs

[mihalyr]

@Eonfge I gave up on it some time ago, but I can try it again.

Nowdays, I just have the tar archive of 23.2.4 extracted into my ~/opt/dbeaver folder and installed the desktop file. I have updates disabled and don't really miss them TBH, they used to break my plugins and I had to reinstall them all the time.

I checked my org.jkiss.dbeaver.core.prefs file and I don't have the ui.drivers.home=/home/mihalyr/.local/share/DBeaverData/drivers line there anymore, maybe I removed it, don't remember. I scanned the file for any other similar paths and I found only one, but that used the /var/home prefix: dialog.default.folder=/var/home/mihalyr/.local/share/DBeaverData/workspace6/General/Scripts

I'll give flatpak another try and let you know.

[mihalyr]

I installed the latest flatpak from Flathub and I get no driver download warnings this time (because I don't have ui.drivers.home pointing to /home but probably it defaults to /var/home which works).

When I go to the CSV export I have the /var/home path pre-selected. I changed it to /home/mihalyr/Downloads manually to see if it works and I see the same problem, the operation looks like it succeeds, there are no errors, but there is no file in /home/mihalyr/Downloads.

If I leave the export path at /var/home/mihalyr/Downloads, the file is created as expected. So this is the same issue as I described in this ticket earlier.

DBeaver even logs this:

2024-09-24 16:04:31.966 - Export to the new file "/home/mihalyr/Downloads/test_20240924.sql.csv"
2024-09-24 16:04:32.158 -   Close output stream

But then:

➜  Downloads ll /home/mihalyr/Downloads/test_20240924.sql.csv
ls: cannot access '/home/mihalyr/Downloads/test_20240924.sql.csv': No such file or directory

I also tried adding ui.drivers.home=/home/mihalyr/.local/share/DBeaverData/drivers to my org.jkiss.dbeaver.core.prefs and I could reproduce the missing driver issue and the driver download dialog popped up. When I changed it to ui.drivers.home=/var/home/mihalyr/.local/share/DBeaverData/drivers it worked again.

Please note that the tar archive version I use works in all of the above cases /home or /var/home does not matter as it shouldn't.

I appears to me that /home is some kind-of virtual FS for DBeaver flatpak and temporary location that only exists inside the flatpak environment, while /var/home points to my real home folder. I'm not sure if this is how other flatpaks works, in which case I can just perhaps accept to use /var/home everywhere with flatpaks instead of the regular /home path. It's not a big deal once you know it, but had to spent some time to find this out, because there are no errors, it just works, only the files are not there were I expected them.

[mihalyr]

I tried one more thing, started DBeaver flatpak and entered it with a shell and I can confirm that the /home path is not the same as /var/home only /var/home is mapped to my host and /home is some virtual location existing only inside the flatpak not seen from the outside. Exporting files to /home go here and I don't see in my real home folder. Same with the drivers, they are downloaded here and then gone on restart, not picked up from my real folder if the config references /home instead of /var/home.

➜  Downloads flatpak enter io.dbeaver.DBeaverCommunity bash
bash-5.2$ cd /home/mihalyr/Downloads 
bash-5.2$ ls -ltr   
total 4
-rw-r--r--. 1 mihalyr mihalyr 2268 Sep 24 16:25 test_20240924.sql.csv
bash-5.2$ cd /var/home/mihalyr/Downloads 
bash-5.2$ ls -l | wc -l
231

Next I tested Slack, another flatpak app and I could reproduce the same issue. First, when the flatpak container starts, it has no /home folder, only /var/home that is mapped to my host as expected. Then I went to download an image from a Slack message, but when the dialog to select a location popped up, I explicitly entered /home/mihalyr/Downloads and Slack saved the image, that I could not find in my normal Downloads folder, but now the location appeared in the flatpak container with the downloaded image.

➜  Downloads flatpak enter com.slack.Slack bash
bash-5.2$ ls -l /   
.flatpak-info  bin/           etc/           lib64/         run/           sys/           usr/           
app/           dev/           lib/           proc/          sbin/          tmp/           var/           
bash-5.2$ ls /home
ls: cannot access '/home': No such file or directory
bash-5.2$ ls /var/home
mihalyr
bash-5.2$ ls /var/home/mihalyr/Downloads/| wc -l
230
bash-5.2$ ls -l /home/mihalyr/Downloads/
total 52
-rw-r--r--. 1 mihalyr mihalyr 50137 Sep 24 16:34 IMG_2787.jpg
bash-5.2$ 

So this looks to me like a more generic flatpak issue not specific to DBeaver. The only thing is that if I use the default dialogs to select paths, the will probably work, but I ran into the issue here maybe because DBeaver flatpak read my config from the non-flatpak version that had /home preset from earlier so I didn't even use the folder select dialog.

All in all, we can close this issue, it is not DBeaver flatpak related, it seems to be a broader issue with the Flatpak + Silverblue combination.