coreos / rpm-ostree

⚛📦 Hybrid image/package system with atomic upgrades and package layering
https://coreos.github.io/rpm-ostree
Other
871 stars 195 forks source link

disable rpmdb monitoring in build/composes #4029

Open cgwalters opened 2 years ago

cgwalters commented 2 years ago

Seen in https://github.com/okd-project/okd-coreos-pipeline/pull/12

(rpm-ostree compose extensions:1): libdnf-WARNING **: 14:13:47.883: failed to setup monitor: Operation not supported

It looks like inotify is being denied in this environment for some reason. But in any case, we shouldn't be trying to monitor the rpmdb for changes when we're doing a build.

LorbusChris commented 2 years ago

copied from https://github.com/okd-project/okd-coreos-pipeline/pull/12#issuecomment-1251133050:

@cgwalters hm, is this not the monitor looking for *.repo files in the compose dir? How else would rpm-ostree know about the yum repos? As the logs show, the c9s.repo is present and defines the baseos repo. If the failed monitor is not the reason, why is it not picked up and what can we do to fix this?

travier commented 1 year ago

Looks fatal from an strace trace:

(rpm-ostree compose extensions:4): libdnf-WARNING **: 14:38:27.859: failed to setup monitor: Operation not supported
[{events=EPOLLIN, data={u32=2147483648, u64=2147483648}}], 1024, -1) = 1
close(3)                                = 0
close(5)                                = 0
close(4)                                = 0
close(8)                                = 0
futex(0x562afeaea5a0, FUTEX_WAKE_PRIVATE, 2147483647) = 1
futex(0x562afeaea4b0, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x7f078ba2b910, FUTEX_WAIT_BITSET|FUTEX_CLOCK_REALTIME, 5, NULL, FUTEX_BITSET_MATCH_ANY) = 0
write(2, "error: Unknown rpm-md repository"..., 41error: Unknown rpm-md repository: baseos
cgwalters commented 1 year ago

Ahh OK pretty sure the problem is this https://github.com/rpm-software-management/libdnf/blob/943e3ac8d9044f126d810a04d49a4d0d3eacdc00/libdnf/dnf-context.cpp#L2273

If inotify fails then the libdnf code doesn't set an error and silently doesn't load the .repo files etc. (This code violates the gerror rules by returning FALSE but not setting an error)

In latest libdnf, this code also changed: https://github.com/rpm-software-management/libdnf/pull/915

cgwalters commented 1 year ago

If inotify fails then the libdnf code doesn't set an error

Hmm wait I may be wrong about that...not totally sure

cgwalters commented 1 year ago
diff --git a/libdnf/dnf-context.cpp b/libdnf/dnf-context.cpp
index ad467aab..b7c74e46 100644
--- a/libdnf/dnf-context.cpp
+++ b/libdnf/dnf-context.cpp
@@ -2272,8 +2272,11 @@ dnf_context_setup(DnfContext *context,
                                G_FILE_MONITOR_NONE,
                                NULL,
                                error);
-        if (priv->monitor_rpmdb == NULL)
+        if (priv->monitor_rpmdb == NULL) {
+            g_printerr ("Failed to monitor rpmdb\n");
             return FALSE;
+        }
+        g_printerr ("monitoring rpmdb\n");
         g_signal_connect(priv->monitor_rpmdb, "changed",
                          G_CALLBACK(dnf_context_rpmdb_changed_cb), context);
     }
@@ -2290,9 +2293,11 @@ dnf_context_setup(DnfContext *context,

     /* initialize repos */
     priv->repo_loader = dnf_repo_loader_new(context);
+    g_printerr ("Got repo loader\n");
     priv->repos = dnf_repo_loader_get_repos(priv->repo_loader, error);
     if (priv->repos == NULL)
         return FALSE;
+    g_printerr ("Got repos\n");

     if (!dnf_context_plugin_hook(context, PLUGIN_HOOK_ID_CONTEXT_CONF, nullptr, nullptr))
         return FALSE;
diff --git a/libdnf/dnf-repo-loader.cpp b/libdnf/dnf-repo-loader.cpp
index c6d4faaa..2e92a029 100644
--- a/libdnf/dnf-repo-loader.cpp
+++ b/libdnf/dnf-repo-loader.cpp
@@ -408,6 +408,7 @@ dnf_repo_loader_refresh(DnfRepoLoader *self, GError **error)
     if (!dnf_context_setup_enrollments(priv->context, error))
         return FALSE;

+
     /* load repos defined in main configuration */
     auto cfg_file_path = dnf_context_get_config_file_path();
     if (cfg_file_path[0] != '\0' &&
@@ -421,16 +422,20 @@ dnf_repo_loader_refresh(DnfRepoLoader *self, GError **error)
     auto repos_dir = dnf_context_get_repos_dir(priv->context);
     for (auto item = repos_dir; *item; ++item) {
         auto repo_path = *item;
+        g_printerr ("looking for yum repos in '%s'\n", repo_path);
         g_autoptr(GDir) dir = g_dir_open(repo_path, 0, NULL);
         // existence of repos directories is not mandatory
-        if (dir == NULL)
+        if (dir == NULL) {
+            g_printerr ("failed to open dir\n");
             continue;
+        }

         /* find all the .repo files */
         while (auto file = g_dir_read_name(dir)) {
             g_autofree gchar *path_tmp = NULL;
             if (!g_str_has_suffix(file, ".repo"))
                 continue;
+            g_printerr ("inspecting %s\n", file);
             path_tmp = g_build_filename(repo_path, file, NULL);
             if (!dnf_repo_loader_repo_parse(self, path_tmp, error))
                 return FALSE;
cgwalters commented 1 year ago
[root@ab613414cbef srv]# /run/hostsrv/src/github/coreos/rpm-ostree.c9/rpm-ostree compose extensions --rootfs=/ --output-dir=/tmp/extensions manifest-c9s.yaml  extensions-c9s.yaml
monitoring rpmdb

(/run/hostsrv/src/github/coreos/rpm-ostree.c9/rpm-ostree compose extensions:6284): libdnf-WARNING **: 16:11:14.159: failed to setup monitor: Operation not supported
Got repo loader
looking for yum repos in ''
failed to open dir
Got repos
error: Unknown rpm-md repository: baseos (discovered 0)
cgwalters commented 1 year ago

https://github.com/coreos/rpm-ostree/pull/4278