search

ElektraInitiative / libelektra

Elektra serves as a universal and secure framework to access configuration settings in a global, hierarchical key database.
https://www.libelektra.org
BSD 3-Clause "New" or "Revised" License
208 stars 123 forks source link

KDB_DB_HOME and KDB_DEFAULT_RESOLVER #3724

Closed kodebach closed 1 year ago

kodebach commented 3 years ago

Setting the KDB_DB_HOME CMake variable only works, if you also change the KDB_DEFAULT_RESOLVER variable. The default resolver is resolver_fm_hpu_b. This version does not use the KDB_DB_HOME value in any way.

Currently, Jenkins jobs (and a few other build servers) use KDB_DB_HOME, but AFAICT none of them set KDB_DEFAULT_RESOLVER to any value that would use KDB_DB_HOME.

We should set KDB_DEFAULT_RESOLVER=resolver_fm_b_b on all build servers that use KDB_DB_HOME and don't already set a different KDB_DEFAULT_RESOLVER.

@markus2330 we should consider making the KDB_DEFAULT_RESOLVER=resolver_fm_bhpu_b to prefer KDB_DB_HOME (if set) over all the other ways of determining a home directory. However, I'm not sure if it is as easy as changing the default, or if that would need changes in the resolver code (e.g. because it assumes KDB_DB_HOME always has a value if it is used).

markus2330 commented 3 years ago

Thank you for reporting this problem! I agree that KDB_DB_HOME is not so intuitive, probably we should completely remove this CMake variable (and the underlying feature). It is only used in fallback mode if /etc/passwd did not contain a home directory. And also in this situation, it needs the USER environment to work. It probably makes more sense to directly use an environment variable that contains the whole home directory, which actually could be XDG_CONFIG_HOME.

kodebach commented 3 years ago

I don't think we should remove this variable. It is immensely useful for development/test builds. Without it you will always create some clutter in your home folder. There is KDB_DB_USER which at least allows us to avoid actual collisions, but the development/test build still depends on system configuration. With KDB_DB_HOME you can isolate the build completely.

However, you are right setting KDB_DB_HOME actually breaks the user:/ namespace in a Multi-User setup and basically converts it into another system wide namespace. Therefore, it is probably a good thing that the default resolver configuration does not use KDB_DB_HOME at all.

Still, the build server configurations should be fixed so they actually use KDB_DB_HOME. And maybe we add a CMake warning if KDB_DB_HOME is set but the configured default resolver doesn't use it.

markus2330 commented 3 years ago

If it is about isolation of builds, maybe something like KDB_DB_ROOT would be better?

kodebach commented 3 years ago

Yes, I like that idea. So the KDB_DB_* options to configure paths would be:

Using this the new KDB_DB_ROOT would result in better isolation than current options (currently dir:/ cannot be isolated AFAIK) and at the same time is significantly simplifies setting up a development/test build.

Since this option should only be used in development/test builds, it should maybe be called e.g. KDB_DEV_DB_ROOT.

markus2330 commented 3 years ago

I was thinking of prefixing everything and always with KDB_DB_ROOT (not only for "dev" as it is better if the resolver we test with and the real resolver is as similar as possible). By default it is /, so it would be the same behavior as now. The use cases would be:

Disadvantage: It would create (deeply) nested folder structure, e.g. with KDB_DB_ROOT=/home/kodebach/.elektra the specifications would be in something like /home/kodebach/.elektra/usr/share/elektra/specification.

KDB_DB_FILE: filename for the default file of a namespace (maybe rename to KDB_DB_DEFAULT_FILE?) KDB_DB_INIT: filename of the file for system:/elektra (maybe rename to KDB_DB_BOOTSTRAP_FILE?)

I like your suggestions. But do we really need this to have configurable at all? What about simply using some hard coded name with the default storage name as file suffix?

  • ~KDB_DB_HOME~: removed

:+1:

kodebach commented 3 years ago

I was thinking of prefixing everything and always with KDB_DB_ROOT (not only for "dev" as it is better if the resolver we test with and the real resolver is as similar as possible).

Also an option yes, but as you said, this would unnecessarily deep nesting.

The use cases would be:

  • having several Elektra's in parallel that should not interfere, e.g. KDB_DB_ROOT=/opt/sthg
  • having all configs in a separate folder for development purposes (so that the rooted Elektra does not interfere with the real configs of the system)

I think we should have separate solutions for these use cases. For parallel installations a suffix is better than a prefix IMO. So e.g. every installation gets it's own folder in /usr/share/elektra/specification. This avoids the long paths and everything is still where you expect it (user:/ data in each users home folder, dir:/ data in $PWD) and not in one central location. For parallel installation I would also want to configure this option at install time and not just at compile time.

For development purposes, I would actually want to have everything in one location (preferably somewhere in the build directory) so I can easily inspect what is going on. So in this case a prefix makes sense.

So my suggestion would be

  1. add a KDB_DB_DEV_ROOT option that causes the resolver to only return paths below the given path. This option would cause the resolver to execute different code. But that is not a problem, because this option should only be used for local development (and maybe one build job for testing). The build jobs run in clean containers/VMs anyway, so they don't need to use this option.
  2. add a KDB_DB_PATH_SUFFIX option that causes the resolver to append a path to the resolved folders before appending the mountpoints filename. So with e.g. KDB_DB_PATH_SUFFIX=myinstall1 the resolver returns /etc/kdb/myinstall1/elektra.ecf instead of /etc/kdb/elektra.ecf for the mountpoint system:/elektra.

But do we really need this to have configurable at all? What about simply using some hard coded name with the default storage name as file suffix?

It's only configurable in CMake. From all other points of view it is essentially hard-coded already. Since the strings are used in multiple places we should use a #define anyway. At that point it doesn't really matter whether you can change it via CMake or not. If it causes is trouble we can remove the CMake config option, but I don't see a reason right now.

markus2330 commented 3 years ago

As argued already, it is not a good idea to have options only for developers. ENABLE_LOGGER, ENABLE_ASAN and ENABLE_DEBUG are already way too much, we should actually reduce them (to keep development environments more consistent). In general, differences in production and development environment should be avoided. Furthermore, the resolver is already way too complicated. We should make it more simple and not add further functionality.

Btw. doesn't KDB_DB_USER already does what we want? To have multiple installations in parallel, you can simply do, e.g. for the xxx installation: -DKDB_DB_SYSTEM="~/.config/xxx/system" -DKDB_DB_SPEC="~/.config/xxx/spec" -DKDB_DB_USER="xxx/user".

I think we can remove KDB_DB_HOME without substitute.

Improving KDB_DEFAULT_RESOLVER is an issue on its own, it should go together with a cleanup of useless variants (only include variants we have use cases for).

kodebach commented 3 years ago

Btw. doesn't KDB_DB_USER already does what we want? To have multiple installations in parallel, you can simply do, e.g. for the xxx installation: [...]

Yes, KDB_DB_USER allows us to configure parallel instances at compile time. But using KDB_DB_ROOT for that was your idea. I actually have no idea, why you suddenly want to support this, when one of Elektra's main goals is to be global configuration storage system.

Improving KDB_DEFAULT_RESOLVER is an issue on its own

The original issue was only about setting KDB_DEFAULT_RESOLVER correctly on the build server (or changing its default value when KDB_DB_HOME is set). The whole tangent of getting rid of KDB_DB_HOME entirely was your idea.

I disagree on the fact that having options for developers is a bad idea, but let's not get into that right now...

I think we can remove KDB_DB_HOME without substitute.

I disagree. I use it currently (together with KDB_DEFAULT_RESOLVER=resolver_fm_b_b) to isolate my development build. And by isolate I mean: Everything (apart from cache files) is contained in my build directory. If I mess something up and e.g. the dump plugin starts writing broken files, I just do rm -rd build and try again. I don't have to clean up lots of files all over my system to get a working dev instance again.

markus2330 commented 3 years ago

The original issue was only about setting KDB_DEFAULT_RESOLVER correctly on the build server

I don't think it is a good idea if we don't use the default resolver on the build server. Most jobs should test the defaults, as also most people will use the defaults.

The whole tangent of getting rid of KDB_DB_HOME entirely was your idea.

I got the idea when you talked about the problems of KDB_DB_HOME in your IDE introduction.

I disagree on the fact that having options for developers is a bad idea, but let's not get into that right now...

It is also not what I said. I said that 3 options are too many, as it easily introduces possibilities that some of these variants make problems, e.g., do not compile anymore. And this is really a no-go as it might shy away people interested to improve Elektra.

I use it currently

Ok, I have to admit this is the use case it was designed for. But can't you achieve something similar via KDB_DB_USER (relatively) pointing to your build directory?

kodebach commented 3 years ago

I don't think it is a good idea if we don't use the default resolver on the build server.

The we shouldn't set KDB_DB_HOME on the build servers. But the current way is neither here nor there.

I got the idea when you talked about the problems of KDB_DB_HOME in your IDE introduction.

The problem was actually that I assumed the build servers are configured "correctly". Once I changed KDB_DEFAULT_RESOLVER everything worked as expected. That's why I wrote the issue.

But can't you achieve something similar via KDB_DB_USER (relatively) pointing to your build directory?

KDB_DB_USER is the suffix that is appended after the home directory is resolved (and before the filename is added). I guess I could still find a way to "bend" the path into my build directory, but it seems a bit unnecessary.

To finally come to a consensus here, I would suggest we simply remove the KDB_DB_HOME setting on the build server (and maybe replace it with KDB_DB_USER ones).

stale[bot] commented 2 years ago

I mark this issue stale as it did not have any activity for one year. I'll close it in two weeks if no further activity occurs. If you want it to be alive again, ping the issue by writing a message here or create a new issue with the remainder of this issue. Thank you for your contributions :sparkling_heart:

stale[bot] commented 2 years ago

I closed this issue now because it has been inactive for more than one year. If I closed it by mistake, please do not hesitate to reopen it or create a new issue with the remainder of this issue. Thank you for your contributions :sparkling_heart:

github-actions[bot] commented 1 year ago

I mark this stale as it did not have any activity for one year. I'll close it in two weeks if no further activity occurs. If you want it to be alive again, ping by writing a message here or create a new issue with the remainder of this issue. Thank you for your contributions :sparkling_heart:

github-actions[bot] commented 1 year ago

I closed this now because it has been inactive for more than one year. If I closed it by mistake, please do not hesitate to reopen it or create a new issue with the remainder of this issue. Thank you for your contributions :sparkling_heart: