search

Derecho-Project / derecho

The main code repository for the Derecho project.
BSD 3-Clause "New" or "Revised" License
187 stars 47 forks source link

Split configuration into 2 files #268

Closed etremel closed 9 months ago

etremel commented 9 months ago

To make it easier for administrators of Derecho groups to manage configuration files, I thought it would be better if the derecho.cfg file was split into two files: One that must be identical on all the nodes, and one (much smaller) that contains only node-specific options. The file that contains identical data for all nodes could then be written once and copied to all the nodes in an automated fashion (i.e. using a script with scp), and only the node-specific configuration files would need to be edited individually on each node. Most options, such as the various buffer and block sizes, are expected to be the same on all nodes, and only a few, such as the local_id, need to be set separately for each node.

This branch changes the Conf initialization routine to expect two files: One that represents the shared group configuration, named "derecho.cfg" by default, and one that represents the local per-node configuration, named "derecho_node.cfg" by default. The shared group configuration is loaded first, then the local configuration, so any options that appear in both files will end up with the values specified in the local config file. This allows the local configuration file to "override" an option that's usually specified in the shared configuration file, which is useful in the case of the port numbers: The port numbers are specified in the shared configuration file, and in most deployments they should be the same on all nodes, but you can specify different port numbers in the local configuration file if you want to run a test deployment where the "nodes" are all running on the local machine and are only distinguished by their port numbers.

Since this is a significant change to the Derecho interface, I understand that it might not be part of the v2.4.0 release, but I thought I would open this pull request anyway so that we can merge it when we are ready.

songweijia commented 9 months ago

This change will affect a lot of the existing applications. I'm wondering if we should keep it in another branch. Btw, we haven't officially release v2.4.0. Let's do v2.4.0 first?

etremel commented 9 months ago

Actually, upon re-examining the code I wrote, I realized that this change will not require any changes to existing applications, because it will seamlessly degrade to the old behavior if "derecho_node.cfg" is not present. I did not add any code that checks for the presence of certain configuration options in each config file, and both config files are only parsed if they exist, so if "derecho.cfg" contains all of the config options, and "derecho_node.cfg" does not exist (or contains no config options), it will not produce any errors. So this code allows you to add a second config file, but does not require you to.

songweijia commented 9 months ago

Oh, OK. I got it. That's because it's just simply load 'derecho.cfg' and 'derecho_node.cfg'. If the latter one does not exists, it gives up loading it. However, the options are 'softly' separated in the two files. Cool, I will merge it!