ncm-symlink: full rewrite needed

[jouvin]

ncm-symlink deserves a full rewrite based on getTree() API and addressing all the ncm-symlink issues currently open, in particular #288 about ensuring that a symlink definition cannot be added twice to the configuration, leading to unpredictable results.

[ned21]

@jouvin Are you planning a new component so people can run them side-by-side during the transition? cf. #648 I think it would be worth considering having a single component that combines both symlink and dirperm since at present it's perfectly possible to get yourself into a situation where dirperm and symlink try to replace the same path, and the ordering of dirperm and symlink is dependent on how a site configures their depends.

[jouvin]

Currently I was more thinking at rewriting symlink in such a way that it is based on a dict and still remain backward compatible with the old schema (I have not tested my ideas but I think it should be doable). But I have also been thinking that it could be good/useful to merge dirperm and symlink.We could use the ncm-system approach with ncm-chkconfig: a new configuration modules with a clean schema able to digest the schema of the legacy configuration modules. What do you think about it?

[jouvin]

One of the question regarding the schema is the need, if any, of operation ordering. I think the reason for using a list rather than a dict in both configuration module was to be able to address this need. If moving to a dict, it would be painful to invent a complex dependency description... Can we consider that processing the dict entries in the key alphabetical order (the key being the path) is enough? I tend to think that yes as I think the only important thing is to ensure you process the parents before the children...

[stdweird]

for the rewrite, a few remarks/requests

• a big :+1: on merging dirperm and symlink. this is just basic filesystem operations
• any code actually making the modification should be part of CAF (see https://github.com/quattor/CAF/issues/69); or at least be able to factor out if you already want to start coding (like the current code in ncm-postgresql) :smile:
• ideally, the new component can use the old schema, but there is a small difference between this component and ncm-systemd: ncm-systemd does not work on EL6, so ncm-chkconfig is not replaced by ncm-systemd entirely. this component should be able to actually replace it or run in parallel. so it would certainly be nice wrt migration from old ncm-symlink/ncm-dirperm, i don't consider it a requirement. (it's certainly a lot more work, including understanding what and how the old components worked in the first place)
• for chkconfig/systemd, there was some discussion of moving the services to /system/services instead of configuring them directly under the component. not sure if this applies here as well, but better decide on the location early (i have no real preference either way).
• wrt the order, a dict is fine to start with, if needed, we can always use the pre/post ordering mechanism from the components (the code in ncm-ncd is pretty clean and portable, could become a CAF::Resolv). i don;t think order will be required much, and certainly not more complex than what ncm-ncd can do.

[jouvin]

I think we basically agree. Personally, not surprising, I think that if we start a new component (and I think this is a good proposal), ensuring backward compatibility with components it replaces is critical. In the template library for example both of these configuration modules are used a lot and it would be too much time trying to edit any template to get rid of the legacy config modules. My idea is a new component, probably with a new schema, that register for changes to the configuration part of the legacy ones and do what is appropriate to process the information provided according to the schema of the legacy components (probably converting it to the new one when starting to keep the code clean). I'll see if I managed to come with a prototype in the coming weeks...

[ned21]

In terms of order of processing, my thought was that sorting the paths by length of components and processing in order of increasing length would be both useful and safer? At the moment I think you can run into problems where a symlink higher up the tree can cause dirperm to have different effects depending on the order in which the components are run. To give an example/unit test:

symlink = /var/foo -> /opt/foo
directory = /var/foo/bar, mode  = 1777

should result in /opt/foo/bar with mode 1777 because /var/foo has fewer components than /var/foo/bar and therefore /var/foo is processed first.

[jouvin]

Good remark! The sorting order may be added as an option.