Provide an API for the evdev.xml registry

[whot]

Note: this is a discussion topic for now, not an issue

Right now, tools parse evdev.xml to get the list ("registry") of all possible models, layouts and their variants, and options. Those are then displayed to the user who selects a subset of them and that selection is then passed as RMLVO to libxkbcommon.

These days the rules are usually hardcoded as "evdev" for historical reasons. For the same reasons, evdev.xml is hardcoded as the path to parse.

Ideally, libxkbcommon should/could provide this registry as well so we don't rely on a third-party source and parser.

A few notes:

• evdev.xml is generated by xkeyboard-config by merging the various parts xml together. A second evdev.extras.xml is available for those options that are too niche to be included in the normal xml file.
• evdev.lst is the deprecated older format and generated from evdev.xml but the conversion is lossy. Specifically, it drops allowMultipleSelection="true" and a few other lesser-used tags.
  • There is no evdev-extras.lst.

Implementation-wise I see two options:

• a full API that provides the information from the xml file, e.g. xkb_registry_layout_get_variants() to get the list of variants from the registry.
• a simpler API that passes a file descriptor to the underlying XML file. That leaves the parsing up to the client and we merely control what they open.

The latter is simpler right now but gets trickier with #108 merged. If we want to allow for $XDG_CONFIG_HOME/xkb/rules/evdev.xml (and we should to have this show up in GUIs) we need to merge the XML files together ourselves and then provide the fd to the merged XML file.

Final comment: this is mostly independent of libxkbcommon so it could be a separate library altogether. The primary reason why it should be part of libxkbcommon is that it should share the same include paths so the list always matches what libxkbcommon can actually load. That could be achieved by having a libxkbcommon-registry.so. Or a completely external library that takes a struct xkb_context and calls xkb_context_include_path_get() and then goes from there.

Comments?

[whot]

After thinking about this some more - there is no point putting this into libxkbcommon other than for convenience maybe. The entity with the primary context is the compositor, but the compositor is not the configuration tool that presents the various options to the user. In the GNOME stack that's mutter vs gnome-control-center (or even gnome-tweak-tools). The latter never has access to the libxkbcommon context anyway so trying to shoehorn this in here is rather pointless.

So doing this as external library and hoping the two agree on include paths seems the best and only option.

[whot]

fwiw, I started a project called libxkbregistry which has the basic boilerplate and header file to indicate what I envision this could look like.

[bluetech]

Thanks for pushing this effort and sorry for the long delay!

Overall the XML files pack some useful information which will be nice to expose. But, I am still interested to know, what is the intended audience and use-cases you envision for this functionality?

At the low-level, I can see two ways to use the information:

• Enumerate the items (with filtering). The audience for that would mostly be UIs which allow the users to pick their MLVO.
• Obtain information on a particular choice. Can be shown in status bars etc. instead of the info from he XKB files themselves. Not sure how easy it to reverse though (e.g. layout -> XML info, rather than the other way).

Another thing where the XML files came up is in the effort to allow custom XKB configurations, where we need some way to augment the system's XML files such that UIs will show the custom options. From what I understand, this is main driving force for this proposal - the library will handle includes, users switch to it and get it for free, and then we can finally control the entire chain and do it right.

The question is, will such projects choose to use libxkbcommon for this, or still prefer to parse the XML files directly?

Implementation-wise I see two options:

Regarding the particulars of the API, I'll leave that for after we decide to go forward, especially when you have libxkbregistry set up already which we can use as a base point.

Final comment: this is mostly independent of libxkbcommon so it could be a separate library altogether.

So we have three layers:

• Use the xkbcommon organization - definitely.
• Use the libxkbcommon repo - I think that would be better, separate repo just adds overhead.
• Use the same .so - My opinion on this depends on one thing - whether the functionality requires an external dependency (XML parser) or not. If not, then I don't see a reason to split it to a separate shared object. But assuming yes, then I wouldn't want libxkbcommon.so to depend on an XML parser, so libxkbcommon-registry.so would be better. We can handle it like we handle libxkbcommon-x11.so and instruct distros to use a separate package for it if they want, etc.

The latter never has access to the libxkbcommon context anyway so trying to shoehorn this in here is rather pointless.

Presumably they will create a compatible context (in regards to include paths mostly) and use that.

[whot]

Thx for the feedback, much appreciated. To answer your questions (or rather confirm):

Right now, "everything" that needs to display the list of available keyboard layouts parses the evdev.xml file directly. With #108 that breaks however since they don't know about the new includes. And, tbh, having an xml-based side-channel is a tad odd anyway.

So the audience will be gnome-control-center, installers, etc. whatever previously parsed the xml file and can now use a library that takes care of special includes.

My prime use-case right now is enumeration, because filtering can easily be done in the caller anyway. Might be that we can add a filtering API later but it's not immediately needed.

The question is, will such projects choose to use libxkbcommon for this, or still prefer to parse the XML files directly?

won't know until we try but I'm pretty sure everyone would be happy to switch to a stable library interface that handles the magic so they don't have to. The effort to rewrite the XML parser to handle include files correctly is about the same as switching to a library that does it for you.

And yes, we will need an XML parser because that's what xkeyboard-config ships. I've been trying to think of a way around it but I'm not happy with any of them. The closest thing would be a systemd-hwdb approach of a tool that compiles to a binary database and then loading that but that has too many corner-cases (doubly so with users editing files in their $HOME).

Having it in the same repo would help with sharing util functions, the current one copies a lot of build, doxygen, etc setup stuff from xkbcommon already.

edit: libxkbcommon-registry.so is my current favourite solution

[whot]

Just a note as a brain backup: the whole idea of this is obviously that the caller only says "load evdev rules" and this API provides the list of MLVO based on the appropriate evdev.xml.

This will still require the user to write an $XDG_CONFIG_DIR/xkb/rules/evdev.xml that gets loaded first, and it will require that xml to contain some XInclude tag to include the system files. This makes our new xml file the master document but unfortunately the xkeyboard-config xml format doesn't lend itself to extension easily. So some trickery is likely required to get the equivalent of the !include %s that we have in libxkbcommon.

Case in point: we only have onemodelList, layoutList, optionList allowed per the DTD. I'm not sure anyone actually cares if there are two of those but.. well. Haven't found a good solution here short of rebuilding the original structure with lots of <xi:include xpointer=... tags.

[whot]

I've just pushed the first working implementation including XInclude support to https://github.com/whot/libxkbregistry - @bluetech if you want to have a look