Define README format for sub-bundles of bindings

[kaikreuzer]

We have more and more bindings that are split into several bundles, where there is a main bundle that holds the binding.xml and the README.md as specified in the docs and additional bundles that bring (optional) support for specific devices/bridges.

Examples are the Bluetooth binding with the bluetooth.blukii bundle, the MQTT binding, Zigbee (currently in openHAB) etc.

We are so far missing a template/recommendation for the READMEs within those additional bundles, so that these can be easily integrated into a documentation website. My suggestion would be to have a placeholder in the main README to identify the place, where the optional bundle content can be included. I can be left to the documentation site whether the content is included inline or as a link to a sub-page.

This issue is imho also closely related to https://github.com/eclipse/smarthome/pull/1853 as it also adds content/documentation for specific thing-types.

//cc @cdjackson & @ghys: FYI as you might have input on this topic as well.

[Tony080]

My suggestion would be to have a placeholder in the main README to identify the place

I do agree with the suggestion. It looks quite clear when I step into a binding folder. However, it's hard to know there are some bundles behind a binding(like Bluetooth binding and ZigBee binding you listed.) by just reading the documentation. Since the bundles usually come up with specified devices/bridges, I believe a "Hardware configuration" part is commonly needed in a bundle's README.md. For example, TI2531 Coordinator configuration in ZigBee binding.

[cdjackson]

A few thoughts from me...

The danger with this is we end up with a very big readme, so I would like to suggest a possibility where the binding author can insert small "snipits" into the main readme for each sub-binding. This would allow the main readme to contain short summaries of each binding - possibly broken into sections that make sense for the binding (eg hardware overview, channel overview...) that can be placed appropriately, and then link out to more detailed information relevant to the sub-binding.

I think this is pretty much in line with the suggestion from @kaikreuzer other than just "a placeholder" we effectively allow "a number of placeholders".

As an example, and using the example from @Tony-Hu, I personally don't think the TI coordinator config should really be in the docs as it is - and certainly we shouldn't have detailed hardware installation information for every dongle included into a main readme as this could make it a very large document.

What I think would work though is to include a short overview of the hardware, with a link to a more detailed description. Again using the TI2531 as an example, we might want a short hardware overview of the TI dongle in the main readme so that users get information on the hardware, but detailed information about how to flash the firmware should probably be in a separate page (it's fine when there's just a single dongle, but when there's 4 or 5, it makes it too long).

So, as a suggestion, something like

{{link file}}

to link to another file,

or

{{include file}}

to include the data inline.

This is conceptually not too dissimilar to what we have done with the ZWave documentation where each thing is split into a separate file.

[sjsf]

I have though about this issue here for a while now. To me it looks like our requirements are not too complicated at all.

After at first being tempted to completely over-engineer it, I fear this stuff can quickly get messy and will turn out far too complicated for everybody: binding developers as well as solution providers who need to feed that to some generator.

One restriction we clearly have is that the "main" binding can define the points where extensions can hook their snippets in, but not the other way round. Meaning: the main binding documentation cannot refer (or link) to specific extension snippets, as it doesn't "know" them. It can only tell where they should be placed.

Also, we have to keep in mind that our READMEs currently are plain markdown. Although we are using jekyll in ESH, I don't think that it would be feasible allowing the full power of the jekyll templating language here. Therefore, simply sneaking placeholder(s) into the markdown by (mis)using some mostly-harmless™ construct (like e.g. an HTML comment) might not be as powerful, but AFAICS good enough for our purpose.

So to me it currently looks like there are several stages to which extent we can make this work:

---

Stage A

The most simple solution would be to simply allow listing of the available extensions like this:

# Bluetooth Binding

This binding provides support for generic Bluetooth devices. 
It has the following extensions:

<!--list-subs-->

It then could be rendered like this:

The links obviously would lead to the full readme of the extension, which would include all details which are useful in addition the the "main" binding documentation. A docu generator can decide whether the sub-sites are listed in a navigation tree or not.

The rendering mainly would need to replace the placeholder with a list of links to the extension pages. It can find them by looking for dots in the names, filtering those which spare their first part with the main binding name (i.e. automatically linking all bluetooth.* extensions to stay in this example).

This solution would be most trivial to implement, but also not flexible at all.

---

Stage B

A little bit more advanced would be something which allows extensions to also provide a markdown snippet which the main page would include.

# Bluetooth Binding

This binding provides support for generic Bluetooth devices.

<!--include-subs-->

This would allow extensions to contribute their own content to the main page, be it just a paragraph or an entire subsection:

The [Bluetooth BlueZ Adapter](../bluetooth.bluez/readme.md) extension supports Bluetooth access via BlueZ on Linux (ARMv6hf)

The concrete content structure of such a snippet or course is mainly predetermined by the main page. It could be a plain paragraph, a sub(sub..)-section, or maybe even just some lines for a table. This might be different though for each binding: E.g. the bluetooth binding might have other requirements than the zigbee binding.

A potential rendering could look like this then:

The rendering would simply need a convention for the name of the snippet (e.g. INTRO.md) and then could look them up using the same heuristic for the names a above.

Compared to the previous solution this provides a little more flexibility, but mainly allows the binding developer to influence a little the style of the contribution.

---

Stage C

When looking e.g. at the bluetooth binding, it becomes apparent immediately that there are different "kinds" of extensions: The bluez bundle brings an additional bridge, while the blukii bundle brings new thing types. Ignoring this fact and simply allow a uniform inclusion of all of them in the same place doesn't seem too compelling though, considering that they would potentially get mixed up by sorting them alphabetically.

Therefore it could make sense to allow extensions to contribute different "kinds" of snippets. The main page would include only a particular "kind" at each place accordingly:

# Bluetooth Binding

This binding provides support for generic Bluetooth devices. It has the following extensions:

<!--list-subs-->

## Bridges

In order to function, this binding requires a Bluetoooth adapter to be present, which handles the wireless communication.
As there is no standard in Bluetooth for such dongles resp. chips, different adapters require a different implementation.
This is why the Bluetooth binding itself does not come with any bridge handlers for such adapters itself, but instead is extensible by additional bundles which can implement support for a specific adapter. 

<!--include-subs INTRO_BRIDGES-->

## Supported Things

These two thing types are supported by this binding itself:

| Thing Type ID | Description                                                                                             |
|---------------|---------------------------------------------------------------------------------------------------------|
| beacon        | A Bluetooth device that is not connected, but only broadcasts annoucements.                             |
| connected     | A Bluetooth device that allows a direct connection and which provides specific services when connected. |

<!--include-subs INTRO_THINGS-->

An extension then would simply contribute an INTRO_BRIDGES.md and/or INTRO_THINGS.md file, containing an arbitrary markdown snippet.

Again, the format (plain paragraph, sub(sub...)section(s)) of the snippet is predetermined by the main page and up to each binding to decide which fits best to the technology. The names (or kinds) of the snippets also are not defined universally by actually are left to the main binding page to decide what is required.

In addition to the previous solution, here the rendering would need to parse the second part of the placeholder for the given name. Still something which I would assume every renderer or a simple script should be capable of. On the other hand we would gain a pretty high degree of flexibility while still staying pretty much agnostic wrt the rendering technology.

---

I think this can be extended and enhanced endlessly. Nevertheless, to my understanding this already will give us all the flexibility we need. And while Stage A should be pretty easy to implement, I don't think adding the further stages adds too much complexity to the generators to not do it.

What do you think?

[kaikreuzer]

Sounds quite straight-forward, but I have one concern, though (which mainly applies to stage c): Our current archetype splits the information about a single thing into separate sections: list of the defined type, which ones are discovered, their configuration, their channels and a full example to see everything interacting.

I feel that if we want to come to stage c (and also to be able to generate this content from the thing xmls), we will have to think about a general restructuring of the README, so that one Thing is described fully in a single section (incl. its configuration, channels, etc.).

If we now decide that for the start a or b is enough, we can postpone such a decision. But it would mean that any sub-readme that is written now will have to be re-written later on.

[sjsf]

Sure, we might think as well about restructuring the readme files.

Nevertheless, having split up the information across several sections is exactly the purpose of "Stage C": You could as well have multiple snippets per extension (e.g. SNIPPET_DISCOVERY.md, SNIPPET_THING_TYPES.md, SNIPPET_CHANNELS.md) and include them accordingly.

I second @cdjackson though, this doesn't make sense for huge bindings, but might be feasible for bindings with very few, small extensions (e.g. where bridges are added by extensions).

Overall, stuff like that might get hard to maintain pretty quickly. On the other hand it is hard to keep an overview either way in huge bindings. That is what we will need #1853 for.

[kaikreuzer]

So what's your recommendation as a first step? Go for stage a or b and leave the content of such sub-readmes completely open/unspecified?

[sjsf]

Yes, my approach would be like that.

At this point I don't see a documentation structure which would work perfectly well across all extensible bindings. However, I think it would make sense to experiment a little on a concrete binding we have already and see how it feels. Maybe in that way we at least can derive a recommendation for future extensible bindings. I suggest to start with the bluetooth binding as it is within the same repository and therefore can be adapted forth and back easily if needed.

As a first step I will try to adapt our docu generation here accordingly. Let's so how that goes...

[sjsf]

As with #5611 we currently have a solution that fits our (current) needs, let's close this one and create a new issue once there are new requirements.