Remote configuration interface (REST / Item repository)

[GoogleCodeExporter]

1. Feature Description
Provide a REST interface for item configuration
Add item provider that supports interfaces to the REST configuration interface, 
and stores item configuration in a machine read/write format (eg XML/JSON)

<<Note this feature request may need to be broken into separate smaller 
features for implementation, but I've presented it here as a single issue>>

2. Example Use Case
In order to have a (web) configuration interface that allows items to be 
edited/created, the item "repository" needs to be writable as well as readable. 

This could be in the format of a database, but this would impose certain 
requirements on the installation, so the alternative is a machine readable 
format 

such as XML or JSON. Keeping the files in text format also makes it easy for 
the user to manipulate directly if needed. 

<Items>
  <Item>
    <Name>The item name</Name>
    <Label>The label shown in the GUI</Label>
    <Binding>The current binding string</Binding>
    <Icon></Icon>
    <Groups></Groups>
    <Mapping></Mapping>
  </Item>
</Items>

Additional item configuration could then be supported, and these would need to 
be exposed to the /rest/items interface, eg -:
<Description>A text description that could be used for mouseover</Description>

Config parameters can be extended to support (for example) different colours or 
icons for each mapping. Mapping can also be range based rather than a simple 
exchange (ie. specify a minimum/maximum value, and instead of mapping this to 
text, we can set different colours).

Bindings could set their own extended attributes in the repository rather than 
relying on complex binding strings. Bindings could expose their item 
configuration via (eg) XML/JSON so that a web GUI can present a nice interface 
to the user to replace the current binding strings.

Original issue reported on code.google.com by ch...@cd-jackson.com on 22 Aug 2013 at 5:30

[GoogleCodeExporter]

Really nice presented!
I would like to contribute with a XML structure suggestion about the last idea 
of "own extended attributes" that describe the binding item configurations. In 
fact what I suggest is a XML file that describes the whole binding, providing 
some meta data about type, author, version, minimal required openHAB version, 
as well as item configuration params:

<binding>
    <name>KNX<name> //the name of the binding
    <type>protocol</type> //binding type, e.g. "protocol", "ui", "other"
    <author>Thomas E.-E.<author>
    <version>1.3.4</version> // version of the binding
    <oh_version>1.3</oh_version> // minimal required openHAB runtime version

    <protocol_params> // protocol related params only when type = protocol 
        <synthax>[{direction}][{dpt}:]{ga}</synthax>
        <allow_multiple>true</allow_multiple>
        <multiple_glue>,</multiple_glue>

        <direction>
            <required>false</required>
            <options>
                <option>
                    <name>Can read bus</name>
                    <value>lt</value>
                </option>
            </options>
        </direction>

        <dpt>
            <required>false</required>
            <options>
                <option>
                    <name>1.001</name>
                    <value>1.001</value>
                </option>
                <option>
                    <name>1.008</name>
                    <value>1.008</value>
                </option>
                <option>
                    <name>3.007</name>
                    <value>3.007</value>
                </option>
                <option>
                    <name>5.001</name>
                    <value>5.001</value>
                </option>
            </options>
        </dpt>

        <ga>
            <required>true</required>
        </ga>

    </protocol_params>
</binding>

Original comment by mishoboss on 23 Aug 2013 at 12:47

[GoogleCodeExporter]

Thanks for the input. I definitely like the extra header information - a really 
useful input. For the binding definition, I see the binding moving away from 
binding strings, and binding data being stored as parameters in an XML file or 
something similar. So, the string definition you've put is not necessary I 
think.

I think we should stick to defining parameters and their relationship.

I've started to implement this, and have it basically working (both in OH and 
within a web interface). Currently I've defined the following (of course I'm 
more than happy to change/refine this with more ideas)...

There are four sections
1) The initial binding information section
2) The general binding configuration information (is the data in openhab.cfg)
3) Interface configuration (eg for bindings that have multiple interfaces -: 
owserver:interface1.host - the interface1 part).
4) Item configuration defining the different parameters of the item

I have two comments that I haven't answered yet -:
1) There should be a link between the item configuration and the 'interface' 
configuration so that users get to choose interfaces that are configured.
2) Does it need more validation settings (maybe regex, data type, length - 
maybe they can all be covered by a regex?).

<!-- Binding configuration for the OWServer binding in openHAB -->
<binding>
    <!-- Full name presented to the user -->
    <name>OWServer - EDS One Wire Server<name>
    <!-- PID name used within the OSGi layer -->
    <pid>owserver</pid>
    <!-- Binding type: protocol, ui -->
    <type>protocol</type>
    <!-- Author -->
    <author>Chris Jackson<author>
    <!-- Current version of the binding to which the file is applicable -->
    <version>1.3.0</version>
    <!-- Minimum version of openHAB required to run this binding -->
    <oh_version>1.3.0</oh_version>

    <!-- Binding configuration settings -->
    <binding.config>
        <setting>
            <name>cache</name>
            <label>Cache Timeout</label>
            <description>Sets the cache timeout in milliseconds.</description>
            <optional>true</optional>
            <default>5000</default>
            <minimum>250</minimum>
            <maximum>20000</maximum>
            <values></values>
        </setting>
        <setting>
            <name>timeout</name>
            <label>Timeout</label>
            <description>Sets the number of milliseconds that the binding will wait for a response.</description>
            <optional>true</optional>
            <default>5000</default>
            <minimum>500</minimum>
            <maximum>10000</maximum>
            <values></values>
        </setting>
        <setting>
            <name>granularity</name>
            <label>Granularity</label>
            <description>Set the time between each of the OWServer item update checks (in milliseconds).</description>
            <optional>true</optional>
            <default>1000</default>
            <minimum>100</minimum>
            <maximum>10000</maximum>
            <values></values>
        </setting>
    </binding.config>

    <!-- The following lines specify the communications interface settings. These may be repeated multiple times -->
    </binding.interface>
    <setting>
            <name>host</name>
            <label>IP Address</label>
            <description>Set the IP address of the OW Server.</description>
            <optional>false</optional>
            <default></default>
            <minimum></minimum>
            <maximum></maximum>
            <values></values>
        </setting>
        <setting>
            <name>port</name>
            <label>IP Port</label>
            <description>Set the IP Port number of the OW Server.</description>
            <optional>true</optional>
            <default>80</default>
            <minimum>1</minimum>
            <maximum>65535</maximum>
            <values></values>
        </setting>
    </binding.interface>

    <!-- Item binding configuration -->
    <binding.items>
        <binding.io>
            <type>Input</type>
            <description></description>
            <parameters>
                <parameter>
                    <name>interface</name>
                    <label>Communications Interface</label>
                    <description>Set the symbolic name of the communication interface.</description>
                    <optional>false</optional>
                    <default></default>
                    <minimum></minimum>
                    <maximum></maximum>
                    <values></values>
                </parameter>
                <parameter>
                    <name>romid</name>
                    <label>ROM ID</label>
                    <description>Set the ROM ID of the One Wire device.</description>
                    <optional>false</optional>
                    <default></default>
                    <minimum></minimum>
                    <maximum></maximum>
                    <values></values>
                </parameter>
                <parameter>
                    <name>parameter</name>
                    <label>Parameter Name</label>
                    <description>Sets the Parameter Name from the One Wire device.</description>
                    <optional>false</optional>
                    <default></default>
                    <minimum></minimum>
                    <maximum></maximum>
                    <values></values>
                </parameter>
                <parameter>
                    <name>Polling period</name>
                    <description>Sets polling period (in milliseconds between each request.</description>
                    <optional>false</optional>
                    <default>5000</default>
                    <minimum></minimum>
                    <maximum></maximum>
                    <values></values>
                </parameter>
            </parameters>
        </binding.io>
    </binding.items>
</binding>

Original comment by ch...@cd-jackson.com on 29 Aug 2013 at 6:18

[GoogleCodeExporter]

Hi Chris, 
I'm glad I've contributed with something.

>> "I've started to implement this, and have it basically working (both in OH 
and within a web interface)"

So, you have already implemented this idea into the openHAB and it works? If 
this is the case, I'll say two things "well done" and "where can I find the 
code?" :)

>> "For the binding definition, I see the binding moving away from binding 
strings, and binding data being stored as parameters in an XML file or 
something similar. So, the string definition you've put is not necessary I 
think." 

Well, I'm not sure if Kai and Thomas will be happy to completely discard the 
current item configuration. My bet is they will want your contribution as a 
completely separate and optional binding, without touching openHAB core. Maybe 
I'm wrong, but to do that, you have to stick with the current item config 
syntax. Or I'm wrong?

>> "1) There should be a link between the item configuration and the 
'interface' configuration so that users get to choose interfaces that are 
configured."

Yes. A dropdown with configured interfaces must be presented on every item 
config. Maybe it could be generated automatically and showed in the UI only if 
there are sub-nodes in <binding.interface> node? Or maybe if there are any 
interface entries stored (e.g. they could be predefined and locked for 
editing). Interfaces should have unique ID associated to each, which I think 
should be generated, not editable.

>> "2) Does it need more validation settings (maybe regex, data type, length - 
maybe they can all be covered by a regex?)."

Yes, a regex parameter is really a good idea. I think there should be 
"<type>Input</type>" for every parameter in item config section. So, you could 
provide inputs, checkboxes, radios, dropdowns.

Original comment by mishoboss on 2 Sep 2013 at 12:49

[GoogleCodeExporter]

>>So, you have already implemented this idea into the openHAB and it works? If 
this is the case, I'll say two things "well done" and "where can I find the 
code?" :)
Ok - more explanation required :)
Currently, I'm in the process of writing a web admin console (HABmin). I've 
boilerplated quite a lot, and have largely implemented this binding interface - 
both within the web GUI, and it also reads the openhab.cfg file. I haven't 
currently implemented write. I've attached a picture.

To keep the changes manageable within OH, I'm modifying the rest.io bundle, 
however it doesn't see that core bundles can be updated on their own (unless 
someone knows how?) so this is an issue. Once I sort this out, I'll release the 
code. Actually, the web interface is on GitHub, so hopefully others will be 
interested :)

I know Kai wants to change the way the REST interface is handled, so I 
anticipate that my current code is temporary (for proof of concept, testing 
etc) and it will need to change when the architecture is updated.

>>Well, I'm not sure if Kai and Thomas will be happy to completely discard the 
current item configuration. My bet is they will want your contribution as a 
completely separate and optional binding, without touching openHAB core. Maybe 
I'm wrong, but to do that, you have to stick with the current item config 
syntax. Or I'm wrong?

No - I think you may be right. However, in my opinion, this isn't the best way. 
If we stick with the binding strings, they are a bit limited. We could 
effectively hide this from the user in the way you've described, and maybe this 
is where we'll end up. I think moving away from the strings into a 
parameterised format is better, but the downside is that it's a bigger change, 
and needs updates to all bindings :((

Original comment by ch...@cd-jackson.com on 2 Sep 2013 at 1:12

Attachments:

• habmin_bindingconfig.png

[GoogleCodeExporter]

Hi guys,

Although I have absolutely no time for this at the moment, I think this is an 
important discussion and I don't want to make you feel that I am not interested 
in it or do not appreciate it - the opposite is the case!
Not having read all details of your ideas, I just want to throw in a few 
thoughts:
- Entries in openhab.cfg: All of these entries are parsed and then fed into the 
OSGi Configuration Admin service. So it is perfectly possible to write any 
other mechanism that does the configuration (or parts of it) and which does not 
have to know about openhab.cfg. See e.g. the Configuration Admin binding, which 
actually allows to configure openHAB through its own web UI :-)
- In OSGi there is the Metatype service, which allows to describe configuration 
parameters and thus to automatically create configuration UIs from this 
information. This is e.g. done for the Felix Webconsole, see 
http://bryanhunt.wordpress.com/2010/07/07/configuring-osgi-services-with-apache-
web-console-and-metatype/. I would like to have this information added to all 
openHAB addons, so it could be used by things like your anticipated UIs.
- Every binding defines an interface "XXXBindingProvider", which is exported by 
the bundle. The "XXXGenericBindingProvider" is then just one implementation of 
it - the one that knows how to parse the configuration strings from an item 
file. We can definitely think of other ways to provide such binding 
information. E.g. something like a structured key/value map, possibly again 
described through Metatype. This way, we could introduce a way that makes it 
easier for configuration UIs to handle such configs (and not having to deal 
with the current config strings).
- adding new configuration/admin services to the REST API requires imho the 
introduction of user roles, so we should think about this as well. So you could 
e.g. have "admins" and "users".
- Items/sitemaps etc. are currently EMF models, i.e. object trees in memory. 
The items and sitemap files are just one possible serialization of them. 
Actually, I planned to also introduce binary serialization as this will speed 
up loading on low-power devices like the RaspPi. Drawback is that it is not 
user-editable, but if we provide REST services for access and a UI for it, this 
might not be such a big problem.

So much for now, I hope you can get a grasp, where I would like to head in the 
future - but I am certainly open-minded and appreciate other opinions :-)
And now I have get back to the release preparations...

Original comment by kai.openhab on 2 Sep 2013 at 8:55

• Changed state: Accepted

[GoogleCodeExporter]

Hi Kai,
Understand you're busy, so no worries. Hopefully once you've got more time we 
can take a look at this - I ought to have a reasonable GUI up and running by 
then (it is open source on GitHub - cdjackson/HABmin).

Currently, I'm concentrating on the client side, but I'm also modifying the 
rest.io bundle to give myself the OH end. I fully expect this to change once 
you've worked out how the REST interface will finally look.

Currently I'm parsing the openhab.cfg file in the same way as openhab does - I 
haven't added writing the file out yet, but that should be easy. I've also got 
a similar parsing of the items files largely working, and should be able to 
serialise these back to disk (sitemaps are next). Again, this isn't the best 
way (far from it), but for now it allows me to develop the UI with a working 
system.

I'll take a look at the link - it looks interesting.

Anyway, I'll leave you to the release prep, and we can discuss this further 
once it's done. I should also have the UI quite well advanced in that time :)

Original comment by ch...@cd-jackson.com on 2 Sep 2013 at 9:22

[GoogleCodeExporter]

Migrated to https://bugs.eclipse.org/bugs/show_bug.cgi?id=423546

Original comment by kai.openhab on 8 Dec 2013 at 8:13

• Changed state: Invalid