Comments on "core" profile and scope of constraints

[mjkoster]

From the July 30 discussion, it seems like the word "core" is indeed subject to much interpretation, and we need to re-orient the discussion on the scope of constraints.

I think the best definition of what is desired in "core" is to enable out-of-the box interoperability for "consumers" meaning as I understand API consumers rather than someone shopping at Best Buy, but maybe that also.

Who are the consumers?

We didn't discuss as much who the profile is for, what class of users, what application domain. Saying we want OOB interoperability across all application domains covered by Thing Description seems difficult. Defining narrow constraints that enable OOB interoperability for constrained devices seems to me to be a much narrower scope than we have for W3C WoT in general, and should be clearly defined as one profile that is targeted for a specific use case.

local/vertical scope - what does this profile constrain, and what is allowed to be adapted to in the endpoints? A profile for digital twin interaction might be optimized for purpose by allowing multiple protocols, e.g. HTTP and MQTT and favor adaptation. A profile for battery powered smart home/building devices would like CoAP or CHIP and can't be expected to adapt much.

horizontal scope - which application domains does the profile optimize for? Are we calling "the web" a domain at the same time we're expecting to support constrained devices that can't adapt to a few base protocols? Should we instead target profiles to stakeholder groups where we enable smart home/building as an ecosystem, etc.?

What if we want to create a WoT Profile to enable TD to be used to automate industrial fieldbus deployments and add digital twin support through OMA LWM2M objects and data models that are compatible IEC61499 function blocks. We might need to define some network interaction model and "protocol bindings" for some fieldbuses like OPC-UA, modbus, and BACnet. The out of box interoperability scenario would be different from consumer smarthome but still quite important.

If we create a profile that is required, would it apply to both smart home/building and industrial fieldbus profiles? What is the tradeoff between narrow scope for the experience of some users vs. broader scope for coverage across industry? If we apply one "core" profile to all of WoT, is that consistent with our overall objectives?

[benfrancis]

The way I think about this is to imagine a smartphone app which acts as a generic Web of Things browser/client (unfortunately as far as I know this doesn't exist yet, but I'd love to create one). What are the constraints that the developer of a greenfield device (or gateway/cloud service bridging brownfield devices) must follow in order to guarantee out-of-the-box interoperability with that client?

This is the lowest common denominator which all devices wanting out-of-the-box interoperability must conform to. Devices can then add additional features (e.g. additional protocols, data formats and semantic annotations) to provide an enhanced experience for specialised clients for a specific use case or application domain.

---

I think the best definition of what is desired in "core" is to enable out-of-the box interoperability for "consumers" meaning as I understand API consumers rather than someone shopping at Best Buy

Agreed.

Who are the consumers?

WoT clients (including gateways which act as clients).

We didn't discuss as much who the profile is for, what class of users, what application domain.

A WoT client supporting the core/default profile should be able to communicate at a basic level with devices across any application domain.

what does this profile constrain?

• At least one (and ideally only one) protocol which all conforming devices must support (I suggest HTTP)
• At least one (and ideally only one) serialisation format for protocol payloads (I suggest JSON)
• Protocol methods, headers, payload formats and expected response codes (including error conditions) for the requests and responses used to carry out the full set of operations on a device (currently this includes: readproperty, writeproperty, observeproperty, unobserveproperty, invokeaction, subscribeevent, unsubscribeevent, readallproperties, writeallproperties, readmultipleproperties and writemultipleproperties)
• A constrained set of security schemes
• A constrained set of discovery mechanisms

What is allowed to be adapted to in the endpoints?

Conforming devices can:

• Implement additional protocols and serialisation formats in addition to the mandatory ones
• Add domain-specific metadata as semantic annotations in Thing Descriptions to provide additional information to specialised clients so that they can provide more specialised UI for particular use cases

A gateway can also be used to bridge any other protocol or data format to the core/default profile.

A profile for digital twin interaction might be optimized for purpose by allowing multiple protocols, e.g. HTTP and MQTT and favor adaptation.

All web things are digital twins so I don't understand why this keeps coming up as an example of a profile.

A profile for battery powered smart home/building devices would like CoAP or CHIP and can't be expected to adapt much.

There are a couple of options for developers in this case:

1. Create a separate resource constrained profile for devices which can't implement the core/default/HTTP+JSON profile (best for greenfield devices)
2. Implement a gateway which bridges these protocols to the core profile (best for brownfield devices)

which application domains does the profile optimize for?

None.

Are we calling "the web" a domain

No.

Should we instead target profiles to stakeholder groups where we enable smart home/building as an ecosystem, etc.?

No, because this will harm cross-domain interoperabilty, which is the stated goal of the Web of Things. "The W3C Web of Things (WoT) is intended to enable interoperability across IoT platforms and application domains." --WoT Architecture

What if we want to create a WoT Profile to enable TD to be used to automate industrial fieldbus deployments and add digital twin support through OMA LWM2M objects and data models that are compatible IEC61499 function blocks. We might need to define some network interaction model and "protocol bindings" for some fieldbuses like OPC-UA, modbus, and BACnet. The out of box interoperability scenario would be different from consumer smarthome but still quite important.

These are (with one arguable exception) not web protocols and should really be out of scope for the Web of Things. If you really want to describe these devices using a WoT Thing Description you can create a protocol binding for that protocol (if they have URI schemes...), but if you want to achieve out-of-the-box interoperability with these devices on the Web of Things you should probably implement a gateway which bridges them to the core profile instead.

If we create a profile that is required, would it apply to both smart home/building and industrial fieldbus profiles?

This question doesn't make sense, but the core/default profile should span all application domains.

What is the tradeoff between narrow scope for the experience of some users vs. broader scope for coverage across industry?

We can have both, by using a progressive enhancement approach. A device can provide a basic set of functionality for generic WoT clients and an enhanced set of functionality for specialised domain-specific clients.

If we apply one "core" profile to all of WoT, is that consistent with our overall objectives?

Yes, because the objective is interoperability across IoT platforms and application domains. If you have separate profiles for separate parts of the Web of Things you create interoperability within application domains/platforms but not between them.

Developers can always choose to implement the core/default profile (either in the device itself or in a gateway or cloud service) in return for out-of-the-box interoperability (with optional enhancements for specialised clients), or not follow the profile at all and use whatever protocol and serialisation format they like, at the expense of out of the box interoperability.

[sebastiankb]

I think interoperability is a very broad term. The intention of WoT is to increase interoperability in IoT by specifying building blocks. One of the building block is the TD which provides a common understanding what Thing's assets are such as its properties, actions, events, its data model, which protocol and security is used etc. I think it is already a great win for IoT to have such a common knowledge entry point (comparable to index.html) that can be further used for discovery, validation, onboarding, etc.

Now it comes to the question what I can do with the content of a TD, especially when I see that there is, e.g., a protocol which I do not support. I’m not sure if this question is really relevant for clients without any domain context. As you know IoT is very diverse and has different facets in different domains like building, industry, healthcare, etc. In that domain established protocols are used and will be also used in the future, e.g., to fulfill time-critical and real-time requirements. Lets take Modbus. Modbus is relative simple and lightweight protocol which can be described very easily by a TD (which is very cool). Consequently, a client that is mainly developed for domains where Modbus is kind of usual I would expect that such clients also support (at least) the Modbus binding. And yes, there is an OOB interoperability for Modbus-based scenarios without having a profile 😉

Unfortunately, HTTP based communication is not that simple like Modbus. HTTP is very open in terms of used payload media formats, sub-protocols, etc. I can understand it would be desirable that there is some basic default assumptions for HTTP-based use cases (e.g., for Thing2Browser, Thing2Cloud) which gives suggestions of the content type (e.g., JSON), protocol behavior, security handling, etc. I think that (generic) clients, which will support http+json anyway, will simply support such a standard profile as long as the profile is quite easy to implement. A producer, for example, who is planning to launch a new Thing, will be happy if there is a nice default profile that fits well with planned technology usage like JSON+HTTP and is easy to follow. In doing so, the implementation is also rewarded by the fact that the probability increases that there will be OOB interoperability for http-json-based use cases using generic clients.

@benfrancis

The way I think about this is to imagine a smartphone app which acts as a generic Web of Things browser/client (unfortunately as far as I know this doesn't exist yet, but I'd love to create one).

Maybe, a lightweight version of WADE can be ported to an mobile app? https://github.com/tum-esi/wade

[benfrancis]

@sebastiankb I agree with most of what you said and we seem to be aligned on the benefits of an HTTP+JSON default profile.

With regards to the topic of scope, there are two things I often hear from WoT Working Group members that surprise me:

1. A Thing Description may not always be fetchable from a web URL, it may be shared by some other means
2. A Thing Description should be able to describe interaction affordances using non-web or non-internet protocols or protocols which don't have a standardised URI scheme

What I don't understand is if a Thing Description is not fetched over the web and does not link to any web resources, what does it have to do with the web and therefore the Web of Things? These are both totally valid things for developers to be doing, but why would such a technology be standardized at the World Wide Web Consortium?

As I understand it property, action and event affordances in a Thing Description have a mandatory forms member, where each Form control has a mandatory href member which must be an IRI/URI.

Does modbus have a URI scheme? I don't see it in this list. (I can't see BACnet either but perhaps there is a non-standard scheme used by some implementations like with MQTT?). If not then how do you describe a modbus device with a Thing Description? And if you can, presumably only modbus clients can communicate with the device. In what way do you think that Thing Description improves interoperability between devices in a way that isn't already possible with modbus?

Wanting a metadata file format to describe a modbus device is a totally valid requirement for a developer to have, but modbus is not a web technology (and therefore unrelated to the Web of Things) and depending on which version of modbus you're using may not even be an internet technology (and therefore possibly even unrelated to the Internet of Things). Why is it relevant to the W3C? Why not standardize that file format at the IETF or or the Modbus Organization and keep the scope of the Web of Things to the web? Or more helpfully, if you do want to communicate with a modbus device from a WoT client in order to integrate it with a wider WoT system, instead of describing a device that most WoT clients won't be able to communicate with, why not use a gateway or service to bridge that device to the Web of Things using a web protocol instead?

In case people are unsure what I mean when I say "web", in practice there are currently only two web protocols I am aware of: HTTP and CoAP. To be generous, here is a much more broad definition of the web from The Verge:

To count as being part of the web, your app or page [or thing] must:

1. Be linkable, and
2. Allow any client to access it.

That second point is my understanding of interoperability, that any thing should be accessible by any client.

The Web of Things I hear some members describing doesn't meet either of these criteria.

If different, what is your understanding of "interoperability" and "the web"?

As I said, I think we agree on the need for and benefits of an HTTP+JSON profile with some sensible defaults. I just wanted to raise the above points because I think they are at the root of most disagreements on the topic of scope.

[egekorkan]

I would agree that what we are doing and have been doing is not Web in the classical sense. However, the work of the group has shown that since the Web has evolved in a way to bring interoperability across different systems of servers and clients, we can do the same for IoT devices. Maybe everything will converge to a handful of protocols as the classical Web has, but it will not be the case for IoT for a long time in my opinion. We simply cannot ignore them since we have shown that we can indeed apply some Web-like techniques to those protocols as well. If the organizations that have developed these protocols had applied them, we would not need to do this work. Modbus or BACnet "people" may not even know about IANA but it does not make their out-of-scope of the work here.

Regarding more specific comments:

What I don't understand is if a Thing Description is not fetched over the web ... what does it have to do with the web and therefore the Web of Things?

I really don't think that where a TD is fetched changes anything. TD is metadata in the end. I find it similar to the documentation of an API, where it doesn't matter where I get the documentation of the API as long as I can trust it and know what backend I am talking with.

What I don't understand is if a Thing Description ... does not link to any web resources, what does it have to do with the web and therefore the Web of Things?

The forms of a TD are not links to a resource in the classical sense, since as you said, a Modbus protocol endpoint is not a link, the href and other vocabularies in the forms explain how to send the request to that device and that is it. If we look at a more classical but not-REST Web, we also find protocols like JSON-RPC that use a URI only to send data to and there is no real resource behind it, thus they are not links to concrete resources. Similarly, if I open a WebSocket connection at a URI, the URI plays no real importance anymore, the resources are somewhere else and the socket allows a custom protocol to access them.

Instead of describing a device that most WoT clients won't be able to communicate with, why not use a gateway or service to bridge that device to the Web of Things using a web protocol instead?

In that case, how would you program the gateway? Using TDs, we can significantly reduce the implementation effort of such bridging. node-wot can do this with only 10 lines of code since the Scripting API allows the abstraction of protocols once the device has a TD. Without a TD, you would read mostly PDF documentations of such devices or in the best case HTML documents that are human but not machine-readable.

[benfrancis]

@egekorkan wrote:

the work of the group has shown that since the Web has evolved in a way to bring interoperability across different systems of servers and clients, we can do the same for IoT devices.

Then why ignore the lessons of how the web actually achieved that interoperability and instead take a completely different approach which is a Web of Things in name only?

Maybe everything will converge to a handful of protocols as the classical Web has, but it will not be the case for IoT for a long time in my opinion.

I'm not expecting all IoT protocols to disappear in favour of HTTP, that would be a terrible idea. Similarly IMAP, POP and SMTP did not disappear because webmail arrived. The web provided an application layer on top of those other protocols. It didn't provide a metadata format for describing all possible email protocols and an email scripting API to communicate with them.

In that case, how would you program the gateway?

With adapters that bridge different protocols to the web. Like the 100+ adapters listed here. (A WoT Thing Description is useless for describing most protocols that connected devices in the smart home use like Zigbee, Z-Wave, Bluetooth, HomeKit, X10, INSTEON, ONVIF etc.)

Using TDs, we can significantly reduce the implementation effort of such bridging. node-wot can do this with only 10 lines of code since the Scripting API allows the abstraction of protocols once the device has a TD. Without a TD, you would read mostly PDF documentations of such devices or in the best case HTML documents that are human but not machine-readable.

This is very misleading. Creating a Thing Description for a modbus device doesn't magically enable a client to understand the modbus protocol, you still have to implement the protocol itself from that human readable specification. The Thing Description makes a negligible difference to interoperability if clients have to implement hundreds of different protocols in order to make use of them.

So anyway, I think a core/default profile which tells WoT clients how to communicate with WoT devices in a standard way over HTTP is a good idea and I've described above what I think that profile would need to contain :) If the WoT profile specification doesn't provide that then I will make sure that the Web Thing Protocol specification does, because based on lessons from the World Wide Web I believe that is the key to providing real ad-hoc interoperability on a Web of Things, according to my understanding of those terms.

[draggett]

From a commercial perspective, have we a clear idea of what matters in relation to the main business cases for IoT? I suspect that an open market of services will emphasise interoperability across the Internet as the basis for combining heterogeneous sources of information. The protocols at the network edge will be an obscure detail for specialists to deal with, as across the Internet, market forces will drive convergence on standards that minimise the barriers to interoperability. I therefore very much support work on common standards for using HTTP over TLS.

For a more amusing version, see: this taskera blog post.

At the moment, the Web of Things IG/WG can be likened to a bunch of friends who are enthusiastic innovators. We're yet to transition to the early adopter phase as we have yet to see significant commercial deployment of the Web of Things. Geoffrey Moore's famous chasm is thus still some way off in the future for the Web of Things.

As enthusiastic innovators we do things because we think they are fun. That isn't enough for the early adopters who need to see compelling business advantages for adopting a new technology. Our work on standards should pay careful attention to that!

[egekorkan]

Regarding what @benfrancis wrote in the comments above:

1. I think what you want is a specific protocol, otherwise even if the Thing only uses HTTP or HTTP based protocols, if there is no specific way to use HTTP (which there isn't), you would have significant implementation effort anyways. That is why such efforts on creating a good protocol for WoT should be done by the WebThing protocol community group and not by the profile TF.
2. I think the scope of WoT does not involve protocols that require special hardware, like Bluetooth, Zigbee, etc. They need to be always bridged. The goal of WoT is so that already HTTP (just as an example) speaking Things do not have to be bridged. Once one creates a middleman like Mozilla Webthings, OpenHab and other similar ones, they start to even bridge HTTP based Things since they cannot go out of their narrowly focused view on how a client should communicate with Things. They generally do not look good once you look deep into their APIs since they generally don't have to have an API that needs to by entities outside of the framework, the user interface handles everything. Once a client outside of the framework needs to interact with the API, they need to learn the way the API works by reading mostly only human readable documentation.

[benfrancis]

@egekorkan wrote:

I think what you want is a specific protocol

Correct. The alternatives I can see being:

1. A Thing Description specification which can describe any use of HTTP - Currently the Thing Description can not do this (e.g. dynamic resources, collections of resources, responding to success/error conditions). This may be achievable by adding more features to the Thing Description specification until it is more or less feature equivalent to OpenAPI, but that adds a lot of complexity for clients to interpret at runtime and makes ad-hoc interoperability challenging.
2. A Thing Description specification which can describe any use of any protocol - This is only possible by defining "protocol binding templates" and inventing URI schemes for protocols which don't formally have them (e.g. MQTT, modbus, BACnet etc.) and it makes cross-platform/cross-domain ad-hoc interoperability effectively impossible.

efforts on creating a good protocol for WoT should be done by the WebThing protocol community group and not by the profile TF.

OK, fair enough. Part of what I'd hoped to figure out was whether the core/default profile would fulfill the requirements for an HTTP sub-protocol, or whether I should continue to pursue that through the Web Thing Protocol Community Group. (Although if the profile isn't going to provide this level of interoperability then I'm not sure what practical purpose it serves).

I think the scope of WoT does not involve protocols that require special hardware, like Bluetooth, Zigbee, etc.

Agreed. I also don't think it should include non-web protocols which require special software.

The goal of WoT is so that already HTTP (just as an example) speaking Things do not have to be bridged.

This isn't currently possible for all uses of HTTP and isn't possible at all for protocols which don't have URI schemes.

Once one creates a middleman like Mozilla Webthings, OpenHab and other similar ones, they start to even bridge HTTP based Things since they cannot go out of their narrowly focused view on how a client should communicate with Things. They generally do not look good once you look deep into their APIs since they generally don't have to have an API that needs to by entities outside of the framework, the user interface handles everything. Once a client outside of the framework needs to interact with the API, they need to learn the way the API works by reading mostly only human readable documentation.

I agree with this. In the case of WebThings Gateway I think there's a risk that the gateway itself (and in particular its add-ons system) becomes the platform, rather than the Web of Things as a whole. I was very conscious of this when designing it. But this is true of other implementations too and I don't think the Web of Things has yet reached the point where client (consumer) and server (producer) implementations can be used interchangeably, mainly because there's just too much diversity in what is currently considered a "web thing". That is the problem I'm hoping to solve. As I mentioned above I think a native mobile (or desktop) app which acts as a general purpose Web of Things client might help bring some of these issues into focus.

If you have to write additional code in order to get a particular WoT client to talk to a particular WoT device then I think the Web of Things has failed.

[egekorkan]

I agree with 100% of the comment above :) We actually have the WADE, which is an electron-based desktop application that uses node-wot to consume TDs, send requests to Things and more that can done via plugins. It is of course not a very professional project but pretty good already :) You can check it out at https://github.com/tum-esi/wade

[benfrancis]

I propose closing this as a duplicate of #3, #10 and #5.