[RFC] Replace centralized channel-type definitions with encoded values.

[mcav]

Problem: The centralized enums of { ChannelType, Type, and Value } are tedious to extend, arbitrarily defined, and tend to be very adapter-specific.

Proposal: Rather than defining ChannelType, Type, and Value in centralized enums, we should pass around encoded values, allowing Adapters to decide how to decode that data, something like this:

struct EncodedValue {
  content_type: String, // e.g. “temperature+json”
  data: Vec<u8>
}

Adapters will decide how to decode EncodedValues themselves. This:

• Simplifies the process of adding new adapters and channels;
• Removes the burden of maintaining cumbersome centralized enums and serialization;
• Maintains the notion of “typed” values in a very web-centric way;
• Allows us to transport both binary and JSON data efficiently;
• Allows channels to handle multiple content_types (e.g. image/*) rather than just one.

  Content-Types

Standards already describe a time-tested way of exchanging typed data and defining content types. (While we could certainly go full-throttle with application/vnd.mozilla-temperature+json for things like “temperature” that haven’t been standardized, a simple temperature+json format would be plenty specific.)

Ultimately, we just want a Content-Type to allow the consumer to go from a binary representation to the type they desire. I suggest MIME Types because adapters can take advantage of standardized types (cameras with image/jpeg) and this pushes us toward a web-compatible direction.

Type-Checking and Code Sharing

I still see a role for taxonomy-like types with this approach. We can define types that need to be standardized and allow them to implement serialization to and from EncodedValue objects. This allows us to retain the existing level of type-safety some people prefer, without restricting authors to defining new types in a rigid global enum.

While there are clear benefits to enforcing types, I think we need to take a step back from this full-on, define-the-world-rigidly approach. I think this proposal would greatly simplify both the Taxonomy and FoxBox code, leaving us in a better position for growth and stability. If you disagree, I'd like to better understand the merits of the current method as compared to this proposal.

Example Content-Types

temperature+json => { “C”: 42 }
boolean => 0 | 1
thinkerbell-rule+json => { ... }
color+rgba => 255 255 255 0
color+json => { r: 0, g: 0, b: 0, a: 255 }
image/jpeg
null => (no data)

[Yoric]

Thanks for starting this conversation. I like the direction in which this is heading (also, the connection with the idea you mentioned in #53) , although I believe that your current proposal is not the solution yet.

I have just posted a counter-RFC here.

A few more specific remarks in the meantime:

• yes, a big centralized enum is problematic, three big centralized enums even more so;
• the main purpose of ChannelKind is not type-safety but discoverability (e.g. OvenTemperature is much more specific than temperature) – I don't see anything in your proposal that covers this need;
• the main purpose of Value is also not type-safety, but making sure that all adapters parse from/serialize to the same format – whatever we do, we need to either ensure this automatically (as we do now), or make this the easiest solution for adapter developers, otherwise we're going to end up with a mess;
• the main purpose of Type is type-safety, but yes, you are right, that's not so important at this layer.

Edit Published link to counter-RFC.

[Yoric]

Oh, I forgot other issues that should be solved:

• implementing the Taxonomy API itself, and Thinkerbell in particular, becomes rather awkward if each channel takes as argument a Vec<u8> and passes a Vec<u8> - do we really need to deserialize/reserialize everything for communications that stay inside the process?
• if we decide to go in this direction, responding with Vec<u8> might make it much more annoying for the Adapter developer to publish some data as short-lived URLs.

[mcav]

I'll respond to your points here and then look at your counter-proposal:

the main purpose of ChannelKind is not type-safety but discoverability ... I don't see anything in your proposal that covers this need;

In this proposal, we'd still annotate channels with the types they accept, such that your API could still return 'discoverable' endpoints that describe what they do.

the main purpose of Value is also not type-safety, but making sure that all adapters parse from/serialize to the same format

Yes, as I mentioned under the "Type-Checking" section, if multiple adapters share the same types, we should share the same de/serialization code, which still fits into taxonomy.

implementing the Taxonomy API itself, and Thinkerbell in particular, becomes rather awkward if each channel takes as argument a Vec and passes a Vec

I don't think that's true, but you're right that Thinkerbell would be affected. In practice, I think this just means that Thinkerbell would gain some enum-like type/value conversion code that currently lives in Taxonomy.

if we decide to go in this direction, responding with Vec might make it much more annoying for the Adapter developer to publish some data as short-lived URLs.

I think the opposite is true, or if anything, this is unrelated. I say the opposite might be true, because the types/values here map very well to HTTP content-type and body responses.

[Yoric]

In this proposal, we'd still annotate channels with the types they accept, such that your API could still return 'discoverable' endpoints that describe what they do.

But discoverability is not about types (at least not how I understand the word "type"). It's about capabilities/features. For me, temperature is a type, but set oven temperature, set heater temperature and get sensor temperature are three very different features that involve that type.

I do not see this in the current version of your proposal.

Yes, as I mentioned under the "Type-Checking" section, if multiple adapters share the same types, we should share the same de/serialization code, which still fits into taxonomy.

Fair enough. I was just answering one of your comments, I don't remember which one. Anyway, I attempt to go further in that direction in my proposal.

I don't think that's true, but you're right that Thinkerbell would be affected. In practice, I think this just means that Thinkerbell would gain some enum-like type/value conversion code that currently lives in Taxonomy.

Actually, thinking a bit more about it, I believe that there is a way to solve this without that type/value conversion code. Still, you'll understand if I find it a bit weird to need to serialize and deserialize things when we pass them internally. Note that my counter-RFC has basically the same issue, by the way.

I think the opposite is true, or if anything, this is unrelated. I say the opposite might be true, because the types/values here map very well to HTTP content-type and body responses.

Indeed, they map nicely. On the other hand, with your proposal, I believe that any Adapter that wishes to publish a short-lived URL needs to come up with their own mechanism for doing so, dealing with timeouts, memory/disk pressure, etc. The current implementation, with BinaryParts, can delegate this nicely to the Foxbox.