search

home-assistant / architecture

Repo to discuss Home Assistant architecture
313 stars 100 forks source link

IR blaster service/entity with a common data format #464

Closed marcan closed 1 year ago

marcan commented 3 years ago

Context

Currently, Home Assistant treats IR blasters as the "Remote" entity type. This entity allows sending specific remote commands, and optionally learning them. This works fine for devices with traditional remote controls where pressing a button sends a specific IR code, where users can program in their own controls.

However, some IR devices use the system not to send keypresses, but to send more complicated packets of data. A typical example is home aircon/climate units, where the system state is usually kept in the remote control. Pressing any button updates the state in the remote control, and then sends the entire configuration as one long IR packet to the aircon. Another example is certain Odelic ceiling lights, where the IR packet format directly sets warm/cool LED brightness.

These devices cannot be sanely controlled with the current system, because each full packet is treated as one button that needs to be learned or code that needs to be captured. The current status quo has led to projects like SmartIR, an integration which attempts to collect the cartesian product of all possible settings, all possible devices, and all possible IR blasters. This is, needless to say, completely insane, error-prone, and will never be complete (many aircon remotes have possible configuration combinations numbering in the millions). While it works, its approach is clearly suboptimal and misguided - the same thing can be accomplished much more completely for each device with a few minutes of IR packet reverse engineering, and few lines of code. But for that we need a shared IR blaster protocol, so that these kinds of integrations do not depend on specific IR blaster implementations and packet formats. SmartIR has some conversion code, but really dealing with device-specific IR code formats belongs in the low-level integrations driving those devices, it shouldn't be the job of upstream users/integrations.

Even for devices with simple button codes, having the ability to specify them in a device-independent way means people could build smarter integrations and share codes more easily.

Proposal

I would like to add low level support for IR blasters, starting with the Broadlink component. This would be a service that allows you to send IR packets in a device-independent format. This could be added to the Remote entity type, or be a new entity type; I would like to know what makes more sense.

Here is my idea for the data format. This is deliberately more general than the Broadlink format, so that it can represent a few higher level details better; implementations should do their best to support as much of it as possible.

{
    "type": "ir",
    "format": "pulse",
    "fc": 38000,
    "packets": [
        {
            "count": 1,
            "data": [6000,3000,800,1600,800,800]
        },
        {
            "count": 5,
            "data": [6000,3000,6000,800]
        },
    ]

The intent of the multipacket format is to slightly generalize a common pattern in key-based IR systems, where an initial packet contains the key code and subsequent regular repeats only have a "repeat code" that indicates the key was held down. Obviously any given IR data sequence in this format can be merged into a single long packet by just concatenating and duplicating arrays, so it should be easy to adapt to any dumber IR blaster; however, by having a bit of higher level structure like this, we can take advantage of blasters that can do that, e.g. Broadlink supports a repeat count so it could support a single repeated packet without duplicating everything. It's much harder to derive structure like this from pre-baked long packets, so it makes sense to specify it from the get go. Anyway, most devices will probably end up using a single packet with one or two repeats.

Implementation detail: the data is specified in microseconds, but many IR blasters will use a slower clock/time quantum. When converting durations, implementations should use a "running time" accumulator to quantize each pulse width in the context of the entire duration of the packet, not individually. Most IR protocols are self-synchronizing and so this doesn't matter. However, devices might exist which require strictly timed packets and run off of an asynchronous clock. For those, doing it this way adds jitter to individual edges but keeps the overall time accurate, which should work better.

Encodings

On top of the basic ability to send arbitrary IR packets, it would be useful to collect common IR packet formats into a library. The vast majority of consumer IR protocols can be described in three or four encoding styles, sometimes with different timings or variants. It would be useful to gather these into common code that downstream integrations can use. I think this belongs in a utility module that other code can just import. So, for example, you could easily build a packet for "RC5 code foo" or "Sony IR code bla" without having to rewrite all the encoding code. For those common protocols, this makes it much easier to just have lists of codes or work with bytes in packets.

E.g. the aircon in my last apartment used a variant of the NEC protocol with different timings, and a standard bytewise packet format where each byte is sent plain first, then inverted, all after a 3-byte header. Given a suitable library with nice defaults, this could all be done like this:

    data = [... data bytes representing aircon state ...]
    packet = nec_encode([0x01, 0x10, 0x00] + complement_encode(data), tp=421, th=3368)
    args = { "type": "ir", "packets": [{"packet": packet}] }
    # then call the service with that data

Which would apply complement encoding (suffixing each byte with its inverse), add a 3-byte header, then NEC encode with data pulse width 421µs and header pulse width 3368µs (from which default 1/0 encoding gap and header gap pulse widths would be derived using default NEC rules; those could be specified separately if needed).

But it might make sense to also support a subset of this as a packet format, so that simple e.g. template integrations can be built, just sending data like:

          service: remote.send_data
          data_template:
            entity_id: remote.rm3_bedroom
            type: ir
            format: rc5
            code: 0x1234

Which would imply a toggle bit at bit 11 and a start bit at bit 13 per RC5 convention. In this case, this functionality should be shared in the base object of the entity, so implementations only need to implement the raw packet format and get everything else for free from the core.

Learning?

This proposal doesn't cover learning/reverse engineering existing codes. It probably makes sense to have something for that too, so that devs can more easily use the learning features to investigate existing devices. The codes would be learned in this same format (or as much as can be inferred from what the device returns). Thoughts?

Consequences

I don't think there are significant downsides to this proposal; it's just adding a new interface to existing components. Integrations are free to support it or not, and the existing remote interface continues to work.

That said, once a raw IR interface is present, especially if learning/receive access is also implemented, the existing remote button interface could be rewritten on top of it to make it implementation independent. Then all a given implementation of the remote component for IR blasters needs to do is provide the raw pulse interface. At that point though, there would have to be migration code since the storage would probably switch to this format... This is optional though.

CC @felipediel for the broadlink component. I'd like to know if this makes sense; if it does I'd be happy to send a PR with the required changes to the Broadlink component/core, and then prototype some IR formatting tools and integrations (I have at least two packet/idempotent style and three keypress-style devices I'd like to control... the packet style ones definitely could use proper integrations)

elupus commented 3 years ago

Initial thought is that the standard format should be defined in a library, not home assistant. Then we can odopt this format in home assistant.

I do agree that we really should support stuff like sending a rc5,nec code without the raw data though. I had the intention of integrating use of my irgen (https://github.com/elupus/irgen) library in core for this purpose. But time ran away from me.

marcan commented 3 years ago

If you think all the gory details should be handled in a module that HA should just import, I'd be happy to give this a shot and publish it on pypi. Then all HA needs to do is define a service that accepts that format, and compatible integrations would implement it by just calling into the module to convert (I could put stuff like Broadlink conversion into the module).

felipediel commented 3 years ago

I like this idea, in this way we could offer access to LIRC, IRDB etc. The starting point would be to build a library to deal with the conversions and host it on PyPi. I think https://github.com/elupus/irgen is a good start.

Learning the codes without user intervention is the big challenge. You can have a look at https://github.com/bengtmartensson/IrScrutinizer and https://github.com/bengtmartensson/IrpTransmogrifier for reference.

marcan commented 3 years ago

I have a bit of a different idea for the architecture, so if you don't mind I'll use irgen as a protocol reference but otherwise roll my own approach and you can let me know what you think :-)

For learning, I think initially we should just support a raw mode, just a standardized version of the existing Broadlink codes for example, so that at least codes are portable. But it is definitely possible to try to interpret the codes as standard protocols and see what "sticks", though that requires a bit of finesse with the heuristic matching.

felipediel commented 3 years ago

Portability is a must. I don't think we should change the current syntax, just add new options. Users have been through many breaking changes lately and they won't like to know that they now need to learn about rc protocols and adapt their codes.

In the Broadlink integration we are using remote.send_command with the b64: prefix to indicate base64 codes (w/ Broadlink protocol). How about adding prefixes for these new protocols, like rc5: rc6: etc?

marcan commented 3 years ago

The thing is there's a lot of variation, so we need a rich set of configuration options. There is quite a range of use cases here. For an integration adding support for a specific device, we need to support a flexible data structure so the integration can do things like tweak pulse widths and specify the exact data bytes that are sent, programmatically, without having to generate strings. But for typical remote button codes, we'd want some kind of shorthand that is easy to use.

I think this warrants two APIs. One, the "full fat" interface, would be a new service that takes a data structure like I described. That structure would be defined in the python module, which knows how to make all the conversions, and would be extended as required as new protocols are added. This would be mainly for developers to interface with. It would use proper data structures (coming from JSON/YAML/python structures/whatever) so you don't have to deal with formatting text.

Then, a simple "parse text" interface would frontend that, to provide shorthand notation for a standard subset of codes. The module would have an entry point that takes such a text descriptor and knows how to instantiate a code object from it (from which you can convert to other formats etc). These codes would still allow overriding some parameters, but would not support all features of the full format, though it can support many.

So we could have short codes like this:

rc5:0,16 # TV volume up
rc5:f=56000:0,16 # same, but with a nonstandard carrier
rc5:f=56000,c=9:0,16 # held for 9 repeats (~1 second)
nec:00,ad # NEC command addr=00,data=AD (with standard inversions, sent as 00,ff,ad,52)
nec:c=9:00,ad # held for 9 repeats (sends the full code only once and then just repeat codes)
nec:a=2:12,34,ad # NEC command addr=1234, data=AD (16-bit plain addr, sent as 12,34,ad,52)
nec:a=2:80,c5,5e;80,c5,78 # back-to-back NEC commands. My ceiling lights use this (!). This sets warm=40%, cool=70%. They can't be independent transmissions, there is a timeout and they only work in close proximity!
nec:a=3,th=3368,tp=421:01,10,00,40,ff,cc,00,00,00,00,00,00,00,00,00,81,00,00 # aircon data packet with 24-bit plain address and the rest of data sent with inversions, and nonstandard timings; this is what my old apartment's aircon used
raw:6000,3000,500,500,1000,500 # raw data in microseconds

You normally wouldn't be storing complex codes like that last NEC one like this (you'd want an integration that understands the data format), but that's just to illustrate that it is possible to specify. I'd also provide a command line tool to format or send these, for experimentation.

To more formally specify the format:

<protocol>:[option=value[,option=value...]:]<data>[;<data>]

The data format would be protocol-specific. RC5 would use pairs of decimal numbers (address,command). NEC would use sequences of hex-encoded bytes (defaulting to adding inversion for every byte, but with an option to skip it for a number of header bytes as used by some devices). b64 would mean Broadlink for compatibility and of course would use base64 data. Raw mode would just be a list of microsecond timings.

There is one additional wrinkle: toggle bits. The IR generator library needs to be able to persist some state (at least in memory) between invocations, to be able to deal with toggle bits. Since this is protocol-dependent, I want to delegate all this logic to the module, so callers only have to provide storage. I propose we do this with a simple Python dictionary (or compatible interface) object. When instantiating a code using the library, you pass in an (initially empty) dict, and the library is in charge of storing toggle state (per protocol/code as necessary) in there, so all you have to do is make sure you keep passing the same dict. If you don't use this feature the library will otherwise work, but toggle bits won't be correct. The dict would be treated as a simple string key: numeric value dictionary, so if callers want to implement something fancier (like some shared data structure or disk-backed storage) they can do that by implementing a simple subset of the dict protocol.

How does that sound?

marcan commented 3 years ago

Just dumped a prototype here: https://github.com/marcan/circa

It doesn't have a proper setup.py or anything yet, and only supports RC5 and (a very flexible version of) NEC so far, but let me know what you think. The README has a few examples, it can receive/transmit codes from Broadlink devices, convert encodings, and guess format variants (timings etc)/simplify parameters automatically.

felipediel commented 3 years ago

How would remote.send_data look like in its most complete variation? Once this service is created, its structure cannot be changed without another ADR.

felipediel commented 3 years ago

I liked the automagically decode.

marcan commented 3 years ago

I would call it send_code in line with the terminology I chose for circa's objects. Circa already has a structure format, I just pushed a command line option to dump it:

$ python -m circa convert -s rc5:1,10
{
    "format": "rc5",
    "data": [
        {
            "addr": 1,
            "cmd": 10
        }
    ]
}

Or for a more complex example (some newlines removed):

$ python -m circa convert -s "nec:tp=455,ph=3722,a=-1,pi=120662,b=6:11,da,27,00,c5,00,10,e7;11,da,27,00,42,00,00,54;11,da,27,00,00,49,2c,00,a0,00,00,06,60,00,00,c1,00,00,4e"
{
    "format": "nec",
    "pulse_time": 455,
    "preamble_time_high": 3722,
    "address_bytes": -1,
    "packet_interval": 120662,
    "burst_count": 6,
    "data": [
        [17,218,39,0,197,0,16,231],
        [17,218,39,0,66,0,0,84],
        [17,218,39,0,0,73,44,0,160,0,0,6,96,0,0,193,0,0,78]
    ]
}

The idea, of course, is for both the text and struct encodings to remain backwards compatible as new features are added. All the options have defaults (except for format and data) and defaults are relative to other options, so for example, specifying just pulse_time for nec changes all the other timings by default too. This is how I avoid having a massive dump of parameters all the time, since usually relative timings are more consistent across protocol variants.

So you'd end up with something like this:

  service: remote.send_code
  data:
    entity_id: remote.rm3_bedroom
    type: ir
    code:
      format: rc5
      data:
        - addr: 1
          cmd: 10

This assumes that it is acceptable to delegate the structure under code to circa's parser. If the whole structure must be set in stone in HA, then it doesn't make much sense to do it this way, since then HA would still have to keep changing for every IR encoding or feature added. In that case, it would make more sense to just define a structure for the raw pulse representation of circa (which is unlikely to change), and have users use circa on the calling side to convert to that before invoking the service.

marcan commented 3 years ago

Any further comments? I will be adding and testing a few more protocols to circa soon and I'll be happy to prototype HA integration, but I'd like a bit of feedback on the proposal for the new service :)

felipediel commented 3 years ago

Hi @marcan. I like the idea and I think you should move on with circa. I'm just not sure how we are gonna bring this structure to Home Assistant.

I don't know if we can accept code: as a dictionary in the schema and delegate its structure to the implementations of the class. This needs to be discussed with the core team.

I think we will end up setting some parts in stone, eg format: and data:. These parameters are universal, aren't they? Let's take this code for example:

nec:tp=455,ph=3722,a=-1,pi=120662,b=6:11,da,27,00,c5,00,10,e7;11,da,27,00,42,00,00,54;11,da,27,00,00,49,2c,00,a0,00,00,06,60,00,00,c1,00,00,4e

We could have something like this:

  service: remote.send_code
  data:
    entity_id: remote.rm3_bedroom
    type: ir
    format: nec
    data: 11,da,27,00,c5,00,10,e7;11,da,27,00,42,00,00,54;11,da,27,00,00,49,2c,00,a0,00,00,06,60,00,00,c1,00,00,4e
    options:
      pulse_time: 455
      preamble_time_high: 3722
      address_bytes: -1
      packet_interval: 120662
      burst_count: 6

Or:

  service: remote.send_code
  data:
    entity_id: remote.rm3_bedroom
    type: ir
    format: nec
    data: 11,da,27,00,c5,00,10,e7;11,da,27,00,42,00,00,54;11,da,27,00,00,49,2c,00,a0,00,00,06,60,00,00,c1,00,00,4e
    options: tp=455,ph=3722,a=-1,pi=120662,b=6

Just ideas. We will probably end up with something better. We need to hear more opinions because once this is done, it cannot be changed easily.

dgomes commented 3 years ago

From a enduser point of view all that parameterisation should be hidden from the user...

the external library (circa) could carry a library of codes and users would just choose their equipment name/model from a list (because that is all they should care about)

felipediel commented 3 years ago

I think that an integration to build IR/RF devices via config flow would be the ideal scenario. Then we can send the codes with remote.send_command.

marcan commented 3 years ago

The point of this is to support devices beyond pure code libraries; code libraries can already be done (just more awkwardly and blaster-specific) without circa.

A climate control unit controlled by IR packets is not a code library, it's a full-blown integration that knows how to convert the required parameters (mode/temperature/fanspeed/more) into an IR packet, that then gets encoded with circa.

So the main point I'm trying to work out here is that in the HA side, we necessarily have two components for this use case:

And those components need to talk to each other. We need to agree on a method/data format between them. This interface is primarily for developers to write such integrations, not for end users.

This format needs to support raw IR codes, for weirdo devices that circa doesn't support yet, as a universal fallback. On top of that, supporting higher-level codes allows the device integration side to not have to itself import circa.

So here are some ways that might work:

  1. Use the existing remote.send_command interface, just extended to support circa text representations. This still delegates the code details to circa (so the exact text format would be subject to change/extension outside of the HA process) and it means both sides have to convert to/from text. I fear this might kind of encourage "ugly" ad-hoc text code formatting on the sender side, instead of using circa itself.
  2. Define a new method with a structured raw packet representation. This would be basically circa's raw format (or even a slightly more simplified/constrained version), and would be effectively set in stone. It means both sides need circa: the device integration for encoding codes to raw format, and the blaster integration for converting to device-specific raw formats like Broadlink. It also means that if IR blasters exist that can e.g. only transmit specific higher-level formats, those would be awkward, either not supported for anything, or having to use circa's magic decoding support to reverse engineer the high-level representation, which is not a great idea.
  3. Define a new method with a structured high-level packet representation. This could be a whole dict delegated to circa, or not; e.g. it would work to define format/data/options at a higher level. However, you still depend on circa for options, and note how the data format is currently not quite fixed. For example, Broadlink data formats are text. NEC codes are arrays of bytes. RC5 codes have a {addr, cmd} dict since they are higher level. Raw codes are arrays of mark/space widths. And currently circa always supports multiple data packets, so the whole thing is actually an array of codes, with optional repeat counts. We could use just the text format for data, but then that doesn't buy us much over just using text for the whole thing a la send_command.

So I guess my question here is, what are the main goals of the method format on the HA side? Do you want the structure to be fully fixed for compatibility reasons? Is there an advantage over using a text encoding vs a full dict? Ultimately, if the goal is to set the encoding in stone, that is at odds with the idea of circa evolving support for new formats/variants on its own; then you could use raw encoding instead to let that flexibility happen on the calling side, but that throws the possibility of higher-level IR blasters under the bus. Pros and cons...

Of course improving the support for "just button remote" devices is also important, but I think what will drive the architecture on the HA side is the more complex use case of packet-based devices. Simpler devices can always work on top of whatever we build for that, and we can discuss where things like code databases should be kept, how such generic integrations would work, etc.

dgomes commented 3 years ago

My point is HA should not be concerned with IR formats, all that should be delegated to the supporting library.

I don't seen the need to use text formats, we can pass a tuple to circa... I just don't think it is right for a user configuring HA to have to know all the tiny details about IR protocols.

The developer can know about everything, but that is inside circa.

So I'm probably with option 1.

marcan commented 3 years ago

But it isn't inside circa; the developer of an HA integration for a specific IR device needs to know the IR protocol details for the device they're supporting.

A typical flow for supporting a climate control device (aircon) would be like this:

So HA shouldn't care about IR formats, but IR formats have to travel through HA, and HA integrations need to care about IR formats, so we need to decide what that boundary looks like.

My original idea was:

frenck commented 1 year ago

This architecture issue is old, stale, and possibly obsolete. Things changed a lot over the years. Additionally, we have been moving to discussions for these architectural discussions.

For that reason, I'm going to close this issue.

../Frenck