Support for legacy OOK devices

[whaleygeek]

The green button devices use a different modulation scheme on the radio interface (OOK) compared to the MiHome range. I would like to add support for these, as for many people, these devices are cheaper and more accessible.

[whaleygeek]

I now have some green button devices on loan, and am planning that this will be the next addition to the software when I next find some time for this project.

[hotear]

Looking forward to testing this.

[whaleygeek]

I have just added some (very basic) support for legacy devices. This is early stage code yet, but I have a bench of 8 green-button devices all switching on and off.

sudo python legacy.py and follow the on screen instructions.

I still have to add addressing to the messages so that you can set a unique house code, and also so that you can select which of the 4 channels to control. Also the radio FIFO seems to overflow sometimes and you will get a 'Failed to send repeated payload' message after a while. But it is a start.

I'm not doing anything on the code interface yet (device objects), I'm just getting the low level radio interface working and tested. But this is a first step.

[whaleygeek]

I have mostly implemented the code for the green button devices now. This includes custom house codes, and constructing messages for any of the 4 switch numbers and the 'all devices' messages, plus a learn-mode application to allow you to program your switches and test them.

The underlying radio code is tested on the Raspberry Pi but the new message testing code is yet to be tested. I'll do this some time in the next few days.

[whaleygeek]

The message encoder is now working for the green button devices, and I can switch about 7 of them on and off reliably. However, there appears to be a discrepancy in the protocol documentation and the sample code I was provided. I am waiting for clarification before testing the switch addressing and house addressing payloads.

So at the moment, you can turn switches on and off, but there is only support for one address within the house (you can make any number of devices learn this address and then the all switch on and off together).

My test at the moment is:

press the button on the front of 7 switches, holding for 5 seconds until the LEDs are flashing (they are now in learn mode)

sudo python legacy.py - this will send a switch one ON and switch one OFF message repeatedly. On the first message received, the switches will learn this code and then respond to on and off messages repeatedly.

If any of my stargazers or forks wanted to try this out with your own green button devices and report back to me, that would be helpful. (Looks in the direction of @hotear for a response!)

[PatchworkBoy]

Works here with my 2x ENER002 green-button’d switches...

[whaleygeek]

I've been unable to get this to work with different device addresses, regardless of what device address I use with the new code, switches all respond as switch 1 only. I'm awaiting more detailed technical documentation before I can look more closely into the cause of this.

[whaleygeek]

To confirm whether this is something introduced by the Python or not, I thought I would try with the reference C 'switching' code provided by Energenie. I took the standard code and built it and run it, and proved that switch1 turns on and off when 'learnt' to this address. That worked fine.

I then modified the files as below, and hard coded 0x8E, 0xE8 and 0x8E, 0xEE into the on/off calls (which are device codes 0110 switch 2 on and 0111 switch 2 off). When I ran this code with the device in learn mode, it learnt the code and turned the LED off, but then did not respond.

I then put the codes back to 0xEE, 0xEE and 0xEE, 0xE8 (switch 1 on and off, device control codes 1110 and 1111) and the switch (which I thought had been learnt with switch 2 codes) now turned on and off.

This is exactly the same behaviour that I get with my Python code. So I suspect that the C code never worked either.

app_main.c

main(){ ... legacyTime = time(NULL);

    while (1){
            currentTime = time(NULL);
            if (difftime(currentTime, legacyTime) >= 1)                     // Number of seconds between Legacy message$
            {
                    legacyTime = time(NULL);
                    switch (state) {
                    case 0:
                            printf("1ON\n");
                            HRF_send_OOK_msg(0xEE, 0xEE); // socket 1 on
                    break;
                    case 1:
                            printf("1OFF\n");
                            HRF_send_OOK_msg(0xEE, 0xE8); // socket 1 off
                    break;
                    }
                    state = (state+1) % 2;
            }
    }

dev_HRF.c

void HRF_send_OOK_msg(uint8_t high, uint8_t low){ ...

            buf[15] = high;
            buf[16] = low;

[PatchworkBoy]

The original C works fine here programming and switching socket 1 & 2 with no modifications... Tried a full socket reset rather than learn mode? (Press n' hold for longer)

[whaleygeek]

@PatchworkBoy what devices do you have (ENER002?) And can you independently control them from the unmodified C code? If not, did you change the control bits or the house code, or both, to be able to independently control the two sockets?

Energenie have said that different devices require different house codes in order to control the sockets independently. I have a house_address parameter in my python payload encoder that I can use to confirm this.

[PatchworkBoy]

I have this set: https://energenie4u.co.uk/catalogue/product/ENER002-2PI-RT

Yes, can independently control both sockets after recompiling to individual binaries for each socket. No changes to the signals etc, just commented out the unnecessary socket commands & timer loops.

The unmodified code merely learns and loops thru socket on offs as per your test code. Timing hitting learn in the loop to get separate sockets to learn was a pain.

There is no 'house code' - afaik this is an FSK requirement, not legacy ook switching??? (Or isn't handled within the ener002 folder sample code). They just learn what they're told by the controller - whether they're sw1 thru 4. Put desired socket into learn mode, send s1 on. Unplug. Plug in socket to be used as s2, put into learn mode, send s2 on. Repeat as desired for socket 3 & 4.

This is with the hoperf software contained in the ENER002 folder of the ENER314-RT download.

M

On 29 Mar 2016, at 16:08, David Whale notifications@github.com wrote:

@PatchworkBoy what devices do you have (ENER002?) And can you independently control them from the unmodified C code? If not, did you change the control bits or the house code, or both, to be able to independently control the two sockets?

Energenie have said that different devices require different house codes in order to control the sockets independently. I have a house_address parameter in my python payload encoder that I can use to confirm this.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

[scousepi]

Hi,

I'm not sure this will help, but I have about 14 sockets working (the green button and the blue button ones) using modified C code and OOK switching and a 2-way Pi-Mote. I had to hard code the 20-bit address fields to set up groups of 4 sockets which can be switched individually. These don't support the all on and all off feature as they all have different 20-bit address fields.

I'm not sure this is explaining it too well, but I do know it works. If it's any use I've attached the modified C code. This works with the 2-way Pi-Mote as it allows the customisable address field - the older Pi-Mote doesn't. The code is based on the TxDemoOOK code supplied by Energenie.

mainc.zip

[gpbenton]

@scousepi How did you come up with the values to put in the 20-bit address fields? In the past I found a couple that worked (which was enough for me), but it was purely by trial and error.

[whaleygeek]

@scousepi Thanks, I've come to that conclusion myself, and I have a house_address in the Python payload builder that does just that automatically. Just got to find time to test hopefully later today when I am back in the lab with access to my devices.

@gpbenton The 20 bits of house address are encoded (just like all other bits in the message). The bits are pulse-stretched and surrounded in clock sync bits. So a byte stores just two bits. In hex, E is a '1' and 8 is a 'zero' which is clearer if you look at the binary: E=1110 (i.e a long pulse) and 8 = 1000 (i.e. a short pulse). Thus there is always a transition between 1 and zero in every bit, and the length of the pulse indicates the value of the bit. So for a 20 bit address you need 10 bytes (two encoded bits per byte) with only values 0x88 0x8E 0xE8 or 0xEE - otherwise you will be missing the clock sync bits and the receiving end will not detect the data bits correctly as it will loose it's time sync reference with the transmitting end quite quickly (due to lack of an accurate clock at both ends). This is normal for cost reduced embedded consumer products.

Alternatively, you can think of it as the clock and the data are sent merged together in a single on-air stream of bits. 1xx01xxx0 are the clock bits, and xDDxxDDx are the data bits. The receiving end can then reliably reconstruct this into a synchronised stream of clocks and data (consider the case where your address has a long run of zeros in it, a receiver without any form of clock synchronisation would drift very quickly and loose all reference of where the data bits start and end). This scheme is also really easy to decode using a simple 'bit high time' timer at the receiving end.

The sample C code is a bit terse and doesn't explain this well. In my Python code, I pass in a raw house address, and the python expands the raw data bits into the sync-bit enclosed payload that actually makes it out to the on-air interface via the radio FIFO, hiding this complexity from the programmer (which is how it should be, really).

Hope this helps.

[whaleygeek]

I mean 1xx01xx0 for the clock and xDDxxDDx for the data - sorry, typo.

[PatchworkBoy]

This may help - the green button sockets respond perfectly to the full COCO GDR2 / Chacon 54660 commandset when using other 433Mhz transceivers (how I currently control mine via RFXTRX433E)

They learn/respond to COCO’s HouseCodes A thru D, and Unitcodes 1 thru 4. Maximum of 16 unique addresses.

This is also the same protocol used by Maplin sockets - ala: http://www.maplin.co.uk/p/remote-controlled-mains-socket-set-single-n78ka - altho the Maplin sockets don’t have a learn mode, they have 4-step house code and unit code wheels on the back of the socket that you change with a screwdriver/little fingernail.

I believe both sockets (Energenie Green button & Maplin ones I mention) are also recognised by the RC-Switch library Type B example: https://github.com/sui77/rc-switch/blob/master/examples/TypeB_WithRotaryOrSlidingSwitches/TypeB_WithRotaryOrSlidingSwitches.pde

NB: This is a different protocol to that used by the Energenie Green Button 5-Gang or the earlier 4 gang.

[gpbenton]

@whaleygeek Thanks for the explanation. That was much clearer than the example code, which I can now understand. The unit codes always worked for me, using the example code, so I never looked in to what was happening.

[scousepi]

This is all really fascinating and I'm looking forward to testing the Python code. To answer the question from @gpbenton I reconstructed the formula from the C source (which generated the quasi-random 20-bit address) in a spreadsheet and by using different seed values calculated the different home addresses I subsequently hard coded.

[whaleygeek]

Ok @gpbenton @scousepi @PatchworkBoy @hotear I now have this working with 4 Energenie ENER002 green button plugs. I did a long-button reset on all plugs, then learnt the code (using the TxOOkDemo) and proved it works in the C. I have then managed to get the legacy.py to reliably turn all switches on and off together, and also to control each of the 4 sub addresses on that house code independently.

I have yet to re-test this using the learn mode inside legacy.py, but my initial testing shows that the control process is now reliable for these legacy devices. Let me know if you discover anything different from this.

[hornbyp]

I've been using the 'legacy' ENER002 sockets for a while with Picaxe microcontrollers and thought I'd pass on what I've learned, in case it's useful.

There are several versions of these sockets - but they look the same. The difference is that later models can learn TWO 20 bit controller codes, rather than just one. This means that you don't have to clone an existing Remote Control, if you want to use both it and your Pi (or whatever). You can tell which version you have, by continuing to hold down the green button after it has started flashing. On the earlier versions, nothing more will happen, but it will eventually 'flicker' on the later ones. (I think the flicker means you have just done a Factory Reset, and deleted both stored codes).

The original Pimote uses an HS1527 to encode the signal, so a look at the datasheet for that component might be informative. I found that the ENER002 is very forgiving with regards to 'pulse' width. You can teach a socket a code using pulse widths of about 200uS (ie same as Energenie's controller), but it will still respond if you lower the pulse width right down to 800uS (and vice versa).

If only needs to see the same command twice before it is recognised. Teaching a new code requires (about) ten repetitions.

The (original) Pimote documentation shows the 4 bit commands for controlling a group of FOUR sockets, but you can actually have a group of SEVEN. The missing codes are:- (D3->D0)

Socket '5' ON = 1010 Socket '5' OFF = 0010 Socket '6' ON = 1001 Socket '6' OFF = 0001 Socket '7' ON = 1000 Socket '7' OFF = 0000

(The latter doesn't work with the Pimote, because the HS1527 produces no o/p when the inputs are all '0').

[hotear]

Thanks @whaleygeek for your efforts on this. I have now incorporated it into my switching program. My code now sends a signal to plug 1 every 30 seconds to switch it on. I did a clean on the plug by holding down the green button for about 10 seconds (till it flashes super fast) then waited. The slow flashing (while it is in learn mode) stops as soon as the first on signal is sent so the plug obviously picked up the message, however it doesn't come on and 30 seconds later when the on message is sent again it still doesn't come on. If I press the green button again for 5 seconds and enter program mode again, the next on message causes to the flashing to stop (so it again must have picked up something) but the plug still doesn't come on. I'm doing some further testing.

[hotear]

OK, I've made some progress. I decided to see what would happen if I manually turned the plug on first and then waited for the next 'ON' message. The next 'ON' message turned the plug off. I thought that perhaps the on, off was inverted so I negated the state but the plug will still not come on. Oddly even in the negated code, if the plug is manually turned on then it still turns off on the next message.

[whaleygeek]

Hi Nick,

Yes, there are some 'artifacts' inserted by the scheduling in the python code which cause this effect. If you roll back to the githash link in the readme, the switches do work reliably. Although note that the on-air bitrate is incorrect, so what you are learning is not the same as what you might learn from a RF button handheld.

I'm currently pushing the core radio interface back into C to get the repeated packets butting together nicely, and then I am confident that they will be running at the right bitrate (i.e. compatible with codes learnt from a hand controller) and also switch on and off reliably. It's going to be a week or so until I will complete that work, so feel free to use the older release linked from the readme in the meantime to experiment, until I finish this bit of coding work.

[whaleygeek]

Hi @hotear I've done some more work on the low level driver today. This is not integrated into the python code yet, but you could usefully test it for me on your plugs at your end.

Get the latest code.

cd pyenergenie/src/energenie/drv

build the test code with

./build_rpi

Run the test code with

sudo ./hrfm69_test

It should turn switch 1 on and off repeatedly, once every second. I've tested it here with my plugs and it works fine.

I've more work to do on this, including extending the length of the transmit burst further and integrating it with the existing Python code. But it will be useful to see someone else trying it out too!

Thanks

[whaleygeek]

You might have to re-learn the address (I suggest a 10 second hold of the button to clear all pairing memory), as I corrected the on air bitrate in this version, older versions were transmitting at the wrong bitrate and your plugs might have learnt an invalid pattern that is incompatible with the correct bitrate messages.

[whaleygeek]

i.e. the full instructions are, from a raspberry pi terminal prompt, type the following:

git clone https://github.com/whaleygeek/pyenergenie cd pyenergenie/src/energenie/drv ./build_rpi sudo ./hrfm69_test

[whaleygeek]

@PatchworkBoy Just read your message on https://www.domoticz.com/forum/viewtopic.php?t=10142 note that I'm doing quite a lot of development on this code over the next few days. Today the new C driver is reliably switching green button OOK devices. Will be pushing this upwards through the stack into the Python code over the next few days, and then moving on to adding an object layer to the Python to make it more useful to applications.

Also note your earlier comment here about 'no house code' - the default C code uses a OTP code of 0x6C6C6 (encoded two bits per byte using pulse length encoding). The actual code programmed into your RF hand controllers is unique and serialised at the factory. So this is the closest you have to a 'house code'. Using code on the Pi you can change this code. hrfm69_test.c is only a test harness and has this code hard coded in it's test vectors. The Python I am "knitting upwards to" allows you to override this default OTP code for any code you like (including a code you might have allocated to a purchased hand controller). So, with 4 switches in a group, you could easily have multiple OTP addresses (10 bit address) and 4 switches allocated to each of those, so you could build a large system with this.

Please watch the issues in this repo for progress as it happens. If you have specific needs for your home automation, please document those in #31 and I will track them from there.

My focus at the moment is getting the physical layer working reliably and robustly for all devices. The MiHome adaptors (FSK) already work well with the Python.

[whaleygeek]

Also note, that in my physical layer interface, I use a home grown soft-SPI (which is basically bit-bashed GPIO), so the code has no dependence on the SPI peripheral. Energenie do do another product that allows GPIO control of the transmitted payloads, and you can get python code for that from here: https://pypi.python.org/pypi/energenie

I don't specifically support that mode of use in my code (yet) but might add support for it later.

MiHome House Monitor is good, coupled with some MiHome adaptor plus devices, you can get whole house energy usage, and then drill into specific device usage as overlays on that data.

I'm planning an architecture that will automatically switch between FSK (MiHome) and OOK (legacy) devices and treat them each as an application layer abstraction independently of what the physical layer is doing to achieve that messaging.

[whaleygeek]

src/energenie/drv/radio2_test.py now working with OOK transmit build process now builds radio_rpi.so into parent directory legacy.py now uses the new radio2.py interface (faster C radio driver)

Note, the fsk (MiHome) transmitter and receiver still uses the (slower) solution, but there are no payload repeats so that is not an issue. Later versions will use the new radio2 architecture in both ook and fsk.

[hotear]

Does this mean that the latest version should now correctly turn the green plug on? I can test it later today if so and let you know the results.

[whaleygeek]

Hi @hotear yes, it works really reliably here with my 7 Energenie green button devices. Please do test and report back if you have a chance, thanks!

I haven't fully tested all the device addressing yet as I am still waiting to borrow a hand-controller from Energenie to try this out - so the legacy.py test code just turns switch one of the random Energenie address on and off repeatedly. But it uses the new C based radio driver, which transmits more repeated packets now (420ms of burst per transmission) which significantly improves the reliability.

I tested by putting all my plugs into learn mode (a full clear with 10s of button push) then just sudo python legacy.py (the C code is pre-built in the downloaded package and ready to go).

Let me know if it works your end, and also what sort of range you get out of it.

If you are feeling brave, you could try some of the other test modes in legacy.py (there is code pre-written for testing all 4 device addresses, but I haven't re-tested all that yet as I have only just finished integrating the new physical layer driver code).

Thanks!

[whaleygeek]

You can also change the default address from 0x6C6C6 to anything you like by setting house_address parameter in the payload builder.

HOUSE_ADDRESS = None # default

change to eg

HOUSE_ADDRESS = 0x77665

You should also be able to independently switch more than 4 plugs now by using a mix of different house addresses and device addresses in the payload builders, e.g.

ONE_OFF = encoder.build_switch_msg(False, device_address=1, house_address=0x77755)

and also you can consider giving them friendly names...

KETTLE_OFF = encoder.build_switch_msg(False, device_address=1, house_address=0x77755)

[whaleygeek]

Don't get too carried away with the naming though, when my work on the physical layer is completed, I plan to work up the stack a bit more and create device classes so you will be able to just say kettle.off() for example and that could be bound to a MiHome plug or a green button plug, transparently.

[whaleygeek]

I've got a new feature under development on a feature branch here: https://github.com/whaleygeek/pyenergenie/tree/long-tx

This uses a longer transmit burst length, which you can configure with the TIMES constant in legacy.py - previous drivers were limited to 15 payload repeats, but you can set this quite long now to hopefully improve reliability.

I've also turned off the debug trace in the driver by default so you don't get lots of messages scrolling up the screen (just the switch ON/OFF messages).

legacy.py is also pre configured to run the learning mode then the switch cycler code - so I have managed to prove that with this new driver, it is possible to independently control each of the 4 switch sub codes.

I haven't tested different house codes yet, so I won't be merging this back into master until that is tested.

[hotear]

OK, I've had chance to incorporate the latest version into my application but I had some problems as I am also using the purple plugs still and the receive function is commented out in radio2. So I tried to use radio for the purple plugs and radio2 for the green plugs by importing them both but when it was doing the transmit for the green plug it seems to get stuck at "waiting for sent...". I've reverted back to the previous version for now.

[whaleygeek]

@hotear ok, yes, that won't work, because it will try to open two separate connections to the GPIO pins on the radio, and get very confused. You're best to wait until I have finished moving and testing the rest of the FSK transmit and receive functionality. Keep an eye on #34 because I'll close that one when all radio functionality is together in a single build.

[whaleygeek]

I've now tested the OOK transmitter against an Energenie hand controller, and verified that the Python code and codes generated from the hand controller are completely compatible (by capturing an on-air burst from the hand controller with an SDR setup).

This issue will be closed down as completed when I merge back to master. But anyone desperate to try this out can use the latest commit on the long-tx branch if they want to try it.

[mjworsley]

Just as a heads up, the work on long-tx works fine with my ENER010 4-gang. Thanks for this, @whaleygeek -- look forward to being able to use this in anger :)

[DavidWhaleIET]

Thanks @mjworsley this is really useful to know, as I don't have a 4gang myself yet. I'm presuming that the 4-gang is basically a single device that sub-decodes device addresses 1,2,3,4 and maps them one to each of the gangs of the socket?

Note that you can now change the HOUSE_CODE address (in the legacy.py test script) and allocate any code to your devices. I managed to capture an RF burst of my Energenie hand controller and manually decode it, program it into HOUSE_CODE, and then both my hand controller and python code could control the same sockets.

I'm just about to open an issue to remember that eventually I want to write an OOK receiver so that users without an SDR radio can use the Energenie to learn the house codes of their hand controllers without any extra kit.

[DavidWhaleIET]

Note, that due to the HOUSE_CODE override in legacy.py, it is now possible to install and control (2^24) independent devices in your house - that should be more than enough for full home automation!! i.e. the house address is 20 bits, and there are 4 device codes, so you have plenty of combinations of addresses that you can 'learn' into your devices now!

[mjworsley]

Yes, exactly -- single green-button on the 4-gang, so I popped that into learn mode, then I used legacy.py to teach the first socket on the gang. Once that was done, the other sockets on the 4-gang "just worked(TM)".

[whaleygeek]

long-tx branch merged, this now works on legacy OOK devices. Closing.