search

whaleygeek / pyenergenie

A python interface to the Energenie line of products
MIT License
81 stars 51 forks source link

Scheduling of transmit messages might cause lost receive messages #9

Open whaleygeek opened 9 years ago

whaleygeek commented 9 years ago

In working on the latest monitor.py and adding a switching function, it occurred to me that the handling of transmits could be done more intelligently, to ensure that the radio is not in transmit mode when a receive message from an adaptor is expected.

The manufacturers have confirmed that the normal report rate of energy readings is every 10 seconds. So, if the RPi radio is in transmit mode sending a switch message when a receive message is expected, the receive message will be missed and a hole in the data created.

A smarter way to do this would be to keep the timestamp of the last received report in the dictionary for each device, and add transmit messages to a queue. The transmit queue can then be scheduled to send a transmit message in a time-slot when a receive message from a device is not expected, and this will reduce the probability of the radio being in transmit mode when a device is trying to send it's report.

Some sort of binary adaptive scheme for time-slots could be designed, so that if there is only one adaptor, the transmit is scheduled 5 seconds away from it's known report. If there are two adaptors with two different report times, the gap inbetween the two could be halved and the transmit window moved between the two.

This doesn't cater for the situation where two devices choose to send their reports at precisely the same time (presumably the devices delay a random time on startup to prevent mains fail resets causing all devices in a house to report at the same time on recovery of mains).

whaleygeek commented 9 years ago

The switch change messages don't go out on a very accurate schedule (uses async timer rather than sync timer), so this might indicate why some of the switch change response messages are missing from the log (colliding with other activity). Needs more investigation.

timestamp,mfrid,prodid,sensorid,flags,switch,voltage,freq,reactive,real 1444239056.63,4,2,1675,11111,1,235,49.94921875,-5,8 1444239066.59,4,2,1675,11111,1,235,49.8984375,-4,6 1444239076.59,4,2,1675,11111,1,235,49.8984375,-4,7 1444239096.6,4,2,1675,11111,1,235,49.94921875,-4,7 1444239106.56,4,2,1675,11111,1,235,49.8984375,-4,6 1444239110.34,4,2,1675,10000,0,None,None,None,None 1444239116.55,4,2,1675,11111,0,235,49.94921875,0,2 1444239126.55,4,2,1675,11111,0,234,49.8984375,2,0 1444239136.54,4,2,1675,11111,0,233,49.8984375,0,0 1444239146.52,4,2,1675,11111,0,233,49.8984375,2,0 1444239156.51,4,2,1675,11111,0,235,49.8984375,2,0 1444239166.56,4,2,1675,11111,0,238,49.8984375,2,0 1444239170.36,4,2,1675,10000,1,None,None,None,None 1444239176.58,4,2,1675,11111,1,237,49.8515625,0,4 1444239186.48,4,2,1675,11111,1,239,49.8984375,-3,7 1444239196.53,4,2,1675,11111,1,239,49.8515625,-4,8 1444239206.45,4,2,1675,11111,1,240,49.8984375,-4,7 1444239216.43,4,2,1675,11111,1,239,49.8984375,-4,8 1444239226.43,4,2,1675,11111,1,240,49.8984375,-4,7 1444239236.41,4,2,1675,11111,0,241,49.8984375,0,3 1444239246.39,4,2,1675,11111,0,239,49.8984375,2,0 1444239256.38,4,2,1675,11111,0,238,49.8984375,2,0 1444239266.38,4,2,1675,11111,0,238,49.94921875,2,0 1444239276.37,4,2,1675,11111,0,239,49.8984375,2,0 1444239286.35,4,2,1675,11111,0,239,49.94921875,2,0 1444239296.34,4,2,1675,11111,1,239,49.8984375,-2,4 1444239306.36,4,2,1675,11111,1,239,49.94921875,-4,8 1444239316.39,4,2,1675,11111,1,240,49.94921875,-4,8 1444239326.29,4,2,1675,11111,1,240,49.8984375,-3,7 1444239336.29,4,2,1675,11111,1,239,49.94921875,-4,8 1444239346.34,4,2,1675,11111,1,239,49.8984375,-4,7 1444239356.28,4,2,1675,11111,0,239,49.94921875,0,3 1444239366.22,4,2,1675,11111,0,240,49.8984375,2,0 1444239376.22,4,2,1675,11111,0,240,49.8984375,2,0 1444239386.22,4,2,1675,11111,0,240,49.8984375,0,0 1444239396.2,4,2,1675,11111,0,240,49.8515625,2,0 1444239406.18,4,2,1675,11111,0,240,49.8984375,2,0 1444239426.16,4,2,1675,11111,1,240,49.8984375,-3,7 1444239436.15,4,2,1675,11111,1,240,49.8515625,-5,8 1444239446.14,4,2,1675,11111,1,241,49.8984375,-4,8 1444239456.12,4,2,1675,11111,1,239,49.8515625,-4,8 1444239466.11,4,2,1675,11111,1,238,49.8515625,-4,8 1444239476.08,4,2,1675,11111,0,240,49.8515625,0,4 1444239486.08,4,2,1675,11111,0,240,49.8515625,2,0 1444239496.06,4,2,1675,11111,0,240,49.80078125,2,0 1444239506.05,4,2,1675,11111,0,240,49.8515625,2,0 1444239516.03,4,2,1675,11111,0,241,49.80078125,2,0 1444239526.02,4,2,1675,11111,0,241,49.80078125,0,0 1444239530.34,4,2,1675,10000,1,None,None,None,None 1444239536.0,4,2,1675,11111,1,241,49.8515625,0,4 1444239546.01,4,2,1675,11111,1,241,49.80078125,-3,7 1444239555.97,4,2,1675,11111,1,241,49.80078125,-4,7 1444239565.97,4,2,1675,11111,1,241,49.80078125,-3,7 1444239575.95,4,2,1675,11111,1,241,49.80078125,-4,8 1444239585.94,4,2,1675,11111,1,241,49.8515625,-4,8 1444239590.35,4,2,1675,10000,0,None,None,None,None 1444239595.91,4,2,1675,11111,0,241,49.8515625,0,3 1444239605.91,4,2,1675,11111,0,241,49.80078125,2,0 1444239615.91,4,2,1675,11111,0,240,49.80078125,2,0

whaleygeek commented 8 years ago

I plan to look at the intelligent message scheduler once the device_classes work is completed. The plan is to insert a message scheduler in both the transmit and receive pipelines, so that it learns the report timing patterns of reportable devices, and queues transmit messages into timeslots designed to avoid these reports.

whaleygeek commented 8 years ago

Noted in #63 that this still needs to be implemented.

HMM? 7. PERFORMING: To be able to build a well performing system with very few message collisions and message losses

POSSIBLE a. by dynamically learning report patterns of MiHome devices

POSSIBLE b. by intelligently deferring and schedulling transmit messages to avoid transmit slots of reporting devices

whaleygeek commented 8 years ago

Also noted in #63

(note, a message scheduler if inserted in the middle, would do callbacks to say that the request has been processed, so timestamps can be updated. Also same scheduler could handle retries perhaps, if the device is tx and rx, then when you send a switch change, it would normally report back that the switch had changed, so if you don't get it, or if it is in the wrong state, could retry a send again until it changes)

whaleygeek commented 8 years ago

A note made when merging ook_radio_config branch:

Note also that when radio transmit parameters are changed, this changes the amount of time that the radio is in transmit mode (and therefore alters the amount of time it is in receive mode, and ultimately how wide or narrow the receive window is). There will eventually need to be some way for the radio module to pass back up the chain how long a certain set of parameters will leave the receive window closed, so that a deterministic message scheduler can be written that schedules messages into known transmit and receive slots. No need to implement that yet, but we need to allow for it in the future.

whaleygeek commented 8 years ago

A note about the scheduler. If #51 is implemented, it would be possible to completely simulate all Energenie devices on a Raspberry Pi - this would make it much easier to test the message scheduler, because both ends would be under software control.

whaleygeek commented 5 years ago

design thinking:

I'm thinking of storing a 'ready rolled tx message' (a DeferredMessage) in a buffer in the C code, partial decoding the incoming payload in the C using a prefix trigger to detect a report from the eTRV, and if it is an eTRV report, turn the radio link around immediately and do a transmit, then mark the DeferredMessage as transmitted (with an option to just keep retrying it every time a new report comes in, in case the last one was missed due to a collision with another device). This will give real time performance for 1 queued tx message to the eTRV.