search

egeoffrey / egeoffrey-controller

The eGeoffrey controller manages the configuration of all the modules and coordinates sensors and run alerting rules
1 stars 0 forks source link

".0" added to integer value #24

Open eporocrail opened 3 years ago

eporocrail commented 3 years ago

When I send an integer value to an integer data formatted eGeoffrey sensor and have eGeoffrey send this value out via a different sensor ".0" is appended.

eporocrail commented 3 years ago

I collected as much info as I could: Incoming and outgoing eGeoffrey sensor:

format: int description: remote control service: configuration: topic: out/+/myplace/remote/key mode: push name: mqtt icon: outdent

format: string description: display service: configuration: topic: in/CD66AC/display mode: actuator name: mqtt icon: desktop

Rule:

severity: info triggers:

Result:

[2020-11-22 07:19:48][controller/hub] INFO: [2020-11-22 07:19:48] [remote] "remote control": 42 [2020-11-22 07:19:48][controller/alerter] DEBUG: [remote_display][default] running rule [2020-11-22 07:19:48][controller/alerter] DEBUG: Created session 2840261902 -> {'macro': u'default', 'variable_id': u'value', 'rule_id': u'remote_display'} [2020-11-22 07:19:48][controller/alerter] DEBUG: [remote_display][default][value] requesting db for GET remote: {'start': -1, 'end': -1} [2020-11-22 07:19:48][controller/alerter] DEBUG: [remote_display][default] received from db value: [42.0] [2020-11-22 07:19:48][controller/alerter] DEBUG: [remote_display][default] AND block evaluates to True [2020-11-22 07:19:48][controller/alerter] DEBUG: [remote_display][default] rule evaluates to True [2020-11-22 07:19:48][controller/alerter] INFO: [remote_display][info] send remote code to display [2020-11-22 07:19:48][service/mqtt] INFO: sending message 3342.0 to in/CD66AC/display [2020-11-22 07:19:49][controller/hub] INFO: [2020-11-22 07:19:48] [display] "display": 3342.0

Hope that helps.

eporocrail commented 3 years ago

I did some more tests.

Which ever format I select for the incoming sensor is irrelevant. A number enters eGeoffrey and the number with ".0" appended is forwarded by eGeoffrey. Yes indeed also for format "image", "task" and so on. Always the same result.

user2684 commented 3 years ago

Thanks, I will investigate better the issue, weird I never faced it despite having similar rules. Moved the issue to egeoffrey-controller since this is independent from the GUI looks like.

user2684 commented 3 years ago

I finally managed to replicate it and actually you're right, the int sensor is reporting a decimal number but at least here this is not preventing the rule to run correctly (see below). I should have noticed it otherwise since I have plenty of similar rules. Did you experience something different? 

egeoffrey-controller_1     | [2020-12-07 15:07:27][controller/alerter] DEBUG: [integer_test][_default_] received from db integer: [1.0]
egeoffrey-controller_1     | [2020-12-07 15:07:27][controller/alerter] DEBUG: [integer_test][_default_] evaluating condition integer ([1.0]) == value (1): True
egeoffrey-controller_1     | [2020-12-07 15:07:27][controller/alerter] DEBUG: [integer_test][_default_] AND block evaluates to True
egeoffrey-controller_1     | [2020-12-07 15:07:27][controller/alerter] DEBUG: [integer_test][_default_] rule evaluates to True
eporocrail commented 3 years ago

The incoming code from the IR receiver was not grabbed. I will test it again.

eporocrail commented 3 years ago

[2020-12-07 15:40:37][controller/alerter] DEBUG: [remote/52][default] running rule [2020-12-07 15:40:38][controller/alerter] DEBUG: Created session 3423521824 -> {'macro': u'default', 'variable_id': u'remote', 'rule_id': u'remote/52'} [2020-12-07 15:40:38][controller/hub] INFO: [2020-12-07 15:40:37] [remote] "remote control": [2020-12-07 15:40:38][controller/alerter] DEBUG: [remote/52][default][remote] requesting db for GET remote: {'start': -1, 'end': -1} [2020-12-07 15:40:38][controller/alerter] DEBUG: [remote/52][default] received from db remote: [52.0] [2020-12-07 15:40:38][controller/alerter] DEBUG: [remote/52][default] evaluating condition remote ([52.0]) = red (52): False [2020-12-07 15:40:38][controller/alerter] DEBUG: [remote/52][default] AND block evaluates to False [2020-12-07 15:40:38][controller/alerter] DEBUG: [remote/52][default] rule evaluates to False

user2684 commented 3 years ago

I tried multiple times with a number of different configurations but without being able to reproduce it :-( I need unfortunately to ask you an additional effort. First if you look into "eGeoffrey" / "Database" if the latest value of the sensor recorded there is the one you see here. Wonder if there is a decimal somewhere not showing up in this log where there shouldn't be leading to this behaviour. If everything is fine, wonder if you can fina a way to reproduce it by creating two sensors with the same format of those and setting the values manually like if coming from the associated service so I can try and reproduce it. Thanks again for your patience

eporocrail commented 3 years ago

I sent a code to the sensor and the result as is in the database: Geoffrey/sensors/remote 1783 140.91 Kb 37 days ago 15 seconds ago 52.0

eporocrail commented 3 years ago

The sensor receives the number without decimal. It is stored in the database with decimal. Then the value with decimal is processed by the rule and the result is transmitted with decimal. I will try to create something which can be reproduced without IR receiver.

eporocrail commented 3 years ago

It is very easy to reproduce:

Sensor:

description: demo decimal format: int

Rule:

severity: info triggers:

When the sensor is set to a value this value is stored into the database with decimal. The rule takes the value with decimal and transmits thr value created.

eporocrail commented 3 years ago

I fiddled around a little bit more.

To reproduce the effect the sensor which is set with the new value has to be an MQTT sensor. Then the value received by my module is a decimal.

Displaying the value on a not with MQTT associated sensor does NOT display the decimal. Also when the format of the receiving sensor is "string" the decimal is not displayed.

That is the reason why it looks to be not reproducible.

Within eGeoffrey the decimal is not shown, although stored into database with decimal.

user2684 commented 3 years ago

I'm starting to get confused...I tried multiple times without luck, let me go step by step. First I created a sensor with format set to integer and associated to the mqtt service. Then I simulated pushing in a decimal value with mosquitto_pub -t mqtt_in -m 1.2

But the values seems to be correctly saved as int, after being normalized:

egeoffrey-gateway_1        | 1609402876: New connection from 172.21.0.1 on port 1883.
egeoffrey-gateway_1        | 1609402876: New client connected from 172.21.0.1 as mosq-rD9MYV14spw1uJi7xu (p2, c1, k60).
egeoffrey-service-mqtt_1   | [2020-12-31 09:21:16][service/mqtt] DEBUG: received 1.2 for mqtt_int on topic mqtt_in
egeoffrey-gateway_1        | 1609402876: Client mosq-rD9MYV14spw1uJi7xu disconnected.
egeoffrey-controller_1     | [2020-12-31 09:21:16][service/mqtt] DEBUG: received 1.2 for mqtt_int on topic mqtt_in
egeoffrey-controller_1     | [2020-12-31 09:21:16][controller/hub] DEBUG: [mqtt_int] requesting database to store: {u'timestamp': 1609406476, u'value': 1}
egeoffrey-controller_1     | [2020-12-31 09:21:16][controller/hub] DEBUG: [mqtt_int] requesting database to store: {u'timestamp': 1609406476, u'value': 1}
egeoffrey-controller_1     | [2020-12-31 09:21:16][controller/hub] INFO: [2020-12-31 09:21:16] [mqtt_int] "": 1
egeoffrey-controller_1     | [2020-12-31 09:21:17][controller/hub] INFO: [2020-12-31 09:21:16] [mqtt_int] "": 1

Hence the rule triggers just fine:

egeoffrey-controller_1     | [2020-12-31 09:21:34][controller/alerter] DEBUG: [integer_test][_default_] running rule
egeoffrey-controller_1     | [2020-12-31 09:21:34][controller/alerter] DEBUG: Created session 802147880 -> {'macro': '_default_', 'variable_id': u'integer', 'rule_id': u'integer_test'}
egeoffrey-controller_1     | [2020-12-31 09:21:34][controller/alerter] DEBUG: [integer_test][_default_][integer] requesting db for GET mqtt_int: {'start': -1, 'end': -1}
egeoffrey-controller_1     | [2020-12-31 09:21:34][controller/alerter] DEBUG: [integer_test][_default_] running rule
egeoffrey-controller_1     | [2020-12-31 09:21:34][controller/alerter] DEBUG: Created session 802147880 -> {'macro': '_default_', 'variable_id': u'integer', 'rule_id': u'integer_test'}
egeoffrey-controller_1     | [2020-12-31 09:21:34][controller/alerter] DEBUG: [integer_test][_default_][integer] requesting db for GET mqtt_int: {'start': -1, 'end': -1}
egeoffrey-controller_1     | [2020-12-31 09:21:34][controller/alerter] DEBUG: [integer_test][_default_] received from db integer: [1.0]
egeoffrey-controller_1     | [2020-12-31 09:21:34][controller/alerter] DEBUG: [integer_test][_default_] evaluating condition integer ([1.0]) == value (1): True
egeoffrey-controller_1     | [2020-12-31 09:21:34][controller/alerter] DEBUG: [integer_test][_default_] AND block evaluates to True
egeoffrey-controller_1     | [2020-12-31 09:21:34][controller/alerter] DEBUG: [integer_test][_default_] rule evaluates to True
egeoffrey-controller_1     | [2020-12-31 09:21:34][controller/alerter] INFO: [integer_test][alert] OK
egeoffrey-controller_1     | [2020-12-31 09:21:34][controller/alerter] DEBUG: [integer_test][_default_] received from db integer: [1.0]
egeoffrey-controller_1     | [2020-12-31 09:21:34][controller/alerter] DEBUG: [integer_test][_default_] evaluating condition integer ([1.0]) == value (1): True
egeoffrey-controller_1     | [2020-12-31 09:21:34][controller/alerter] DEBUG: [integer_test][_default_] AND block evaluates to True
egeoffrey-controller_1     | [2020-12-31 09:21:34][controller/alerter] DEBUG: [integer_test][_default_] rule evaluates to True
egeoffrey-controller_1     | [2020-12-31 09:21:34][controller/alerter] INFO: [integer_test][alert] OK

I'm sure I'm missing something. Can you also please post the logs from controller/hub (no debug needed) like my first snippet above to see how the number is actually normalized? Thanks!

eporocrail commented 3 years ago

I took the data of the rule being evaluated with the controller message before and after it. There is not much of the controller.

[2020-12-31 13:52:54][controller/hub] DEBUG: [remote] requesting database to store: {'timestamp': 1609422774, u'value': '52'} [2020-12-31 13:52:54][controller/alerter] DEBUG: [remote/52][default] running rule [2020-12-31 13:52:54][controller/hub] INFO: [2020-12-31 13:52:54] [remote] "remote control": 52 [2020-12-31 13:52:54][controller/alerter] DEBUG: Created session 2780453905 -> {'macro': u'default', 'variable_id': u'remote', 'rule_id': u'remote/52'} [2020-12-31 13:52:54][controller/alerter] DEBUG: [remote/52][default][remote] requesting db for GET remote: {'start': -1, 'end': -1} [2020-12-31 13:52:54][controller/alerter] DEBUG: [remote/52][default] received from db remote: [52.0] [2020-12-31 13:52:54][controller/alerter] DEBUG: [remote/52][default] evaluating condition remote ([52.0]) = red (52): False [2020-12-31 13:52:54][controller/alerter] DEBUG: [remote/52][default] AND block evaluates to False [2020-12-31 13:52:54][controller/alerter] DEBUG: [remote/52][default] rule evaluates to False [2020-12-31 13:53:07][controller/hub] DEBUG: [out/myplace/garden/wthr/heartbeat] requesting database to store: {u'timestamp': 1609422787, u'value': 1, 'retain': {u'count': 1, u'raw': 0}}

Rule:

severity: warning triggers:

Sensor:

format: string description: remote control service: configuration: topic: out/+/myplace/remote/key mode: push name: mqtt icon: outdent

That is all I can say to it.

Maybe you are able to get the result by just setting the value of the sensor in eGeoffrey??????

user2684 commented 3 years ago

Sorry for the long time passed by...I tried multiple times and with many attempts, event built a real sensor to attach to mqtt without much luck. Now from your logs seems controller/hub has the right value and apparently the same right value is saved since the second log line from controller/hub is actually coming from controller/db. When data is requested back from controller/alerter to controller/db then seems the float casting taking place. Which is a non sense since normalization takes place way before, by controller/hub before requesting controller/db to save the value. Just if you have time, there are a couple of additional information you get get if you wish:

eporocrail commented 3 years ago

I am busy introducing TIME into the system to solve the "tipping bucket" issue. That leads to a partly new design of the system. The remote control data will be processed by the new module directly without eGeoffrey. Than this issue is not relevant for me anymore. (can be closed). If you would be interested in solving the issue none the less I am willing to provide you the requested data. Waiting for your answer.

user2684 commented 3 years ago

Thanks, yeah if you don't mind and without any hurry would be great to have this additional set of logs. I'll keep it open since I believe it is an issue and in the long term is something I'd love to fix. Thanks!

eporocrail commented 3 years ago

A storm tore apart my weather station. I am now focussing open a different project. If the topic gets relevant again I will op[en a new issue.

user2684 commented 3 years ago

Hi, I'll reopen this and keep it open if you don't mind since I was never able to reproduce the problem and I believe it is still there and could somehow come back again for other users. Thanks!