search

daq-tools / kotori

A flexible data historian based on InfluxDB, Grafana, MQTT, and more. Free, open, simple.
https://getkotori.org/
GNU Affero General Public License v3.0
110 stars 17 forks source link

Ingesting data from MPP Solar inverters #50

Open amotl opened 3 years ago

amotl commented 3 years ago

Hi there,

@danb35 is currently into:

Now to figure out how to tell mpp-solar how to send data in a format Kotori will read.

With kind regards, Andreas.

Originally posted by @danb35 in https://github.com/daq-tools/kotori/issues/49#issuecomment-810666082

amotl commented 3 years ago

Dear @danb35,

thanks for sharing your use case. At [1], I see that the mpp-solar program already supports a whole bunch of different output processors.

I haven't looked into the respective details yet, but the optimal way to forward the metrics/measurements to MQTT to make Kotori pick them up would be to send them in a JSON container (flat dictionary of key/value pairs) to a configurable MQTT topic. In that manner, all values will be stored within the same record and have the same timestamp. Maybe this is already possible or otherwise @jblance will be happy about another contribution to add this feature in form of another output processor?

On the other hand, Kotori will also accept readings in a "discrete" format [2], like the one the _tag_mqtt output processor_ seems to implement:

tag_mqtt outputs the data to the supplied MQTT broker using the supplied tag as the topic, e.g. {tag}/max_charger_range 120.0.

In order to do that, one would probably use the --output and --tag options like

# Define Kotori channel.
export MQTT_TOPIC=mqttkit-1/testdrive/area-42/node-1

# Invoke "mpp-solar".
mpp-solar --port /dev/hidraw0 --protocol PI16 --command QPIRI --name FirstInverter --output tag_mqtt --tag ${MQTT_TOPIC}/data

Let me know how that goes.

With kind regards, Andreas.

[1] https://github.com/jblance/mpp-solar/blob/master/docs/usage.md#list-available-output-processors [2] https://getkotori.org/docs/handbook/acquisition/protocol/mqtt.html#single-readings

danb35 commented 3 years ago

With a few alterations to the command, I ran:

(mppsolar) root@solar:~# export MQTT_TOPIC=solar/testdrive/node1
(mppsolar) root@solar:~# mpp-solar -p /dev/hidraw1 -P PI18 -c GS --name TestInverter -o tag_mqtt --tag ${MQTT_TOPIC}/data -q kotori

...and it seems to have worked:

2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/grid_voltage' with payload '120.3'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'grid_voltage', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'grid_voltage', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/grid_frequency' with payload '59.9'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'grid_frequency', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'grid_frequency', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/ac_output_voltage' with payload '120.3'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'ac_output_voltage', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'ac_output_voltage', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/ac_output_frequency' with payload '59.9'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/ac_output_apparent_power' with payload '1336'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'ac_output_apparent_power', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'ac_output_frequency', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'ac_output_apparent_power', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'ac_output_frequency', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/ac_output_active_power' with payload '1264'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'ac_output_active_power', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'ac_output_active_power', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/output_load_percent' with payload '52'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'output_load_percent', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'output_load_percent', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/battery_voltage' with payload '27.0'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'battery_voltage', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'battery_voltage', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/battery_voltage_from_scc' with payload '27.0'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'battery_voltage_from_scc', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/battery_voltage_from_scc2' with payload '0.0'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'battery_voltage_from_scc2', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'battery_voltage_from_scc', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'battery_voltage_from_scc2', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/battery_discharge_current' with payload '0'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'battery_discharge_current', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'battery_discharge_current', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/inverter_heat_sink_temperature' with payload '29'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/battery_charging_current' with payload '1'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/battery_capacity' with payload '100'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'battery_charging_current', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'battery_charging_current', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'battery_capacity', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'inverter_heat_sink_temperature', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/mppt1_charger_temperature' with payload '33'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'battery_capacity', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'mppt1_charger_temperature', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'mppt1_charger_temperature', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'inverter_heat_sink_temperature', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/mppt2_charger_temperature' with payload '0'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'mppt2_charger_temperature', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'mppt2_charger_temperature', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/pv1_input_power' with payload '10'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'pv1_input_power', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/pv1_input_voltage' with payload '38.8'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'pv1_input_voltage', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'pv1_input_power', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/pv2_input_voltage' with payload '0.0'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'pv2_input_voltage', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'pv2_input_voltage', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'pv1_input_voltage', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/pv2_input_power' with payload '0'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/setting_value_configuration_state' with payload 'Something changed'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'setting_value_configuration_state', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'pv2_input_power', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'pv2_input_power', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'setting_value_configuration_state', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/mppt1_charger_status' with payload 'charging'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'mppt1_charger_status', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'mppt1_charger_status', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/load_connection' with payload 'connect'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/mppt2_charger_status' with payload 'abnormal'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/battery_power_direction' with payload 'charge'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'battery_power_direction', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'mppt2_charger_status', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/dc/ac_power_direction' with payload 'AC-DC'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'dc/ac_power_direction', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'battery_power_direction', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Unknown message type on topic "solar/testdrive/node1/data/dc/ac_power_direction" with payload "AC-DC", ignoring.
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'load_connection', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/line_power_direction' with payload 'input'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/node1/data/local_parallel_id' with payload '0'
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'local_parallel_id', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'local_parallel_id', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'load_connection', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'mppt2_charger_status', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'data', 'slot': 'line_power_direction', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'node1'}
2021-04-02T07:22:29-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'data', 'slot': 'line_power_direction', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'node1_data_events', 'label': 'node1_data', 'measurement': 'node1_data_unknown', 'gateway': 'node1'}

It looks like the data's accepted and saved, but was expecting to see an "instant dashboard" created, and that doesn't seem to be the case. No doubt it's because I'm a complete n00b when it comes to Influx, Grafana, and the like, but I don't see where I could actually access my data.

danb35 commented 3 years ago

The other possibility is that mpp-solar does support a json output format, but I don't think it currently sends that via MQTT:

(mppsolar) root@solar:~# mpp-solar -p /dev/hidraw1 -P PI18 -c GS --name TestInverter -o json
{"_command": "GS", "_command_description": "General status query", "grid_voltage": 120.1, "grid_frequency": 59.9, "ac_output_voltage": 120.1, "ac_output_frequency": 59.9, "ac_output_apparent_power": 1333, "ac_output_active_power": 1252, "output_load_percent": 52, "battery_voltage": 27.0, "battery_voltage_from_scc": 27.0, "battery_voltage_from_scc2": 0.0, "battery_discharge_current": 0, "battery_charging_current": 1, "battery_capacity": 100, "inverter_heat_sink_temperature": 29, "mppt1_charger_temperature": 33, "mppt2_charger_temperature": 0, "pv1_input_power": 21, "pv2_input_power": 0, "pv1_input_voltage": 50.0, "pv2_input_voltage": 0.0, "setting_value_configuration_state": "Something changed", "mppt1_charger_status": "charging", "mppt2_charger_status": "abnormal", "load_connection": "connect", "battery_power_direction": "charge", "dc/ac_power_direction": "AC-DC", "line_power_direction": "input", "local_parallel_id": 0}
amotl commented 3 years ago

Dear @danb35,

thanks for your response.

It looks like the data's accepted and saved, but was expecting to see an "instant dashboard" created, and that doesn't seem to be the case.

Sorry about that, we will have to investigate the root cause.

Just a remark

First, referring to:

(mppsolar) root@solar:~# export MQTT_TOPIC=solar/testdrive/node1

Can you add yet another segment to the MQTT topic? The "quadruple" convention needs {realm}/{network}/{gateway}/{node}, so e.g. use export MQTT_TOPIC=solar/testdrive/area42/node1 in your case.

Debugging Kotori <-> Grafana

Authentication

The most common issue with Grafana is the authentication part. If you change your Grafana credentials, you will have to update /etc/kotori/kotori.ini accordingly.

Grafana version

Also, it might be about the Grafana version. Kotori only recently gained support for Grafana 7.x and we have not been able to release any updated packages yet. Attaching to this, may I ask which versions of Grafana and Kotori you are currently running?

Investigate the log files

When it comes to debugging those kinds of issues, I wanted to add that hints within the kotori.log might point to the root cause of the problem.

In order to be able to investigate the outcome of Kotori talking to Grafana, please watch the kotori.log file exactly when new data points are being ingested after Kotori was restarted.

Background

Please be aware that Kotori might "remember/cache" the state that it has talked to Grafana already. This mechanism is intentional because we don't want to hammer Grafana each time a data point is ingested. The "state" is based on the data point's signature, so this mechanism balances between efficiency and convenience, as new fields arriving on the wire will still be propagated by updating the Grafana panel.

Conclusion

So, if there are some flaws within that "Grafana refresh" logic, Kotori might stop talking to Grafana altogether. Please restart Kotori in order to clear that state, so that subsequent data ingest events will definitively run the whole code path definitively including going to Grafana. Then, maybe the output of kotori.log will give some hints.

I hope this helps. Please let me know if you can still not make it work, then we will have to dig deeper.

With kind regards, Andreas.

amotl commented 3 years ago

The other possibility is that mpp-solar does support a JSON output format, but I don't think it currently sends that via MQTT.

Indeed, being able to divert that JSON to an arbitrary MQTT topic would be perfect for the use case of submitting measurement data "en bloc".

export MQTT_TOPIC=solar/testdrive/area42/node1
mpp-solar -p /dev/hidraw1 --protocol PI18 --command GS --name TestInverter --output json_mqtt --tag ${MQTT_TOPIC}/data.json -q kotori

Note the --output json_mqtt option and the /data.json MQTT topic suffix.

danb35 commented 3 years ago

It's probably worth noting that my Kotori instance is already logging data from my WeeWx system (which is also submitted via MQTT) without a problem. At least in general terms, the appropriate pieces seem to be working properly together. Some other responses are going to be a little out of order, since the log output is huge.

json_mqtt with mpp-solar

I've opened an issue about that here: https://github.com/jblance/mpp-solar/issues/75

Authentication

I had changed the Grafana credentials when I initially installed Kotori (Grafana seemed to require that), and that had caused problems (about which I raised an issue at the time), but that's now resolved.

Grafana version

Grafana is at 6.6.1--it's notifying me that there's an update available, but I haven't updated it. Kotori version is 0.24.5-1~bionic.

Restart Kotori and retry

Tried that--see log below.

The "quadruple" convention needs {realm}/{network}/{gateway}/{node},

Like I said, I'm a n00b with a lot of this stuff. Easy enough to fix on this one, of course. And that does seem to have made a difference in the result:

(mppsolar) root@solar:~# export MQTT_TOPIC=solar/testdrive/area42/node1
(mppsolar) root@solar:~# mpp-solar -p /dev/hidraw1 -P PI18 -c GS --name TestInverter -o tag_mqtt --tag ${MQTT_TOPIC}/data -q kotori

In the log (incidentally, how do I turn off debug logging? It's helpful for now, and I obviously enabled it at some point in the past, but I can't find where I did that, and it would be good to turn it off so the log files don't get too huge again):

2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/grid_voltage' with payload '119.6'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/grid_voltage', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/grid_voltage', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/grid_frequency' with payload '60.0'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/grid_frequency', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/grid_frequency', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/ac_output_voltage' with payload '119.6'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/ac_output_frequency' with payload '60.0'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/ac_output_frequency', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/ac_output_voltage', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/ac_output_voltage', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/ac_output_frequency', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/ac_output_apparent_power' with payload '1339'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/ac_output_apparent_power', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/ac_output_apparent_power', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.storage.influx          ] INFO    : Creating database "solar_testdrive"
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/ac_output_active_power' with payload '1262'
2021-04-02T08:44:05-0400 [kotori.daq.storage.influx          ] INFO    : Creating database "solar_testdrive"
2021-04-02T08:44:05-0400 [kotori.daq.storage.influx          ] INFO    : Creating database "solar_testdrive"
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/ac_output_active_power', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/ac_output_active_power', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.storage.influx          ] INFO    : Creating database "solar_testdrive"
2021-04-02T08:44:05-0400 [kotori.daq.storage.influx          ] INFO    : Creating database "solar_testdrive"
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/output_load_percent' with payload '52'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/output_load_percent', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/battery_voltage' with payload '27.9'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/output_load_percent', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/battery_voltage_from_scc' with payload '27.9'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/battery_voltage_from_scc', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/battery_voltage_from_scc2' with payload '0.0'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/battery_discharge_current' with payload '0'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/battery_charging_current' with payload '3'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/battery_discharge_current', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/battery_discharge_current', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/battery_voltage_from_scc', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/battery_charging_current', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/battery_charging_current', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/battery_capacity' with payload '100'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/battery_voltage_from_scc2', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/battery_voltage', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/battery_voltage', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/battery_voltage_from_scc2', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/battery_capacity', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/battery_capacity', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/inverter_heat_sink_temperature' with payload '29'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/inverter_heat_sink_temperature', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/inverter_heat_sink_temperature', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/mppt1_charger_temperature' with payload '34'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/mppt2_charger_temperature' with payload '0'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/pv1_input_power' with payload '33'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/pv2_input_power' with payload '0'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/pv2_input_power', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/pv1_input_voltage' with payload '64.8'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/mppt1_charger_temperature', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/pv1_input_power', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/mppt2_charger_temperature', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/mppt1_charger_temperature', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/pv2_input_power', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/mppt2_charger_temperature', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/pv1_input_voltage', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/pv1_input_power', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/pv1_input_voltage', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/pv2_input_voltage' with payload '0.0'
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/pv2_input_voltage', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:05-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/pv2_input_voltage', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:06-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'battery_voltage': 27.9}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:06-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'ac_output_frequency': 60.0}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:06-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'inverter_heat_sink_temperature': 29.0}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:06-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : response: {u'message': u'Datasource added', u'datasource': {u'readOnly': False, u'name': u'solar_testdrive', u'database': u'solar_testdrive', u'url': u'http://localhost:8086/', u'basicAuth': False, u'jsonData': {}, u'access': u'proxy', u'isDefault': False, u'typeLogoUrl': u'', u'orgId': 1, u'user': u'root', u'version': 1, u'basicAuthUser': u'', u'secureJsonFields': {}, u'password': u'root', u'type': u'influxdb', u'id': 3, u'basicAuthPassword': u'', u'withCredentials': False}, u'name': u'solar_testdrive', u'id': 3}
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02T08:44:06-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'grid_voltage': 119.6}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:06-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02T08:44:06-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'ac_output_voltage': 119.6}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:06-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'pv2_input_voltage': 0.0}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:06-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Client Error 404: {"message":"Dashboard not found"}
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'battery_capacity': 100.0}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:06-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Client Error 404: {"message":"Dashboard not found"}
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02T08:44:06-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'mppt1_charger_temperature': 34.0}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:06-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Client Error 404: {"message":"Dashboard not found"}
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'ac_output_active_power': 1262.0}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:06-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Client Error 404: {"message":"Dashboard not found"}
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02T08:44:06-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'pv1_input_power': 33.0}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:06-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:06-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02 08:44:07,005 [urllib3.connectionpool   ] WARNING: Connection pool is full, discarding connection: localhost
2021-04-02T08:44:07-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'ac_output_apparent_power': 1339.0}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Grafana response: {u'status': u'success', u'uid': u'_cEkmc_Gk', u'url': u'/d/_cEkmc_Gk/solar-testdrive', u'slug': u'solar-testdrive', u'version': 1, u'id': 4}
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking dashboard "solar-testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] WARN    : Precondition failed: name-exists (`A dashboard with the same name in the folder already exists`)
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking dashboard "solar-testdrive"
2021-04-02 08:44:07,127 [urllib3.connectionpool   ] WARNING: Connection pool is full, discarding connection: localhost
2021-04-02T08:44:07-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'pv1_input_voltage': 64.8}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] WARN    : Precondition failed: name-exists (`A dashboard with the same name in the folder already exists`)
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking dashboard "solar-testdrive"
2021-04-02 08:44:07,207 [urllib3.connectionpool   ] WARNING: Connection pool is full, discarding connection: localhost
2021-04-02T08:44:07-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'pv2_input_power': 0.0}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] WARN    : Precondition failed: name-exists (`A dashboard with the same name in the folder already exists`)
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking dashboard "solar-testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/setting_value_configuration_state' with payload 'Something changed'
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/setting_value_configuration_state', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] ERROR   : Processing MQTT message failed from topic "solar/testdrive/area42/node1/data/setting_value_configuration_state":
    [Failure instance: Traceback: <type 'exceptions.ValueError'>: could not convert string to float: Something changed
    /usr/lib/python2.7/threading.py:801:__bootstrap_inner
    /usr/lib/python2.7/threading.py:754:run
    /opt/kotori/lib/python2.7/site-packages/twisted/_threads/_threadworker.py:46:work
    /opt/kotori/lib/python2.7/site-packages/twisted/_threads/_team.py:190:doWork
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:197:process_message
    ]
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/setting_value_configuration_state
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/mppt1_charger_status' with payload 'charging'
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/mppt1_charger_status', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] ERROR   : Processing MQTT message failed from topic "solar/testdrive/area42/node1/data/mppt1_charger_status":
    [Failure instance: Traceback: <type 'exceptions.ValueError'>: could not convert string to float: charging
    /usr/lib/python2.7/threading.py:801:__bootstrap_inner
    /usr/lib/python2.7/threading.py:754:run
    /opt/kotori/lib/python2.7/site-packages/twisted/_threads/_threadworker.py:46:work
    /opt/kotori/lib/python2.7/site-packages/twisted/_threads/_team.py:190:doWork
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:197:process_message
    ]
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/mppt1_charger_status
2021-04-02 08:44:07,293 [urllib3.connectionpool   ] WARNING: Connection pool is full, discarding connection: localhost
2021-04-02T08:44:07-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'mppt2_charger_temperature': 0.0}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/mppt2_charger_status' with payload 'abnormal'
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/mppt2_charger_status', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] ERROR   : Processing MQTT message failed from topic "solar/testdrive/area42/node1/data/mppt2_charger_status":
    [Failure instance: Traceback: <type 'exceptions.ValueError'>: could not convert string to float: abnormal
    /usr/lib/python2.7/threading.py:801:__bootstrap_inner
    /usr/lib/python2.7/threading.py:754:run
    /opt/kotori/lib/python2.7/site-packages/twisted/_threads/_threadworker.py:46:work
    /opt/kotori/lib/python2.7/site-packages/twisted/_threads/_team.py:190:doWork
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:197:process_message
    ]
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/mppt2_charger_status
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/load_connection' with payload 'connect'
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/load_connection', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] ERROR   : Processing MQTT message failed from topic "solar/testdrive/area42/node1/data/load_connection":
    [Failure instance: Traceback: <type 'exceptions.ValueError'>: could not convert string to float: connect
    /usr/lib/python2.7/threading.py:801:__bootstrap_inner
    /usr/lib/python2.7/threading.py:754:run
    /opt/kotori/lib/python2.7/site-packages/twisted/_threads/_threadworker.py:46:work
    /opt/kotori/lib/python2.7/site-packages/twisted/_threads/_team.py:190:doWork
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:197:process_message
    ]
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/load_connection
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/battery_power_direction' with payload 'charge'
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/battery_power_direction', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] ERROR   : Processing MQTT message failed from topic "solar/testdrive/area42/node1/data/battery_power_direction":
    [Failure instance: Traceback: <type 'exceptions.ValueError'>: could not convert string to float: charge
    /usr/lib/python2.7/threading.py:801:__bootstrap_inner
    /usr/lib/python2.7/threading.py:754:run
    /opt/kotori/lib/python2.7/site-packages/twisted/_threads/_threadworker.py:46:work
    /opt/kotori/lib/python2.7/site-packages/twisted/_threads/_team.py:190:doWork
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:197:process_message
    ]
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/dc/ac_power_direction' with payload 'AC-DC'
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/dc/ac_power_direction', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/battery_power_direction
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Unknown message type on topic "solar/testdrive/area42/node1/data/dc/ac_power_direction" with payload "AC-DC", ignoring.
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/line_power_direction' with payload 'input'
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/line_power_direction', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Processing message on topic 'solar/testdrive/area42/node1/data/local_parallel_id' with payload '0'
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] ERROR   : Processing MQTT message failed from topic "solar/testdrive/area42/node1/data/line_power_direction":
    [Failure instance: Traceback: <type 'exceptions.ValueError'>: could not convert string to float: input
    /usr/lib/python2.7/threading.py:801:__bootstrap_inner
    /usr/lib/python2.7/threading.py:754:run
    /opt/kotori/lib/python2.7/site-packages/twisted/_threads/_threadworker.py:46:work
    /opt/kotori/lib/python2.7/site-packages/twisted/_threads/_team.py:190:doWork
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:197:process_message
    ]
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/line_power_direction
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Topology address: {'node': 'node1', 'slot': 'data/local_parallel_id', 'realm': 'solar', 'network': 'testdrive', 'gateway': 'area42'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Storage location: {'node': 'node1', 'slot': 'data/local_parallel_id', 'realm': 'solar', 'network': 'testdrive', 'database': 'solar_testdrive', 'measurement_events': 'area42_node1_events', 'label': 'area42_node1', 'measurement': 'area42_node1_sensors', 'gateway': 'area42'}
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02T08:44:07-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'grid_frequency': 60.0}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.dashboard] INFO    : No missing panels to add
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.dashboard] INFO    : No missing panels to add
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02 08:44:07,468 [urllib3.connectionpool   ] WARNING: Connection pool is full, discarding connection: localhost
2021-04-02T08:44:07-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'local_parallel_id': 0.0}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.dashboard] INFO    : No missing panels to add
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02 08:44:07,551 [urllib3.connectionpool   ] WARNING: Connection pool is full, discarding connection: localhost
2021-04-02T08:44:07-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'battery_voltage_from_scc2': 0.0}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02 08:44:07,646 [urllib3.connectionpool   ] WARNING: Connection pool is full, discarding connection: localhost
2021-04-02T08:44:07-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'battery_charging_current': 3.0}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02 08:44:07,723 [urllib3.connectionpool   ] WARNING: Connection pool is full, discarding connection: localhost
2021-04-02T08:44:07-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'output_load_percent': 52.0}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02 08:44:07,794 [urllib3.connectionpool   ] WARNING: Connection pool is full, discarding connection: localhost
2021-04-02T08:44:07-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'battery_discharge_current': 0.0}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Grafana response: {u'status': u'success', u'uid': u'_cEkmc_Gk', u'url': u'/d/_cEkmc_Gk/solar-testdrive', u'slug': u'solar-testdrive', u'version': 2, u'id': 4}
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking dashboard "solar-testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02 08:44:07,904 [urllib3.connectionpool   ] WARNING: Connection pool is full, discarding connection: localhost
2021-04-02T08:44:07-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {'battery_voltage_from_scc': 27.9}, 'tags': {}, 'time_precision': 'n', 'measurement': 'area42_node1_sensors'}
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.manager] INFO    : Provisioning Grafana dashboard "solar-testdrive" for database "solar_testdrive" and measurement "area42_node1_sensors"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking/Creating datasource "solar_testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.dashboard] INFO    : No missing panels to add
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] ERROR   : Grafana provisioning failed for storage={"node": "node1", "slot": "data/pv2_input_voltage", "realm": "solar", "network": "testdrive", "database": "solar_testdrive", "measurement_events": "area42_node1_events", "label": "area42_node1", "measurement": "area42_node1_sensors", "gateway": "area42"}, message={'pv2_input_voltage': 0.0}:
    [Failure instance: Traceback: <class 'grafana_api_client.GrafanaPreconditionFailedError'>: Precondition failed: version-mismatch (`The dashboard has been changed by someone else`)
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:269:process_message
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/manager.py:148:provision
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/dashboard.py:110:make
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/api.py:142:create_dashboard
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:73:create
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:64:make_request
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:174:make_raw_request
    ]
2021-04-02T08:44:07-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/pv2_input_voltage
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.dashboard] INFO    : No missing panels to add
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.dashboard] INFO    : No missing panels to add
2021-04-02T08:44:07-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.dashboard] INFO    : No missing panels to add
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.services.mig            ] ERROR   : Grafana provisioning failed for storage={"node": "node1", "slot": "data/ac_output_active_power", "realm": "solar", "network": "testdrive", "database": "solar_testdrive", "measurement_events": "area42_node1_events", "label": "area42_node1", "measurement": "area42_node1_sensors", "gateway": "area42"}, message={'ac_output_active_power': 1262.0}:
    [Failure instance: Traceback: <class 'grafana_api_client.GrafanaPreconditionFailedError'>: Precondition failed: version-mismatch (`The dashboard has been changed by someone else`)
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:269:process_message
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/manager.py:148:provision
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/dashboard.py:110:make
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/api.py:142:create_dashboard
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:73:create
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:64:make_request
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:174:make_raw_request
    ]
2021-04-02T08:44:08-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/ac_output_active_power
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.dashboard] INFO    : No missing panels to add
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.dashboard] INFO    : No missing panels to add
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Get folder with uid="instagraf"
2021-04-02T08:44:08-0400 [kotori.daq.services.mig            ] ERROR   : Grafana provisioning failed for storage={"node": "node1", "slot": "data/battery_capacity", "realm": "solar", "network": "testdrive", "database": "solar_testdrive", "measurement_events": "area42_node1_events", "label": "area42_node1", "measurement": "area42_node1_sensors", "gateway": "area42"}, message={'battery_capacity': 100.0}:
    [Failure instance: Traceback: <class 'grafana_api_client.GrafanaPreconditionFailedError'>: Precondition failed: version-mismatch (`The dashboard has been changed by someone else`)
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:269:process_message
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/manager.py:148:provision
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/dashboard.py:110:make
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/api.py:142:create_dashboard
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:73:create
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:64:make_request
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:174:make_raw_request
    ]
2021-04-02T08:44:08-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/battery_capacity
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.dashboard] INFO    : No missing panels to add
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Grafana response: {u'status': u'success', u'uid': u'_cEkmc_Gk', u'url': u'/d/_cEkmc_Gk/solar-testdrive', u'slug': u'solar-testdrive', u'version': 4, u'id': 4}
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.dashboard] INFO    : No missing panels to add
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Grafana response: {u'status': u'success', u'uid': u'_cEkmc_Gk', u'url': u'/d/_cEkmc_Gk/solar-testdrive', u'slug': u'solar-testdrive', u'version': 3, u'id': 4}
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.services.mig            ] ERROR   : Grafana provisioning failed for storage={"node": "node1", "slot": "data/mppt1_charger_temperature", "realm": "solar", "network": "testdrive", "database": "solar_testdrive", "measurement_events": "area42_node1_events", "label": "area42_node1", "measurement": "area42_node1_sensors", "gateway": "area42"}, message={'mppt1_charger_temperature': 34.0}:
    [Failure instance: Traceback: <class 'grafana_api_client.GrafanaPreconditionFailedError'>: Precondition failed: version-mismatch (`The dashboard has been changed by someone else`)
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:269:process_message
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/manager.py:148:provision
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/dashboard.py:110:make
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/api.py:142:create_dashboard
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:73:create
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:64:make_request
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:174:make_raw_request
    ]
2021-04-02T08:44:08-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/mppt1_charger_temperature
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.dashboard] INFO    : No missing panels to add
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Getting dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.dashboard] INFO    : No missing panels to add
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.services.mig            ] ERROR   : Grafana provisioning failed for storage={"node": "node1", "slot": "data/mppt2_charger_temperature", "realm": "solar", "network": "testdrive", "database": "solar_testdrive", "measurement_events": "area42_node1_events", "label": "area42_node1", "measurement": "area42_node1_sensors", "gateway": "area42"}, message={'mppt2_charger_temperature': 0.0}:
    [Failure instance: Traceback: <class 'grafana_api_client.GrafanaPreconditionFailedError'>: Precondition failed: version-mismatch (`The dashboard has been changed by someone else`)
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:269:process_message
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/manager.py:148:provision
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/dashboard.py:110:make
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/api.py:142:create_dashboard
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:73:create
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:64:make_request
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:174:make_raw_request
    ]
2021-04-02T08:44:08-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/mppt2_charger_temperature
2021-04-02T08:44:08-0400 [kotori.daq.services.mig            ] ERROR   : Grafana provisioning failed for storage={"node": "node1", "slot": "data/pv2_input_power", "realm": "solar", "network": "testdrive", "database": "solar_testdrive", "measurement_events": "area42_node1_events", "label": "area42_node1", "measurement": "area42_node1_sensors", "gateway": "area42"}, message={'pv2_input_power': 0.0}:
    [Failure instance: Traceback: <class 'grafana_api_client.GrafanaPreconditionFailedError'>: Precondition failed: version-mismatch (`The dashboard has been changed by someone else`)
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:269:process_message
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/manager.py:148:provision
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/dashboard.py:110:make
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/api.py:142:create_dashboard
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:73:create
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:64:make_request
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:174:make_raw_request
    ]
2021-04-02T08:44:08-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/pv2_input_power
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.dashboard] INFO    : No missing panels to add
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Grafana response: {u'status': u'success', u'uid': u'_cEkmc_Gk', u'url': u'/d/_cEkmc_Gk/solar-testdrive', u'slug': u'solar-testdrive', u'version': 5, u'id': 4}
2021-04-02T08:44:08-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking dashboard "solar-testdrive"
2021-04-02T08:44:08-0400 [kotori.daq.services.mig            ] ERROR   : Grafana provisioning failed for storage={"node": "node1", "slot": "data/grid_frequency", "realm": "solar", "network": "testdrive", "database": "solar_testdrive", "measurement_events": "area42_node1_events", "label": "area42_node1", "measurement": "area42_node1_sensors", "gateway": "area42"}, message={'grid_frequency': 60.0}:
    [Failure instance: Traceback: <class 'grafana_api_client.GrafanaPreconditionFailedError'>: Precondition failed: version-mismatch (`The dashboard has been changed by someone else`)
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:269:process_message
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/manager.py:148:provision
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/dashboard.py:110:make
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/api.py:142:create_dashboard
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:73:create
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:64:make_request
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:174:make_raw_request
    ]
2021-04-02T08:44:08-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/grid_frequency
2021-04-02 08:44:08,985 [urllib3.connectionpool   ] WARNING: Connection pool is full, discarding connection: localhost
2021-04-02 08:44:08,992 [urllib3.connectionpool   ] WARNING: Connection pool is full, discarding connection: localhost
2021-04-02 08:44:09,002 [urllib3.connectionpool   ] WARNING: Connection pool is full, discarding connection: localhost
2021-04-02T08:44:09-0400 [kotori.daq.services.mig            ] ERROR   : Grafana provisioning failed for storage={"node": "node1", "slot": "data/ac_output_apparent_power", "realm": "solar", "network": "testdrive", "database": "solar_testdrive", "measurement_events": "area42_node1_events", "label": "area42_node1", "measurement": "area42_node1_sensors", "gateway": "area42"}, message={'ac_output_apparent_power': 1339.0}:
    [Failure instance: Traceback: <class 'grafana_api_client.GrafanaPreconditionFailedError'>: Precondition failed: version-mismatch (`The dashboard has been changed by someone else`)
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:269:process_message
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/manager.py:148:provision
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/dashboard.py:110:make
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/api.py:142:create_dashboard
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:73:create
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:64:make_request
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:174:make_raw_request
    ]
2021-04-02T08:44:09-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/ac_output_apparent_power
2021-04-02 08:44:09,039 [urllib3.connectionpool   ] WARNING: Connection pool is full, discarding connection: localhost
2021-04-02 08:44:09,043 [urllib3.connectionpool   ] WARNING: Connection pool is full, discarding connection: localhost
2021-04-02T08:44:09-0400 [kotori.daq.services.mig            ] ERROR   : Grafana provisioning failed for storage={"node": "node1", "slot": "data/local_parallel_id", "realm": "solar", "network": "testdrive", "database": "solar_testdrive", "measurement_events": "area42_node1_events", "label": "area42_node1", "measurement": "area42_node1_sensors", "gateway": "area42"}, message={'local_parallel_id': 0.0}:
    [Failure instance: Traceback: <class 'grafana_api_client.GrafanaPreconditionFailedError'>: Precondition failed: version-mismatch (`The dashboard has been changed by someone else`)
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:269:process_message
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/manager.py:148:provision
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/dashboard.py:110:make
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/api.py:142:create_dashboard
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:73:create
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:64:make_request
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:174:make_raw_request
    ]
2021-04-02T08:44:09-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/local_parallel_id
2021-04-02T08:44:09-0400 [kotori.daq.graphing.grafana.dashboard] INFO    : No missing panels to add
2021-04-02T08:44:09-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:09-0400 [kotori.daq.graphing.grafana.dashboard] INFO    : No missing panels to add
2021-04-02T08:44:09-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:09-0400 [kotori.daq.services.mig            ] ERROR   : Grafana provisioning failed for storage={"node": "node1", "slot": "data/output_load_percent", "realm": "solar", "network": "testdrive", "database": "solar_testdrive", "measurement_events": "area42_node1_events", "label": "area42_node1", "measurement": "area42_node1_sensors", "gateway": "area42"}, message={'output_load_percent': 52.0}:
    [Failure instance: Traceback: <class 'grafana_api_client.GrafanaPreconditionFailedError'>: Precondition failed: version-mismatch (`The dashboard has been changed by someone else`)
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:269:process_message
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/manager.py:148:provision
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/dashboard.py:110:make
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/api.py:142:create_dashboard
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:73:create
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:64:make_request
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:174:make_raw_request
    ]
2021-04-02T08:44:09-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/output_load_percent
2021-04-02T08:44:09-0400 [kotori.daq.graphing.grafana.dashboard] INFO    : No missing panels to add
2021-04-02T08:44:09-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Creating/updating dashboard "solar-testdrive"
2021-04-02T08:44:09-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Grafana response: {u'status': u'success', u'uid': u'_cEkmc_Gk', u'url': u'/d/_cEkmc_Gk/solar-testdrive', u'slug': u'solar-testdrive', u'version': 6, u'id': 4}
2021-04-02T08:44:09-0400 [kotori.daq.graphing.grafana.api    ] INFO    : Checking dashboard "solar-testdrive"
2021-04-02T08:44:09-0400 [kotori.daq.services.mig            ] ERROR   : Grafana provisioning failed for storage={"node": "node1", "slot": "data/battery_voltage_from_scc", "realm": "solar", "network": "testdrive", "database": "solar_testdrive", "measurement_events": "area42_node1_events", "label": "area42_node1", "measurement": "area42_node1_sensors", "gateway": "area42"}, message={'battery_voltage_from_scc': 27.9}:
    [Failure instance: Traceback: <class 'grafana_api_client.GrafanaPreconditionFailedError'>: Precondition failed: version-mismatch (`The dashboard has been changed by someone else`)
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:269:process_message
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/manager.py:148:provision
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/dashboard.py:110:make
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/api.py:142:create_dashboard
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:73:create
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:64:make_request
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:174:make_raw_request
    ]
2021-04-02T08:44:09-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/battery_voltage_from_scc
2021-04-02 08:44:09,215 [urllib3.connectionpool   ] WARNING: Connection pool is full, discarding connection: localhost
2021-04-02T08:44:09-0400 [kotori.daq.services.mig            ] ERROR   : Grafana provisioning failed for storage={"node": "node1", "slot": "data/battery_charging_current", "realm": "solar", "network": "testdrive", "database": "solar_testdrive", "measurement_events": "area42_node1_events", "label": "area42_node1", "measurement": "area42_node1_sensors", "gateway": "area42"}, message={'battery_charging_current': 3.0}:
    [Failure instance: Traceback: <class 'grafana_api_client.GrafanaPreconditionFailedError'>: Precondition failed: version-mismatch (`The dashboard has been changed by someone else`)
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
    /opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
    --- <exception caught here> ---
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:269:process_message
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/manager.py:148:provision
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/dashboard.py:110:make
    /opt/kotori/lib/python2.7/site-packages/kotori/daq/graphing/grafana/api.py:142:create_dashboard
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:73:create
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:64:make_request
    /opt/kotori/lib/python2.7/site-packages/grafana_api_client/__init__.py:174:make_raw_request
    ]
2021-04-02T08:44:09-0400 [kotori.daq.services.mig            ] DEBUG   : Channel base topic is solar/testdrive/area42/node1/data/battery_charging_current
2021-04-02 08:44:09,253 [urllib3.connectionpool   ] WARNING: Connection pool is full, discarding connection: localhost

I do now see "solar-testdrive" listed among the instant dashboards, but there doesn't seem to be any data there.

amotl commented 3 years ago

Hi @danb35,

by observing the log output, I can well see there are some things which might want to be improved within the Kotori code base re. error handling, specifically when it comes to non-float values like strings in this case. Despite those warning and error messages croaking on specific measurement values, it still looks like everything would work, though.

In the Kotori log, I see things like

Grafana provisioning failed [...] Traceback: <class 'grafana_api_client.GrafanaPreconditionFailedError'>: Precondition failed: version-mismatch (`The dashboard has been changed by someone else`)

However, for all the string fields where conversion to float fails, this is probably not a bad sign. Still: The dashboard has been changed by someone else might really mean that you updated this dashboard in the meanwhile so Kotori is not able to do it again?

Maybe deleting the dashboard and letting Kotori recreate it (just by restarting it) might help. However, maybe it fails under the current conditions that those measurements are arriving one by one. It should work in general, though.

If you can't make it work by all means, have you considered manually adding some fields to the dashboard / panel?

On the other hand, it would be really nice to resolve this issue if the culprit is in the code base. I just investigated the test suite and found [1]. While this part of the test suite verifies that the data acquisition succeeds, it does not go the whole way like [2] in order to actually verify the state of the Grafana panel, making it qualify as a real e2e test. So, there is also room for improvements here to strengthen the test assertions for the different data acquisition flavors in order to be absolutely sure that Grafana instant dashboard creation also works when ingesting data by using single-readings.

With kind regards, Andreas.

[1] https://github.com/daq-tools/kotori/blob/0.26.6/test/test_daq_mqtt.py#L64-L83 [2] https://github.com/daq-tools/kotori/blob/0.26.6/test/test_daq_grafana.py

Incidentally, how do I turn off debug logging?

I believe there should be something in /etc/default/kotori, at least in order to tame the log level.

danb35 commented 3 years ago

Still: The dashboard has been changed by someone else might really mean that you updated this dashboard in the meanwhile

I had not done anything manually with the dashboard at that time.

Maybe deleting the dashboard and letting Kotori recreate it (just by restarting it) might help.

I deleted the dashboard, restarted Kotori, and re-ran the mpp-solar command. The dashboard is created, but it doesn't appear that any data is actually saved (as below)

have you considered manually adding some fields to the dashboard / panel?

I have, but it doesn't appear the data is actually populated:

image

image

amotl commented 3 years ago

... but it doesn't appear the data is actually populated.

I see, thank you. Absolutely, it looks like the datasource was not able to be created within Grafana.

It's probably worth noting that my Kotori instance is already logging data from my WeeWx system (which is also submitted via MQTT) without a problem.

May I ask you if you added a separate Kotori configuration snippet for your solar realm already? So, a similar file like weewx.ini, but probably a solar.ini instead.

On a package-based installation, the files I am referring to are usually found within the /etc/kotori/apps-{available,enabled} directories.

danb35 commented 3 years ago

May I ask you if you added a Kotori configuration snippet for your solar realm already?

root@kotori:/etc/default# cat /etc/kotori/apps-enabled/solar.ini 
; ------------------------------------------
; Name:     mqttkit
; Date:     March/April 2016
; About:    A generic distributed monitoring platform for
;           collecting sensor data in wide area network setups.
; Channel:  Transport: MQTT over TCP; Format: JSON
; Storage:  InfluxDB
; See also: https://getkotori.org/docs/applications/mqttkit.html
; ------------------------------------------

[solar]
enable      = true
type        = application
realm       = solar
mqtt_topics = solar/#
application = kotori.daq.application.mqttkit:mqttkit_application

# How often to log metrics
metrics_logger_interval = 60

[mqttkit-2]
enable      = false
type        = application
realm       = mqttkit-2
mqtt_topics = mqttkit-2/#
app_factory = kotori.daq.application.mqttkit:mqttkit_application

# How often to log metrics
metrics_logger_interval = 60
amotl commented 3 years ago

Thank you. Hm, I am running out of ideas now. However, as outlined within https://github.com/daq-tools/kotori/issues/50#issuecomment-812561791, I will try to reproduce the scenario on my end on behalf of improving the test case for single-readings.

In the meanwhile, in order to work around the issue, you might want to also create your Grafana data source manually. Look at those entries coming from the amazonas_ecuador example and your weewx_* and create a dedicated one for solar_testdrive.

danb35 commented 3 years ago

Looks like JSON-over-MQTT is coming soon: https://github.com/jblance/mpp-solar/issues/75#issuecomment-812912153

Since that seems to be what Kotori would prefer, seems like the best course of action would be to wait for that feature to land and revisit at that time.

jblance commented 3 years ago

Is there a particular format that is preferred? Do you have an example?

amotl commented 3 years ago

Dear John,

thank you very much for considering to improve support for data acquisition with Kotori using the json_mqtt output formatter on behalf of https://github.com/jblance/mpp-solar/issues/75.

Is there a particular format that is preferred? Do you have an example?

You can find a basic and canonical example based on mosquitto_pub at [1].

So, if you can make mpp-solar publish the equivalent of the following messages to the MQTT broker, it would be great:

CHANNEL_TOPIC=amazonas/ecuador/cuyabeno/1/data.json
mosquitto_pub -t $CHANNEL_TOPIC -m '{"ac_output_voltage": 230.0, "ac_output_frequency": 5.0}'

With kind regards, Andreas.

[1] https://getkotori.org/docs/getting-started/basic-mqtt.html#send-sample-telemetry-packet

amotl commented 3 years ago

Ah, you already implemented this on behalf of https://github.com/jblance/mpp-solar/commit/128338e. That was quick! Thanks a stack.

danb35 commented 3 years ago

...and this now appears to be working:

root@solar:~# export MQTT_TOPIC=solar/testdrive/area42/node1
root@solar:~# mpp-solar -p /dev/hidraw1 -P PI18 -c GS -o json_mqtt -q kotori --mqtttopic ${MQTT_TOPIC}/data.json

...and the log:

2021-04-06T05:34:44-0400 [kotori.daq.storage.influx          ] DEBUG   : Storage success: {'fields': {u'monthET': 0.0, u'barometer_inHg': 30.099, u'outHumidity': 92.0, u'altimeter_inHg': 30.1019320487891, u'rainAlarm': 0.0, u'windGust_mph': 0.0, u'dayET': 0.0, u'interval_minute': 1.0, u'dewpoint_F': 47.47099757972822, u'consBatteryVoltage_volt': 4.6, u'wind_samples': 19.0, u'rain_in': 0.0, u'insideAlarm': 0.0, u'yearRain_in': 36.64, u'extraAlarm5': 0.0, u'cloudbase_foot': 531.591459152678, u'heatindex_F': 48.694, u'outsideAlarm1': 0.0, u'yearET': 0.0, u'outsideAlarm2': 0.0, u'inDewpoint_F': 54.136135882751645, u'forecastRule': 44.0, u'dayRain_in': 0.0, u'stormRain_in': 0.0, u'windrun_mile': 0.0, u'sunrise': 1617707160.0, u'outTemp_F': 49.7, u'appTemp_F': 49.12537046180201, u'windchill_F': 49.7, u'soilLeafAlarm2': 0.0, u'extraAlarm1': 0.0, u'extraAlarm2': 0.0, u'extraAlarm3': 0.0, u'extraAlarm4': 0.0, u'hourRain_in': 0.0, u'extraAlarm6': 0.0, u'extraAlarm7': 0.0, u'extraAlarm8': 0.0, u'humidex_F': 50.88152482746953, u'rainRate_inch_per_hour': 0.0, u'forecastIcon': 6.0, u'rxCheckPercent': 81.14583333333333, u'soilLeafAlarm4': 0.0, u'pressure_inHg': 30.07206492365863, u'inTemp_F': 72.8, u'soilLeafAlarm3': 0.0, u'usUnits': 1.0, u'soilLeafAlarm1': 0.0, u'leafWet4': 0.0, u'ET_in': 0.0, u'lowOutTemp': 49.6, u'txBatteryStatus': 0.0, u'windSpeed_mph': 0.0, u'rain24_in': 0.0, u'inHumidity': 52.0, u'windSpeed10_mph': 0.0, u'highOutTemp': 49.7, u'monthRain_in': 0.0, u'sunset': 1617752880.0}, 'tags': {}, 'time_precision': 's', 'time': 1617701640, 'measurement': 'richmondhill_vantagepro2_sensors'}
2021-04-06T05:34:44-0400 [kotori.daq.services.mig            ] DEBUG   : Provisioning Grafana with GrafanaManager

...and the result: image

amotl commented 3 years ago

Hi @danb35,

it is nice to see this works for you well. Thank you very much for this quick turnaround, @jblance!

With kind regards, Andreas.

danb35 commented 3 years ago

Now having imported an existing dashboard from grafana.com and edited it a bit, here's what I'm able to get: image

It's working well, thanks for the guidance.

amotl commented 3 years ago

Hi Dan,

thank you very much for sharing your outcome dashboard-wise, I really appreciate it!

As Kotori's gallery [1] does not include a corresponding energy monitoring use-case yet, I would love to finally create such a walkthrough and I believe using mpp-solar there would make absolute sense.

So, if you would like to volunteer on such a document, may I humbly ask you if you can share some images of your hardware setup? It would be nice to exhibit more real world scenarios where Kotori is used, optimally similar to https://getkotori.org/docs/gallery/hydro2motion.html or https://getkotori.org/docs/gallery/weewx.html.

With kind regards, Andreas.

[1] https://getkotori.org/docs/gallery/

danb35 commented 3 years ago

Here are a few pictures:

The panels--eight used 250W panels, mounted to the roof of my workshop: image

The inverter--a MPP Solar all-in-one 2.4kW unit: image

The batteries--eight 272 Ah LiFePO4 cells in series, with the BMS toward the right: image

The data that's fed into Kotori comes from the inverter, which has a USB interface. That's connected to a virtual machine running mpp-solar, which in turn feeds the data to Kotori (running on a different VM), as discussed above in this issue. The BMS is also supposed to be capable of communicating with the computer, to get more detailed information about the battery status, but I'm having some trouble getting that to work.

amotl commented 3 years ago

Hi Dan,

very nice, I really love those pictures from the real world. Thanks a stack for sharing them.

With kind regards, Andreas.

amotl commented 3 years ago

The BMS is also supposed to be capable of communicating with the computer, to get more detailed information about the battery status, but I'm having some trouble getting that to work.

While I can't promise anything, I want to offer that I might be able to look into that. Feel free to share any more details about it.

danb35 commented 3 years ago

The BMS has a RS485 port, and came with a USB-RS485 adapter and Windows software that's supposed to communicate with it--but the software doesn't connect. I've tried replacing the USB-RS485 adapter without success. I'm not sure at this point if it's a problem with how I'm connecting to the device (i.e., wiring), or if the BMS is just defective in this regard. Unfortunately, support from the vendor is minimal.

I've opened an issue against mpp-solar (https://github.com/jblance/mpp-solar/issues/83) to support this unit, and he seems willing to add that support, but right now I can't get the unit to respond at all.

amotl commented 3 years ago

Ah all right, you are in touch with @jblance about this topic already. So, as soon as the computer will be able to communicate with this device, you will be in good hands. Good luck with everything in this regard and thank you for sharing more details about it.