iotux / ElWiz

ElWiz - a program to read data from Tibber Pulse and similar devices

MIT License

109 stars 16 forks source link

readme

ElWiz - a program to read data from Tibber Pulse

Contents

ElWiz - a program to read data from Tibber Pulse
- Contents
- Breaking changes
- Intro
  - What you need
  - Nice to have but not required
- Installation
  ElWiz configuration
  
  The file config.yaml.sample is copied to config.yaml. If you install programs on the same machine as your local broker, you probably don't need to make any further changes in config.yaml. Otherwise, it will be necessary to enter the IP address and possibly the username and password of your own MQTT broker. The most important parameters in the configuration file look like this:
```
---
# Replace with your own MQTT broker
mqttUrl: "mqtt://localhost:1883"
mqttOptions:
  username:
  password:

# meterModel can be kaifa, aidon or kamstrup
meterModel: kaifa

# Tibber Pulse listening topic
topic: tibber

# ElWiz publishing topics
pubTopic: meter/ams
pubStatus: meter/status
pubNotice: meter/notice

# Publish options for list 1, 2, 3 & status
list1Retain: false
list1Qos: 0
list2Retain: false
list2Qos: 0
list3Retain: true
list3Qos: 1

statusRetain: false
statusQos: 0

# ElWiz event messages
willMessage: ElWiz has left the building
greetMessage: ElWiz is performing

# Tibber Pulse event messages
onlineMessage: Pulse is talking
offlineMessage: Pulse is quiet

# Debug mode at startup
DEBUG: false
debugTopic: debug/hex

# User has production (solar panels)
hasProduction: false

#############################################
# Pssible cacheType values
#   file
#   redis
cacheType: file

# Possivle storage type
#   mongodb
#   mariadb 
#   custom
#   none
storage: none

#############################################
# Possible publishing modes
#   hassPublish
#   basicPublish
#   customPublish
publisher: hassPublish

#############################################
# Publish to Home Assistant (defaults to TRUE)?
hassPublish: true
# Home Assistant sensor base topic (defaults to "elwiz/sensor")
haBaseTopic: elwiz

# Don't change the following topic unless you
# have changed the way HomeAssistant read
# MQTT messages
haAnnounceTopic: homeassistant

# Home Assistant sensor base topic (defaults to "elwiz/sensor")
# This is different from "pubTopic" to separate it from basic use of ElWiz
# A separate topic will also prevent "spamming" of HA
haBaseTopic: elwiz
```
  It is worth noting the following:
  
  topic under # Listening topics must match what is specified in mqtt_topic when configuring Pulse. Other changes should not normally be required.
  
  The program has "ready-made" integration for Home Assistant as configured. For other systems some configuration changes are likely needed. A plugin system is used to transform Pulse messages to other formats.
  
  ElWiz chart configuration
  
  The ElWiz chart program has its own configuration file chart-config.yaml. This makes it possible to run the program along with fethprices.js without running ElWiz. The file config.yaml.sample is copied to config.yaml. The default parameters should be okay unless you are runnung in a remote environment. If so, you will need to set the chartConfig wsUrl parameter to the IP address of your chart server (server.js). Furthermore, currencyCode and languageCode should be set according to your own currency and language.
```
---
# The chart server configuration
# Changes will need a server restart
serverConfig:
  debug: false
  savePath: data
  serverPort: 8321
  wsServerPort: 8322
  # MQTT params
  # Replace with your own MQTT broker
  mqttUrl: "mqtt:localhost:1883"
  mqttOptions:
    username:
    password:

  # Priceinfo topic
  priceTopic: elwiz/prices
  chartTopic: elwiz/chart
  # Home Assistant
  hassPublish: true
  haBaseTopic: elwiz
  haAnnounceTopic: homeassistant

  # Factor for calculating green/red zones
  # % added/subracted to/from average price
  # It is permanent until changed here
  fixedAverageOffset: 0.0
  # Used for multiplying the value from the
  # adjustLeftAvgOffset and adjustRightAvgOffset
  # MQTT messages
  adjustmentStepFactor: 1.0

  # Change according to your own language/country
  currencyCode: NOK

# The chart web client configuration
# Reload browser to activate changes
chartConfig:
  # Replace with IP address of server if running remote
  # The port number needs to be
  # equal to the "wsServerPort" above
  wsUrl: "ws://localhost:8322/"

  # Set this to "true" if you want
  # dark mode on mobile devices
  darkMode: false

  # Set true if you want the Y axis to start at 0
  yBeginAtZero: false

  # Web client debug
  debug: false

  # Change according to your own country
  languageCode: nb-NO
  currencyCode: NOK
```
  Setup of Pulse
  
  The first step to connect Pulse to your own network is to force it into AP mode. By doing a hard reset, it will appear in the network as an access point. A paper clip is what is needed. Pulse has a small hole for resetting to factory defaults. It is on the opposite side of where the micro-USB connector is. It is usually most appropriate to supply Pulse with power from a mobile charger or similar. When the power is connected, use an unfolded paper clip in the small hole and press until Pulse begins to flash rapidly (after about 5 seconds). It should now be possible to find it in the network with the SSID Tibber Pulse. You must connect a PC or mobile phone to this. The password is on the back of the Pulse in bold text in a frame. When Pulse has accepted the connection, you can reach it in the browser at address http://10.133.70.1. Pulse's website that appears will look like this:
  
  The fields ssid and psk are filled in with the name of your own WiFi router and password.
  
  The fields mqtt_url and mqtt_port are filled in with the IP address of your own broker and port number 1883 for use without SSL. If the broker is set up to require authentication with username and password, this is entered in the field mqtt_url.
  
  If the username is janedoe and the password is secret1, then this is specified like this:
  
  janedoe:secret1@your.broker.address,
  
  where broker-address can be a FQDN hostname or IP adress.
  
  In the field mqtt_topic a freely chosen topic is entered. It should be different from the topic used in the program to send messages. As tibber is preselected in the program, it may be fine to use here.
  
  The mqtt_topic_sub field is a topic that Pulse subscribes to. To indicate that MQTT messages go the opposite way, you can e.g. use rebbit here. This ensures that it does not conflict with other MQTT messages. So far I have found that by sending the message "reboot", Pulse will respond with "Debug: rebooting" and reboot. If you e.g. sends the message "nonsense", then it will respond with "Debug: Unknown command 'nonsense'". There is more about this in the section Controlling Pulse.
  
  The update_url field seems to need a value. I have used the address of my own broker here. The purpose is obviously for upgrading the firmware in Pulse. It would be interesting to get information about this if anyone has.
  
  The other fields can be left empty unless you want to use SSL. When the fields have been filled in and sent to Pulse, a few seconds pass and it starts flashing green. It is a sign that Pulse is connectdto your own network. When that happens, it is no longer in AP mode and access to the web interface is no longer possible. Once this is done, simply plug the Pulse into the HAN connector of the AMS meter and the Pulse will start delivering MQTT messages.
  
  AMS meter data
  
  Data from the AMS meter comes in 3 different variants. List 1, List 2, and List 3 refer to NVE's documentation for AMS meters. Briefly described, it is as follows:
  - List 1 contains the relevant power consumption measured in kW, as well as the time. This type is received in 2 or 2.5 second intervals.
  - List 2 also contains power, current and voltage which are received in intervals of 10 seconds
  - List 3 contains, in addition to List 2, accumulated data for electricity used so far. This is received every full hour.
  This is described in more detail further down, as well as made visible in the examples below.
  
  Data from Pulse
  
  From Pulse comes a start-up message, status messages and AMS meter data. Pulse, on the other hand, lacks a LastWill message. Such a message should normally be sent to the broker when the device is started. If the broker loses contact with the device, it will send this message to subscribers. To compensate for this shortcoming, there is a "watchdog" feature in ElWiz. This is a counter that counts down with an interval of 1 second. When the program receives a message from Pulse, the program refreshes this counter. If data from Pulse is missing, the counter will continue to count down. When it gets the value 0, the program will send an MQTT message as a warning that there is no data from Pulse. The counter is initially set to 15 seconds, but this value can be changed in the program code.
```
// The watchdog timer
const watchValue = 15;
```
  MQTT data from ElWiz
  
  In ElWiz, the raw data from the AMS meter is converted to readable JSON format. It is not a given that the format is suitable for everyone. The possibility of "plugins" for individual adaptations has therefore been created.
  
  Users of fetchprices will have access to spot prices. Prices from own suppliers are entered in config.yaml, and costs are then calculated in ElWiz.
  
  Example of price data from Nord Pool:
```
"hourly": [
  {
    "startTime": "2024-07-22T11:00:00",
    "endTime": "2024-07-22T12:00:00",
    "spotPrice": 0.3559,
    "gridFixedPrice": 0.1925,
    "supplierFixedPrice": 0.0542
  }
]
```
  See separate documentation in fetchprices.md
  
  Continuous operation
  
  A handy tool to use for programs that must be running around the clock is PM2 https://pm2.keymetrics.io/ With PM2 you have control over stop, start, restart, automatic start after starting the PC/server, memory consumption, logging and much more. It's well worth the trouble to take a look.
  
  Control of Pulse
  
  Pulse has some features that can be controlled using MQTT messages. This is done by sending the messages with the topic specified in the mqtt_topic_sub field in the web interface. This is not documented, but by trying different options, I have found these functions.
  - reboot - Restarts Pulse
  - update - OTA update of driver software (information about "update_url" is missing)
  Those who use the mosquitto broker have access to mosquitto_pub to publish opinions. By using the mqtt_topic_sub that was specified in the setup of Pulse, e.g. rebbit, then a command to Pulse will look like this when you send the message reboot:
```
mosquitto_pub -h localhost -t rebbit -m reboot
Debug: Reboot
```
  By sending the update command, we saw this response:
```
mosquitto_pub -h localhost -t rebbit -m "update"
Debug: Update in progress
Debug: Firmware update failed: -1
```
  Home Assistant (HA) integration
  
  ElWiz has complete auto discovery integration for HA. A prerequisite for this is that Home Assistant MQTT Integration is installed.
  
  When ElWiz starts up, the program will be "discovered" by HA's auto discovery mechanism. This appears in the list of Devices in HA. There ElWiz presents itself as ElWiz. In the Energy panel, the MQTT lastMeterConsumption message can then be registered as the main source of imported electricity.
  
  The integration with HA is described in a separate document (coming)
  
  Signals to the program
  
  In programs that process data, it is mandatory to capture e.g. \<Ctrl C> or kill. This also happens when the program is run by PM2 and using the pm2 stop command. The purpose is to save the program state before it is killed.
  
  When the program is started, it is assigned a process ID (PID). This is printed to the console when the program starts and is used to send signals to the program. When the program is started, this message is written to the console:
```
ElWiz is performing, PID: 32512
```
  References
  
  During the mapping of data from Tibber Pulse, I have had good help from information from @daniel.h.iversen and @roarfred and other posts in this discussion forum https://www.hjemmeautomasjon.no/forums/topic/ 4255-tibber-pulse-mqtt/.
  
  Below are links with useful information for those interested in the decoding.