mandeltuete commented 4 years ago

Hello everybody,i have been using tasmota for a while now and would like to thank you for this great system. I would like to connect my Victron Energy components to the home automation system via tasmota and need a suitable driver. Victron Energy is a great manufacturer of solar components and offers a serial interface for reading out operating data on many devices. The interface uses the VE.Direct protocol. Unfortunately, we lack the necessary programming knowledge to develop a driver ourselves. There are already two projects that are working on reading data from Victron devices via Arduino / ESP8266. It may be possible to reuse parts of this code. I am quite sure that many other users would be happy about a suitable driver.

Links to the projects:

https://github.com/physee/Victron.Arduino-ESP8266 https://github.com/winginitau/VictronVEDirectArduino

VE.Direct Protocol https://www.victronenergy.com/live/vedirect_protocol:faq

Have you looked for this feature in other issues and in the docs?

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

(Please, remember to close the issue when the problem has been addressed)

rg2k commented 4 years ago

Did you look at the MQTT option? You will need a Victron Color Controller (or a Raspberry Pi), but this works really well for integration to most home automation systems.

mandeltuete commented 4 years ago

Yeah, I've been testing it for over a year. But the power consumption of the Raspberry Pi is too high and the Victron firmware is releatively unstable. During this time I had to reflash the SD card several times and bought different cards. I am also annoyed about the purchase of the expensive VE.Direct to USB cable (30€ / pcs.). Since I am using an ESP8266 with Tasmota on my solar system to record weather data anyway, connecting the Victron components would be very practical.

ascillato2 commented 4 years ago

Closing this issue as nobody is working on it. Sorry.

If anyone is interested on working on this, please just ask to reopen. Thanks.

Aikawa24 commented 1 year ago

hi, Fabian Lauer made a Script to use the dbus. ive got a esp32 with tasmota to read my smart meter data and get this into the Cerbo GX

https://github.com/fabian-lauer/dbus-shelly-3em-smartmeter

you can modify the script to get JSON data from Tasmota and integrate it to the Victron system

`#!/usr/bin/env python import platform import logging import sys import os if sys.version_info.major == 2: import gobject else: from gi.repository import GLib as gobject import sys import time import requests # for http GET sys.path.insert(1, os.path.join(os.path.dirname(file), '/opt/victronenergy/dbus-systemcalc-py/ext/velib_python')) from vedbus import VeDbusService class DbusMT681Service: def init(self, servicename, paths, productname='MT681', connection='MT681 JSON service'): deviceinstance = 42 customname = 'MT681-Netz' self.dbusservice = VeDbusService("{}.http{:02d}".format(servicename, deviceinstance)) self._paths = paths logging.debug("%s /DeviceInstance = %d" % (servicename, deviceinstance))

Create the management objects, as specified in the ccgx dbus-api document

self._dbusservice.add_path('/Mgmt/ProcessName', __file__)
self._dbusservice.add_path('/Mgmt/ProcessVersion', 'Unkown version, and running on Python ' + platform.python_version())
self._dbusservice.add_path('/Mgmt/Connection', connection)
self._dbusservice.add_path('/DeviceInstance', deviceinstance)
self._dbusservice.add_path('/ProductId', 0xFFFF) # id assigned by Victron Support from SDM630v2.py
self._dbusservice.add_path('/ProductName', productname)
self._dbusservice.add_path('/CustomName', customname)    
self._dbusservice.add_path('/Connected', 1)
self._dbusservice.add_path('/Role', 'grid')    
self._dbusservice.add_path('/Latency', None)    
self._dbusservice.add_path('/FirmwareVersion', 0.1)
self._dbusservice.add_path('/HardwareVersion', 0)
self._dbusservice.add_path('/Position', 0) # normaly only needed for pvinverter
self._dbusservice.add_path('/Serial', self._getSerial())
self._dbusservice.add_path('/UpdateIndex', 0)
self._dbusservice.add_path('/StatusCode', 0)  # Dummy path so VRM detects us as a PV-inverter.
# add path values to dbus
for path, settings in self._paths.items():
  self._dbusservice.add_path(
    path, settings['initial'], gettextcallback=settings['textformat'], writeable=True, onchangecallback=self._handlechangedvalue)
# last update
self._lastUpdate = 0
# add _update function 'timer'
gobject.timeout_add(250, self._update) # pause 250ms before the next request
# add _signOfLife 'timer' to get feedback in log every 5minutes
gobject.timeout_add(self._getSignOfLifeInterval()*60*1000, self._signOfLife)

def _getSerial(self): meter_data = self._getData()
if not meter_data['StatusSNS']['mt681']['Meter_id']: raise ValueError("Response does not contain 'mac' attribute") serial = meter_data['StatusSNS']['mt681']['Meter_id'] return serial def _getSignOfLifeInterval(self): value = 1 if not value: value = 0 return int(value) def _getData(self): URL = "http://192.168.100.23/cm?cmnd=status%208" meter_r = requests.get(url = URL)

check for response

if not meter_r:
    raise ConnectionError("No response from mt681 - %s" % (URL))
meter_data = meter_r.json()
# check for Json
if not meter_data:
    raise ValueError("Converting response to JSON failed")
return meter_data

def _signOfLife(self): logging.info("--- Start: sign of life ---") logging.info("Last _update() call: %s" % (self._lastUpdate)) logging.info("Last '/Ac/Power': %s" % (self._dbusservice['/Ac/Power'])) logging.info("--- End: sign of life ---") return True def _update(self):
try:

get data from Device

   meter_data = self._getData()
   self._dbusservice['/Ac/Power'] = meter_data['StatusSNS']['mt681']['Power_curr']
   self._dbusservice['/Ac/L1/Voltage'] = 230
   self._dbusservice['/Ac/L2/Voltage'] = 230
   self._dbusservice['/Ac/L3/Voltage'] = 230
   self._dbusservice['/Ac/L1/Power'] = meter_data['StatusSNS']['mt681']['Power_p1']
   self._dbusservice['/Ac/L2/Power'] = meter_data['StatusSNS']['mt681']['Power_p2']
   self._dbusservice['/Ac/L3/Power'] = meter_data['StatusSNS']['mt681']['Power_p3']
   self._dbusservice['/Ac/L1/Current'] = meter_data['StatusSNS']['mt682']['Power_p1'] /230
   self._dbusservice['/Ac/L2/Current'] = meter_data['StatusSNS']['mt682']['Power_p2'] /230
   self._dbusservice['/Ac/L3/Current'] = meter_data['StatusSNS']['mt682']['Power_p3'] /230
   self._dbusservice['/Ac/Energy/Forward'] = meter_data['StatusSNS']['mt681']['Total_out']
   self._dbusservice['/Ac/Energy/Reverse'] = meter_data['StatusSNS']['mt681']['Total_in']
   #logging
   logging.debug("House Consumption (/Ac/Power): %s" % (self._dbusservice['/Ac/Power']))
   logging.debug("---");
   # increment UpdateIndex - to show that new data is available
   index = self._dbusservice['/UpdateIndex'] + 1  # increment index
   if index > 255:   # maximum value of the index
     index = 0       # overflow from 255 to 0
   self._dbusservice['/UpdateIndex'] = index
   #update lastupdate vars
   self._lastUpdate = time.time()              
except Exception as e:
   logging.critical('Error at %s', '_update', exc_info=e)
# return true, otherwise add_timeout will be removed from GObject - 
# see docs http://library.isr.ist.utl.pt/docs/pygtk2reference/gobject-functions.html#function-gobject--timeout-add
return True

def _handlechangedvalue(self, path, value): logging.debug("someone else updated %s to %s" % (path, value)) return True # accept the change def main(): try: logging.info("Start"); from dbus.mainloop.glib import DBusGMainLoop DBusGMainLoop(set_as_default=True)

formatting

  _kwh = lambda p, v: (str(round(v, 2)) + 'KWh')
  _a = lambda p, v: (str(round(v, 1)) + 'A')
  _w = lambda p, v: (str(round(v, 1)) + 'W')
  _v = lambda p, v: (str(round(v, 1)) + 'V')
  #start our main-service
  pvac_output = DbusMT681Service(
    servicename='com.victronenergy.grid',
    paths={
      '/Ac/Energy/Forward': {'initial': 0, 'textformat': _kwh}, # energy bought from the grid
      '/Ac/Energy/Reverse': {'initial': 0, 'textformat': _kwh}, # energy sold to the grid
      '/Ac/Power': {'initial': 0, 'textformat': _w},
      '/Ac/Current': {'initial': 0, 'textformat': _a},
      '/Ac/Voltage': {'initial': 0, 'textformat': _v},
      '/Ac/L1/Voltage': {'initial': 0, 'textformat': _v},
      '/Ac/L2/Voltage': {'initial': 0, 'textformat': _v},
      '/Ac/L3/Voltage': {'initial': 0, 'textformat': _v},
      '/Ac/L1/Current': {'initial': 0, 'textformat': _a},
      '/Ac/L2/Current': {'initial': 0, 'textformat': _a},
      '/Ac/L3/Current': {'initial': 0, 'textformat': _a},
      '/Ac/L1/Power': {'initial': 0, 'textformat': _w},
      '/Ac/L2/Power': {'initial': 0, 'textformat': _w},
      '/Ac/L3/Power': {'initial': 0, 'textformat': _w},
    })

  #logging.info('Connected to dbus, and switching over to gobject.MainLoop() (= event based)')
  mainloop = gobject.MainLoop()
  mainloop.run()

except Exception as e: logging.critical('Error at %s', 'main', exc_info=e) if name == "main": main() `

primemaster commented 1 year ago

Hello all,

I was working the last couple of days on Victron VE.Direct serial RX support in Tastmota. The implementation is based on: https://www.victronenergy.com/upload/documents/VE.Direct-Protocol-3.32.pdf

HTML

The Tasmota main page currently looks like: Screenshot_20230420_225323_mod

JSON

curl -s 'http://<IP>/cm?cmnd=status%2010'

{"StatusSNS":{"Time":"2023-04-20T21:55:23","VictronVE.Direct":[{"field": "PID","unit": "","value": "0xA053","description": "Product ID"},{"field": "FW","unit": "","value": "161","description": "Firmware version (16 bit)"},{"field": "SER#","unit": "","value": "<REDACTED>","description": "Serial number"},{"field": "V","unit": "mV","value": "26460","description": "Main or channel 1 (battery) voltage"},{"field": "I","unit": "mA","value": "-220","description": "Main or channel 1 battery current"},{"field": "VPV","unit": "mV","value": "40800","description": "Panel voltage"},{"field": "PPV","unit": "W","value": "3","description": "Panel power"},{"field": "CS","unit": "","value": "3","description": "State of operation"},{"field": "MPPT","unit": "","value": "2","description": "Tracker operation mode"},{"field": "OR","unit": "","value": "0x00000000","description": "Off reason"},{"field": "ERR","unit": "","value": "0","description": "Error code"},{"field": "LOAD","unit": "","value": "ON","description": "Load output state (ON/OFF)"},{"field": "IL","unit": "mA","value": "300","description": "Load current"},{"field": "H19","unit": "0.01 kWh","value": "2530","description": "Yield total (user resettable counter)"},{"field": "H20","unit": "0.01 kWh","value": "54","description": "Yield today"},{"field": "H21","unit": "W","value": "258","description": "Maximum power today"},{"field": "H22","unit": "0.01 kWh","value": "30","description": "Yield yesterday"},{"field": "H23","unit": "W","value": "238","description": "Maximum power yesterday"},{"field": "HSDS","unit": "","value": "218","description": "Day sequence number (0..364)"},{"Checksum": "0x4e"}],"bytes_read": 622779,"MetaTS": {"Start_TS": 3083017, "End_TS": 3083033, "Delta": 16}}}

Before I create a PR, I'd like to ask:

Is the sensor folder actually the correct place for a Victron device? Current filename: tasmota/tasmota_xsns_sensor/xsns_107_victron.ino
The HTML table has four columns <tr> <th>%s</th> <td></td> <td>%s</td> <td></td> <td>%s</td> <td></td> <td>%s</td> </tr>, is that acceptable? I do not see the possibility for complex tables using the {e}, {s} and {m}

Any suggestions and thoughts?

arendst commented 1 year ago

As you seem to write a book I see that the current column count is an issue.

Also considering the amount of data you feel needs to be provided I suggest to stay using a sensor driver so most people can easily ignore the driver and some victron users can enable it for their use.

BTW the gui was never meant for providing this amount of data. Your backend should handle this.

primemaster commented 1 year ago

Thanks @arendst for you reply.

In the HTML view, the left side serial_data is what is actually transferred on the serial line. The right side looked_up is then translated via a look-up table in the code:

typedef struct {
    const char *label;
    const char *unit;
    const char *description;
} Tripple;

static const Tripple field_metadata[] = { 
  {"V",       "mV",        "Main or channel 1 (battery) voltage"},
  {"V2",      "mV",        "Channel 2 (battery) voltage"},
  {"V3",       "mV",       "Channel 3 (battery) voltage"},
  {"VS",       "mV",       "Auxiliary (starter) voltage"},
  {"VM",       "mV",       "Mid-point voltage of the battery bank"},
  {"DM",       "%%",       "Mid-point deviation of the battery bank"},
  {"VPV",      "mV",       "Panel voltage"},
  {"PPV",      "W",        "Panel power"},
  {"I",        "mA",       "Main or channel 1 battery current"},
  {"I2",       "mA",       "Channel 2 battery current"},
  {"I3",       "mA",       "Channel 3 battery current"},
  {"IL",       "mA",       "Load current"},
  {"LOAD",     "",         "Load output state (ON/OFF)"},
  {"T",        "°C",       "Battery temperature"},
  {"P",        "W",        "Instantaneous power"},
  {"CE",       "mAh",      "Consumed Amp Hours"},
  {"SOC",      "%%",       "State-of-charge"},
  {"TTG",      "Minutes",  "Time-to-go"},
  {"Alarm",    "",         "Alarm condition active"},
  {"Relay",    "",         "Relay state"},
  {"AR",       "",         "Alarm reason"},
  {"OR",       "",         "Off reason"},
  {"H1",       "mAh",      "Depth of the deepest discharge"},
  {"H2",       "mAh",      "Depth of the last discharge"},
  {"H3",       "mAh",      "Depth of the average discharge"},
  {"H4",       "",         "Number of charge cycles"},
  {"H5",       "",         "Number of full discharges"},
  {"H6",       "mAh",      "Cumulative Amp Hours drawn"},
  {"H7",       "mV",       "Minimum main (battery) voltage"},
  {"H8",       "mV",       "Maximum main (battery) voltage"},
  {"H9",       "Seconds",  "Number of seconds since last full charge"},
  {"H10",      "",         "Number of automatic synchronizations"},
  {"H11",      "",         "Number of low main voltage alarms"},
  {"H12",      "",         "Number of high main voltage alarms"},
  {"H13",      "",         "Number of low auxiliary voltage alarms"},
  {"H14",      "",         "Number of high auxiliary voltage alarms"},
  {"H15",      "mV",       "Minimum auxiliary (battery) voltage"},
  {"H16",      "mV",       "Maximum auxiliary (battery) voltage"},
  {"H17",      "0.01 kWh", "Amount of discharged energy (BMV) / Amount of produced energy (DC monitor)"},
  {"H18",      "0.01 kWh", "Amount of charged energy (BMV) / Amount of consumed energy (DC monitor)"},
  {"H19",      "0.01 kWh", "Yield total (user resettable counter)"},
  {"H20",      "0.01 kWh", "Yield today"},
  {"H21",      "W",        "Maximum power today"},
  {"H22",      "0.01 kWh", "Yield yesterday"},
  {"H23",      "W",        "Maximum power yesterday"},
  {"ERR",      "",         "Error code"},
  {"CS",       "",         "State of operation"},
  {"BMV",      "",         "Model description (deprecated)"},
  {"FW",       "",         "Firmware version (16 bit)"},
  {"FWE",      "",         "Firmware version (24 bit)"},
  {"PID",      "",         "Product ID"},
  {"SER#",     "",         "Serial number"},
  {"HSDS",     "",         "Day sequence number (0..364)"},
  {"MODE",     "",         "Device mode"},
  {"AC_OUT_V", "0.01 V",   "AC output voltage"},
  {"AC_OUT_I", "0.1 A",    "AC output current"},
  {"AC_OUT_S", "VA",       "AC output apparent power"},
  {"WARN",     "",         "Warning reason"},
  {"MPPT",     "",         "Tracker operation mode"},
  {"MON",      "",         "DC monitor mode"}
};

This table is from the Victron documentation: Screenshot_20230421_130837

BTW the gui was never meant for providing this amount of data. Your backend should handle this.

Does that mean, that the HTML view should only show the two left most columns (which are actually coming from the serial line)?

arendst commented 1 year ago

Add your intelligence to it like supplying the correct kWh value (multiplied by 0.01) and add the correct unit in the unit column. Let users use command energyres to show their desired resolution. Give those three letter words user recognisable short names. Who cares what the serial output looks like.

primemaster commented 1 year ago

Adding some functionality to compute the final result for the HTML view makes sens. Also to add meaningful names instead of the Victron labels. Regarding the desired resolution I do not fully understand what there should be done. I found EnergyFmt() and Settings->flag2.energy_resolution in some other places of the Tasmota code. You mean to use such kind of formatting functions? If so, where/how can the user dynamically set the value of that flag?

arendst commented 1 year ago

If so, where/how can the user dynamically set the value of that flag?

Look for commands VoltRes, AmpRes, WattRes, EnergyRes all available if an energy driver is active. As you'll be writing a dedicated sensor driver I suggest you take a look at driver xdrv_03_energy.ino how to write your own user support.

As an alternative you might want to NOT write a sensor driver but instead write an energy driver (xnrg_25_victron.ino) which is serviced by the energy driver. It will probably need some default values for voltage, current and power but it can easily be extended with your abundance of register values. In that case have a look at the xnrg_08_sdm120.ino driver which adds some registers too.

Bersaker commented 1 year ago

Sounds interesting... Have you made any progress on this topic?

primemaster commented 11 months ago

Hello @Bersaker, I did not really continue on the implementation. As I have now a second Victron solar controller, I was thinking about adding support for multiple (2-4) controller to the same ESP8266. GND of the UART is the same as BATT-. Which means it is electrically possible if the controllers share the same BATT-. Of course that would mean up to 4 soft serials at 19200 baud. Where I don't know if that would be feasible. Especially as the each controller outputs its status every second taking ~100ms per controller. Which mean that the intervals might collide. Additionally the HTML UI would be a mess.

Any thoughts on supporting multiple controllers?

primemaster commented 10 months ago

Hello @Bersaker, I did not really continue on the implementation. As I have now a second Victron solar controller, I was thinking about adding support for multiple (2-4) controller to the same ESP8266. GND of the UART is the same as BATT-. Which means it is electrically possible if the controllers share the same BATT-. Of course that would mean up to 4 soft serials at 19200 baud. Where I don't know if that would be feasible. Especially as the each controller outputs its status every second taking ~100ms per controller. Which mean that the intervals might collide. Additionally the HTML UI would be a mess.

Any thoughts on supporting multiple controllers?

@arendst : Any thoughts on supporting multiple (up to e.g. 4) Victron controllers with one Tasmota instance?

pbg42 commented 10 months ago

Hello @primemaster, I have two Victron devices

Blue Solar MPPT 75 I 15 solar charger
Phoenix 12/250 converter

for an off-grid PV power supply and I want to log their data into my iobroker using MQTT. I already had a python script for a RASPI, but that is total overkill of hardware, so I want to use a cheap 5€ ESP01 board for each device to transfer the data with MQTT. I am very interested in your implementation, so that I can either complete it / adopt it for the Phoenix converter. I don´t think a multiple channel solution is required ( for me), I prefer 1 device for one task. Are you willing to share your current solution?

primemaster commented 10 months ago

Hello @pbg42,

as I haven't worked on it in some month, I don't know in what shape the code is :-). As there was no suggestion yet by @arendst if support for multiple VE.Direct input would be appreciated, I was planning to rewrite the code to support multiple input UARTs however limit to one connection by default.

pbg42 commented 10 months ago

Hi Micha, I don't care about the shape of the code; as it is already an almost working version, it shows at least which part of the project you made the extensions / modifications. So much much better than starting from scratch alone. So, I would be very happy I you could share your code, I won#t bother you with questions if you are busy. Or, do you think you will be done with rewriting within the next few weeks?

primemaster commented 9 months ago

Hi @pbg42

Hi Micha, I don't care about the shape of the code; as it is already an almost working version, it shows at least which part of the project you made the extensions / modifications. So much much better than starting from scratch alone. So, I would be very happy I you could share your code, I won#t bother you with questions if you are busy. Or, do you think you will be done with rewriting within the next few weeks?

I pushed my current state to https://github.com/micha-m/Tasmota_VE_PoC/commits/master . Please note:

Code is far away from a PR here
The Tasmota code base was forked in April 2023 (needs rebase!)
HTML view is still not in a nice table (not sure if that is actually possible)
JSON / MQTT is not tested
Not tested on ESP32
Seems to keep up with (at least) two software serial inputs => Two controllers would be supported
Prepared for up to 4 controllers
Not using any of those fancy "energy" formatting functions (code size increase significant in my opinion)
Still a lot of debugging prints included
More cleanup (e.g. removing initial single serial support code) needed

I am happy to receive constructive feedback!

pbg42 commented 9 months ago

Thank you @primemaster , I will try to understand, learn and see how I can read and control my two devices based on that. I will give feedback, but don't expect before christmas - too busy with other things. Thanks a lot for sharing! :+1:

pin32 commented 6 months ago

Hi, has this job made any progress?

pbg42 commented 6 months ago

Kind of :-) in this fork https://github.com/pbg42/Tasmota_VE_PoC I added/modified the fork of micha-m:

support for up to 4 victron devices (tested with my 2 owned + 2 simulated by doubling the inputs)
formatted output in SI units in web GUI and MQTT
transfered the table data to PROGMEM to reduce RAM size
prepared reading of HEX protocol and command interface, but this is not functional yet

what is still missing:

rebase to current development branch (it is still based on April 2023 fork from Micha-m)
HEX protocol command trigger and control
documentation of interface in the tasmota docs

so, currently far away from a pull request to tasmota development.

nulleinspeisung commented 5 months ago

hi i would like to test it. I have a victron 100/20 mppt and would like to read data via tasmota. Furthermore i would like to send data via mqtt by global variable. I am not sure how to start. Do i have to change the firmware of tasmota?

do i have to change the script on tasmota webui ? If yes, what script should i use ?

arendst / Tasmota

Integration of Victron Energy components #8433

Create the management objects, as specified in the ccgx dbus-api document

check for response

get data from Device

formatting

HTML

JSON