Open rucksman opened 3 years ago
@rucksman Phew, jessie is considered oldoldstable and not supported anymore (almost 6 years old). To add support for jessie would be hard. While I could lower the cmake version requirement, there are other issues like C++17 which will cause troubles. Why not just upgrade (or partially upgrade) your raspbian installation?
The issue is, that your cmake installation does not find the cmake scripts provided by msgpack and mosquittopp. CMake naming scheme does not reflect installation packages naming and should be find, though.
Well, it is a production machine, so "just upgrade" is not possible or at least not easy.
Ok, if it is really a production machine and connected to the internet, you should actually upgrade as soon as possible. Any security vulnerabilities are not fixed for jessie anymore. Doing a dist-upgrade is actually a regular process for a debian system and should work straight ahead. But (as mentioned before) you can also mix your debian system using apt-pinning.
But, thanks your testing. I can check, if i can lower the cmake requirement, but then you will be stuck with the c++17 requirement for libspeedwire which i am using as a dependency. Doing a backport to c++14 is painful as well and as a developer you do not want to dismiss modern C++ features.
So, i strongly recommend updating your raspbian for your own security (and time saving with compiling cmake and/or a c++17 compiler on your own). A compromise can be, that you do apt-pinning and proceed with a partial upgrade of your system. Cmake and GCC installations should not harm the rest of your system.
The raspberry is not reachable from outside.
Just to test it out I quickly assembled a docker container on a different machine and compiled your version. I had to fetch some files from the repo in order to start SBFspot (the cfg.file, date_time_zonespec.csv etc.).
With
./SBFspot -finq -nocsv -nosql -v5 -mqtt -installer -password:mypassword
I get (skipped some lines in the beginning and replaced more or less sensitive information)
Next poll: Mon Mar 29 16:57:31 2021
address: XXX.XXX.XXX.XXX mac: 00:00:00:00:00:00 name: "lo"
address: XXX.XXX.XXX.XXX mac: 00:00:00:00:00:00 name: "enp1s0"
address: XXX.XXX.XXX.XXX mac: 00:00:00:00:00:00 name: "tun0"
address: XXX.XXX.XXX.XXX mac: 00:00:00:00:00:00 name: "br-XXX"
address: XXX.XXX.XXX.XXX mac: 00:00:00:00:00:00 name: "docker0"
RECEIVED EMETER PACKET time 0x0000000013c0e751
Connecting to Local Network...
Initializing...
SUSyID: XXX - SessionID: XXX
Inverter IP address: XXX.XXX.XXX.XXX from SBFspot.cfg
ERROR: Connection to inverter failed!
Is XXX.XXX.XXX.XXX the correct IP?
Please check IP_Address in SBFspot.cfg!
Mon Mar 29 16:57:38 2021: CRITICAL: Failed to initialize Speedwire connection.Mon Mar 29 16:57:38 2021: INFO: Done.
The IP address of my inverter is correct. Doing the same command on my regular SBFspot machine works. The docker container is running with network_mode host, I can ping the inverter from inside the container. Any idea whats going worng here?
Hmm, hard to tell. Maybe there is still something blocked within your docker daemon. Does the original SBFspot version work with the docker container?
Yes, that works. Installed the original very quickly as described here https://github.com/SBFspot/SBFspot/wiki/Installation-Linux-SQLite. Did not do "make install_sqlite". Issued the same command as described above and got all the data. Renamed SBFspot to SBFspot.backup and copied your binary to the folder where the original (now renamed) file resides. Issued again the same command. Output as described above.
During execution of your version there is a short stop before "RECEIVED EMETER PACKET" and then a longer stop before "ERROR: Connection to inverter failed!".
Ok, this is strange. Apparantly, you cannot reuse the socket using docker. Maybe there is a setting, that allows you to reuse the same network socket...
I just did a try with a fresh installation of raspbian buster and it works: pi@raspberrypi:~/SBFspot/SBFspot/build $ ./SBFspot SBFspot V3.8.0 Yet another tool to read power production of SMA solar inverters (c) 2012-2021, SBF (https://github.com/SBFspot/SBFspot) Compiled for Linux (LE) 32 bit with SQLite support Commandline Args: address: 127.0.0.1 mac: 00:00:00:00:00:00 name: "lo" address: 192.168.0.106 mac: B8:27:EB:EE:2F:FA name: "wlan0" RECEIVED EMETER PACKET time 0x000000000013dd8b Inverter IP address: 192.168.0.98 from SBFspot.cfg pi@raspberrypi:~/SBFspot/SBFspot/build $
And yes, there might me a small delay before getting network packages. My plan is, to rewrite this part using Qt. However, it actually should work with my impl on a raspberry pi.
Ok, this is strange. Apparantly, you cannot reuse the socket using docker. Maybe there is a setting, that allows you to reuse the same network socket...
I do not understand... Probably I was not clear enough, so here in short again my steps:
So reusing the socket does work, otherwise the original version won't work. It is all inside the same docker container and inside the same folder.
EDIT: The host machine is NOT a raspberry but a regular x86 machine.
Yeah, i fully understood you process and it looks just fine. My SBFspot internally reuses local network addresses, while the original does not, because it does not open multiple sockets. I suspect, that this does not work within a docker environment (or might need some tweaking).
Just google a little bit for: "so_reuseaddr docker". There seem to be issues.
Gave it another try. Had a spare raspberry and a spare SD-Card. Installed latest Raspbian and compiled your version:
Next poll: Tue Mar 30 11:19:58 2021
address: 127.0.0.1 mac: 00:00:00:00:00:00 name: "lo"
address: XXX.XXX.XXX.XXX mac: 00:00:00:00:00:00 name: "eth0"
Error polling emeter: Invalid argument
Connecting to Local Network...
Initializing...
SUSyID: XXX - SessionID: XXX (XXX)
Inverter IP address: XXX.XXX.XXX.XXX from SBFspot.cfg
ERROR: Connection to inverter failed!
Is XXX.XXX.XXX.XXX the correct IP?
Please check IP_Address in SBFspot.cfg!
Tue Mar 30 11:20:06 2021: CRITICAL: Failed to initialize Speedwire connection.Tue Mar 30 11:20:06 2021: INFO: Done.
Compiled latest original version, works.
Hmm, very strange. While you tried my fork within the docker container, at least getting the emeter packet worked...
Are you actually familiar with C++? Then you could try to debug it. I am using QtCreator as IDE which offers a nice GUI also for debugging.
In any other case: i am not so happy with the networking implementation of SBFspot. If i find the time, i would like to rewrite it using Qt. As i guess, we could fix those kind of issues once and forever.
Sorry, I am not familiar with C++ at all. When trying to call SBFspot a few times, at least the emeter responded again:
RECEIVED EMETER PACKET time 0x0000000000d6b27b
Although I have trouble to understand what "0x0000000000d6b27b" means.
But still no luck with the inverter ...
Ok, this looks like an unstable connection. So, the UDP packets do not always arrive and are occasionly dropped. I guess, the original implementation has a higher timeout.
Can you pass "-loop" as command line parameter? Then the inverter and energy meter will be polled in a loop. There is also a configuration parameter to set the interval.
Until now I always used my old SBFspot.cfg and only changed the MQTT_ItemFormat. Now I saw that there is some information at the very end of your version of SBFspot.cfg. Is it necessary or is it only very specific to your plant? The loop interval seems to be 1 minute. Can I shorten it? How? Anyway there is no progress with the inverter: "ERROR: Connection to inverter failed!".
The information at the end of the config file is specific for my use cases. It is irrelevant for you. Look for a variable called LiveInterval.
However, i just pushed an update. Now, there is a new target called SBFspot_qt. Currently, i just receives the multicast packets from an energy meter and provides it using MQTT as initially described by me. This is the beginning of a complete rewrite of SBFspot and is dependent on the qt library. If you have the time, you can test that. I'd appreciate that.
Thanks a lot :)
Compiled the latest version. Had to install qtbase5-dev to get rid of errors during cmake. Then it compiled without errors.
How do I invoke SBFspot_qt? Same syntax as SBFspot? I tried with the same command line as before
./SBFspot_qt -finq -nocsv -nosql -v5 -mqtt -loop -installer -password:mypw
It prints the config and stops after
### End of Config ###
Then nothing is happening, have to stop with CTRL-C. There is no MQTT output, at least I do not see any traffic with MQTT Explorer.
When calling the regular SBFspot (no _qt) that was compiled, it connects just fine to the inverter, but obviously there is noch EM data, only inverter data. Do I have to edit MQTT_Data to see EM data (either for SBFspot or SBFspot_qt)?
EDIT: I startet your (old from yesterday = with EM data integrated) SBFspot and the original one with -d5 as a parameter. I can see a difference in the first packet that is sent to the inverter. But I have no idea if this is the expected behaviour. In the end the output of your version is:
select() returned 0
Timeout reading socket
No data!
whereas the original prints
select() returned 1
What kind of errors did you experience with cmake? Then i could fix it.
SBFspot_qt is very simple and does not regard most command line parameters. Only the MQTT stuff is relevant. So, your command is fine.
Ok, do more and more suspect your network setup. Is your network multicast capable? Any firewalls enabled? I would expect somthing like this:
...
MQTT_ItemFormat=MSGPACK
### End of Config ###
Received datagram. size: 600
RECEIVED EMETER PACKET time 0x0000000110a1e4ca
Received datagram. size: 600
RECEIVED EMETER PACKET time 0x0000000110a1e996
Received datagram. size: 600
RECEIVED EMETER PACKET time 0x0000000110a1ed30
Received datagram. size: 600
RECEIVED EMETER PACKET time 0x0000000110a1f0c9
Received datagram. size: 600
RECEIVED EMETER PACKET time 0x0000000110a1f463
...
So, is the original SBFspot not working either?
My new qt based version just listens on the well-known multicast address of SMA devices. So, you should at least receive some packets. MQTT processing is the next step.
Progress... first of all I am really sorry, but I totally forgot that I have problems with receiving data from the EM. The EM is located in the cellar, and the only way to have a network connection there is a powerline switch. Although it is a cheap and "dumb" switch, it seems to filter out multicast data. I remembered this today and connected a very long network cable through the whole house to my internet router. Now I get data with SBFspot_qt and with SBFspot from yesterday.
But the EM data from yesterday's SBFspot is strange. MQTT Explorer shows only
{MSGPACK,MSGPACK,MSGPACK,MSGPACK,MSGPACK,MSGPACK,MSGPACK,MSGPACK,MSGPACK,MSGPACK,MSGPACK,MSGPACK,MSGPACK,MSGPACK,MSGPACK,MSGPACK,MSGPACK,MSGPACK}
SBFspot_qt from today shows meaningful data, at least the msgpack node in node red can decode it. What data is it? The whole data from EM or just one value?
There is still no connection to the inverter with your version from yesterday.
What kind of errors did you experience with cmake?
CMake Error at thirdparty/qmqtt/CMakeLists.txt:40 (find_package):
Could not find a package configuration file provided by "Qt5" (requested
version 5.3.0) with any of the following names:
Qt5Config.cmake
qt5-config.cmake
Add the installation prefix of "Qt5" to CMAKE_PREFIX_PATH or set "Qt5_DIR"
to a directory containing one of the above files. If "Qt5" provides a
separate development package or SDK, be sure it has been installed.
Ok, the cmake issue is expected as long you do not have qt installed. Cmake won't do that for you.
SBFspot_qt does not yet implement other features like importing data from inverter. Currently, it only reads data from EnergyMeter. I will pull in feature by feature into the new implementation. So, SBFspot_qt is your friend and its feature list will grow in the next couple of days. I hope your are still willing to beta-test this... ;)
As of now, i only export the total AC power from the EnergyMeter. Do you need other fields?
My MsgPack implementation exports a key/value map. Power is exported with key 12
. This is the list of my keys for now:
// Static properties
Version = 0, // Protocol version: uint (max 15)
Name = 1, // Device name: string (max length 23)
StartOfProduction = 2, // Timestamp when this inverter got installed: uint32
Latitude = 3, // int: degrees/180*INT_MAX, float: value in degrees
Longitude = 4, // fint: degrees/180*INT_MAX, float: value in degrees
PowerMax = 5, // rated power: any number format
DeviceType = 6, // EnergyMeter, SolarInverter
// Dynamic properties
Timestamp = 8, // Timestamp for this data set: uint32
Interval = 9, // Interval (in seconds) for this data set.
EnergyTotal = 10, // Total yield in Wh: any number format
EnergyToday = 11, // Today's yield in Wh: any number format
Power = 12, // Current power: any number format
PowerMaxToday = 13, // Today's maximum power: any number format
// Key for DC string or AC phase properties (stored in array of maps)
Strings = 16, // Data per PV array or AC phase: map (max length 15)
I have to solve my multicast problem somehow. Any advice on that? Usually I would expect cheap powerline-switches not to filter anything, but obviously my switch does filter multicast packets.
Is there a way to define the intervall in which SBFspot_qt is fetching and delivering data? How would I daemonize it?
As for the EM data: Of course I would take all the data that EM provides. Same goes for the inverter data. You never know when you need specific data, and having is always better than needing ;-)
Thanks for your patience and effort, I am very willing to test!
I am not sure about the data SBFspot_qt delivers. This is what my payload looks like in Node Red after passing the msgpack node:
How do I have to interpret these values?
In general I am mostly interested in the following data (see the official SMA dodumentation https://www.sma.de/fileadmin/content/global/Partner/Documents/SMA_Labs/EMETER-Protokoll-TI-en-10.pdf and https://www.sma.de/fileadmin/content/global/Partner/Documents/SMA_Labs/EMETER-Protokoll-TI-de-10.pdf):
Personally I am not interested in Reactive power and Apparent power, Power factor etc.
Again personally I would not need a way to save EM data into a database, but saving the interter data into a database (MySQL) is a must for me (which works in the original SBFspot at the moment).
Hmm, no idea about this multicast issue. I got a 15 year old DLINK switch, which works correctly.
The interval and daemonizing is in the works.
The MessagePack Serializer send a key-value map (see above for the relevant keys). The power is provided using key 12. So, your measured power is actually -70. Does this sound plausible to you?
No experience from my side with NodeRed. But, it seems it is decoding correctly.
I will add the after values shortly. Currently, i am working on getting the inverter data.
The old router seems to do the job. At least I receive data. Still not sure about the meaning of the data (-70 means what?). So having data separately for consume and supply (at least sums, or even better per phase) would be helpful to understand the data (and compare it to data coming from the official SMA app).
Thanks again for your effort!
This -70 means, that there are -70 Watts measured with your EnergyMeter. This means, that 70 Watts are exported to the Grid. Does this sound plausible to you? The actual consumption and production has to be computed respecting all the devices (EMeters AND inverters/generators). Exporting inverter data is ongoing.
I added per phase data: power, current and voltage. Hopefully, you can interpret the data. Please update from latest sources. :)
The actual consumption and production has to be computed respecting all the devices
I personally would do this in my environment in Node Red. As long as I get all data from EM and inverter, this would be now problem. So from EM the data I mentioned in my post April 1st, and from the inverter all the data that SBFspot gathers.
During compiling I get an error:
[ 47%] Building CXX object CMakeFiles/SBFspot_qt.dir/sma/SmaManager.cpp.o
/home/pi/sbfspot_20210407/SBFspot/SBFspot/sma/SmaManager.cpp: In constructor ‘sma::SmaManager::SmaManager(Config&)’:
/home/pi/sbfspot_20210407/SBFspot/SBFspot/sma/SmaManager.cpp:58:17: error: ‘class QTimer’ has no member named ‘callOnTimeout’
m_liveTimer.callOnTimeout(this, &SmaManager::onLiveTimeout);
@rucksman Another update: I backported to an earlier version of Qt (i use 5.12). So, it should compile now. Also inverter data is now exported via MQTT (only AC Power, Current and Voltage). I will extend and tweak this a little bit in the next days.
You now have to append the -mqtt flag at command line to actually to the MQTT export.
Compiling errors again:
[ 55%] Building CXX object CMakeFiles/SBFspot.dir/Inverter.cpp.o
/home/pi/sbfspot_20210409/SBFspot/SBFspot/Inverter.cpp: In member function ‘E_SBFSPOT Inverter::logonSMAInverter(const std::vector<InverterData>&, long int, const char*)’:
/home/pi/sbfspot_20210409/SBFspot/SBFspot/Inverter.cpp:155:32: error: ‘inverter’ was not declared in this scope
for (uint32_t i=0; inverter!=NULL && i<MAX_INVERTERS; i++)
^~~~~~~~
/home/pi/sbfspot_20210409/SBFspot/SBFspot/Inverter.cpp:155:32: note: suggested alternative: ‘Inverter’
for (uint32_t i=0; inverter!=NULL && i<MAX_INVERTERS; i++)
^~~~~~~~
Inverter
/home/pi/sbfspot_20210409/SBFspot/SBFspot/Inverter.cpp:167:34: error: ‘getInverterIndexByAddress’ was not declared in this scope
int ii = getInverterIndexByAddress(inverters, CommBuf + 4);
^~~~~~~~~~~~~~~~~~~~~~~~~
/home/pi/sbfspot_20210409/SBFspot/SBFspot/Inverter.cpp:170:74: error: assignment of member ‘InverterData::SUSyID’ in read-only object
inverters[ii].SUSyID = get_short(pcktBuf + 15);
^
/home/pi/sbfspot_20210409/SBFspot/SBFspot/Inverter.cpp:171:73: error: assignment of member ‘InverterData::Serial’ in read-only object
inverters[ii].Serial = get_long(pcktBuf + 17);
^
/home/pi/sbfspot_20210409/SBFspot/SBFspot/Inverter.cpp: In member function ‘int Inverter::logOn()’:
/home/pi/sbfspot_20210409/SBFspot/SBFspot/Inverter.cpp:945:86: error: no matching function for call to ‘bthInitConnection(const char [18], std::vector<InverterData>&, const int&)’
rc = bthInitConnection(m_config.BT_Address, m_inverters, m_config.MIS_Enabled);
^
In file included from /home/pi/sbfspot_20210409/SBFspot/SBFspot/Inverter.cpp:45:
/home/pi/sbfspot_20210409/SBFspot/SBFspot/bluetooth.h:90:11: note: candidate: ‘E_SBFSPOT bthInitConnection(InverterData*)’
E_SBFSPOT bthInitConnection(InverterData *invData);
^~~~~~~~~~~~~~~~~
/home/pi/sbfspot_20210409/SBFspot/SBFspot/bluetooth.h:90:11: note: candidate expects 1 argument, 3 provided
/home/pi/sbfspot_20210409/SBFspot/SBFspot/bluetooth.h:91:11: note: candidate: ‘E_SBFSPOT bthInitConnection(const char*, InverterData**, int)’
E_SBFSPOT bthInitConnection(const char *BTAddress, InverterData *inverters[], int MIS);
^~~~~~~~~~~~~~~~~
/home/pi/sbfspot_20210409/SBFspot/SBFspot/bluetooth.h:91:11: note: no known conversion for argument 2 from ‘std::vector<InverterData>’ to ‘InverterData**’
/home/pi/sbfspot_20210409/SBFspot/SBFspot/Inverter.cpp:954:49: error: cannot convert ‘__gnu_cxx::__alloc_traits<std::allocator<InverterData>, InverterData>::value_type’ {aka ‘InverterData’} to ‘InverterData*’
rc = bthGetSignalStrength(m_inverters[0]);
^
In file included from /home/pi/sbfspot_20210409/SBFspot/SBFspot/Inverter.cpp:45:
/home/pi/sbfspot_20210409/SBFspot/SBFspot/bluetooth.h:94:40: note: initializing argument 1 of ‘int bthGetSignalStrength(InverterData*)’
int bthGetSignalStrength(InverterData *invData);
~~~~~~~~~~~~~~^~~~~~~
make[2]: *** [CMakeFiles/SBFspot.dir/build.make:167: CMakeFiles/SBFspot.dir/Inverter.cpp.o] Fehler 1
make[1]: *** [CMakeFiles/Makefile2:177: CMakeFiles/SBFspot.dir/all] Fehler 2
make: *** [Makefile:152: all] Fehler 2
Ok, I had to fix the linux/bluetooth build. Since MacOSX is my main development platform, i forget to update the linux build from time to time....
Ok, you could try again now. :) Thanks!
Compiling did work.
./SBFspot_qt -finq -nocsv -nosql -v5 -d5 -mqtt
There is no output on the commandline. My Node Red payload looks like this:
./SBFspot -finq -nocsv -nosql -v5 -d5 -mqtt
On the commandline the usual SBFspot output appears. Iin MQTT Explorer I get (skipped the unineresting part):
"EToday": 12.772,
"ETotal": 11182.975,
"PACTot": 0,
"UDC1": 0,
"UDC2": 0,
"IDC1": 0,
"IDC2": 0,
"PDC1": 0,
"PDC2": 0
I guess that PACTot, UDC, IDC and PDC* are 0 because it is dark outside?
Should SBFspot deliver EM data already, or do I have to use SBFspot_qt until further notice?
This looks quite good. However, voltage (key: 15) seems to be wrong by a factor of 100 on your side. I will investigate that.
Yes, SBFspot_qt does not yet output much to command line.
Be aware, that your inverter data gets published to a different topic. The scheme is "sbfspot_<serial>/live". So, emeter and inverters publish to different topics.
SBFspot won't deliver EM data any time soon. I will entirely focus on SBFspot_qt and port all the features of the original SBFspot to SBFspot_qt. When SBFspot_qt reaches the point when it can be a drop-in replacement for the original SBFspot, I will try to release that via the original repo.
So I can totally ignore SBFspot and only use SBFspot_qt?
I receive MQTT data in topic "sbfspot_serial" (json) from the inverter and in topic "sbfspot_serial/live" (msgpack) from EM, correct?
I noticed the following output on the commandline:
Found inverter at: QHostAddress("192.168.101.162")
Found inverter at: QHostAddress("192.168.101.163")
Inverter at QHostAddress("192.168.101.162") has serial: XXX
Live timeout: 1618062540
Two questions:
Yes, SBFspot_qt will be the only application, where I can offer support and development. Once SBFspot_qt is in a proper state, we can talk about upstreaming it to the original SBFspot repo.
For now, I only provide the data found under sbfspot_
Hmm, that's interesting, that the Home Manager is also found. Does the home manager also need a password to login?
The "live timout" is printed whenever the poll interval of the inverters is reached. Maybe timeout is not a good wording. I will change that.
When I find the time, I will create a table of all ported features of SBFspot and then you can track progress a little bit better.
However, are you happy with the MQTT data (msgpack) already?
For now, I only provide the data found under sbfspot_/live (.../config and others to follow) using msg pack serialisation. The original JSON serialisation will follow, but not any time soon and with low prio.
Thats perfectly fine for me at the moment as long a all necessary data is transmitted as Node Red does the msgpack decoding for me.
Does the home manager also need a password to login?
No idea, it has no web interface. From SMA's website:
monitors all energy flows in the home, automatically identifies potential savings and facilitates efficient use of solar energy.
So I guess it only transmits data to SMA's Sunny Portal and deals with data from the various components (eg switches on the washing machine which is connected with an SMA Bluetooth socket when enough solar power is produced).
However, are you happy with the MQTT data (msgpack) already?
In general yes. Although I would love to have all the data I mentioned somewhere above. At the moment only one value for power is delivered, but actually there are more (I am referring to the OBIS numbers 1:1.4.0, 1:1.8.0, 1:2.4.0, 1:2.8.0, 1:21.4.0, 1:21.8.0, 1:41.4.0 1:41.8.0, 1:61.4.0 1:61.8.0, 1:22.4.0 1:22.8.0, 1:42.4.0 1:42.8.0, 1:62.4.0 1:62.8.0 which can be found in the buffer). Here is some actual data which is created by a PHP script I found here und modified a bit to show you what I mean (not all of the OBIS numbered that I just mentioned are shown, it should just give you a hint what I am talking about):
Ok, i got it. However, i do compute the power and sum the import/export. So, positive numbers for power mean grid import, negative numbers mean grid export: If obisType == 4: case 1: signed_power_total += value; break; case 2: signed_power_total -= value; break; case 21: signed_power_l1 += value; break; case 22: signed_power_l1 -= value; break; case 41: signed_power_l2 += value; break; case 42: signed_power_l2 -= value; break; case 61: signed_power_l3 += value; break; case 62: signed_power_l3 -= value; break;
Does this serve your needs?
What is actually missing, is energy import/export. It makes actually sense to separate the import/export here. But regarding the power, i do not see the need, do you?
Would you also need the energy import/export per phase? This would actaully break a little bit with my internal API. Also for this, i considered, the sum of all phases would be enough.
I am afraid I did not get your point: What is the difference between "energy import/export" and "power import/export"?
For me personally the minimum of data would be the actual sum of power import (to stick with your terminology) which is OBIS 0140 and the actual sum of power import which is OBIS 0240. Meter readings (OBIS 0180 and OBIS 0280) are not that important. Voltage values are even less important. Per phase import and export (OBIS 2140 and OBIS 2240) would be more important for me, but if that breaks your internal API, then I do have to do without it. In the end it is your project and you decide what you implement.
Well, power is measured in Watts. It is the rate of doing work (at a single point in time) and can be either positive OR negative. Not both. If 0140 is positive 0240 will be zero and if 0240 is positive 0140 is zero. So, OBIS value 1:1.4.0 and 1:2.4.0 can be summed and provided as a single value (negative value means grid export). I now also provide power, current and voltage per phase.
Now, to the energy (which is measured in kWh (or Ws, or Wh)). It is the integral of power over time and provided via OBIS 0180 and OBIS 0280. These values cannot be provided as single value since they grow in their respective direction over time (but not in a single point in time). For these values i do not provide per phase information, because i consider it less relevant.
So, this is what you should receive now via MQTT: acVoltage[0] acVoltage[1] acVoltage[2] acCurrent[0] acCurrent[1] acCurrent[2] acPower[0] [W] (= acVoltage[0] acCurrent[0]) acPower[1] [W] (= acVoltage[1] acCurrent[1]) acPower[2] [W] (= acVoltage[2] * acCurrent[2]) acPowerTotal [W] (= acPower[0] + acPower[1] + acPower[2])
energyExportTotal [Wh] (= acPowerTotal(negative part OBIS 0240) time) energyImportTotal [Wh] (= acPowerTotal(positive part OBIS 0140) time)
Understood. Reasons for my concerns regarding sum values are (at least for me) difficulties with interpreting. A real example: Sum value 0140: 4,6W Sum value 0240: 0W Values 0140 per phase: 0 / 7,4 / 26,2 Values 0240 per phase: 29 / 0 / 0
While the sum values may be correct (0+7,4+26,2-29=4,6), they do not reflect the real situation.
Regarding energy (which I called meter readings): Even the sum values are less reelvant, at least for me.
I will compile the actual version and give feedback.
Compilation error:
[ 36%] Building CXX object CMakeFiles/SBFspot_qt.dir/SBFspot.cpp.o
/home/pi/sbfspot_20210414/SBFspot/SBFspot/SBFspot.cpp: In member function ‘const std::vector<unsigned char>& SbfSpot::encodeLoginRequest(uint16_t, uint32_t, SmaUserGroup, const string&)’:
/home/pi/sbfspot_20210414/SBFspot/SBFspot/SBFspot.cpp:177:31: error: ‘buffer’ was not declared in this scope
writePacketHeader(buffer, 0x01, addr_unknown);
^~~~~~
/home/pi/sbfspot_20210414/SBFspot/SBFspot/SBFspot.cpp:177:31: note: suggested alternative: ‘Buffer’
writePacketHeader(buffer, 0x01, addr_unknown);
^~~~~~
Buffer
/home/pi/sbfspot_20210414/SBFspot/SBFspot/SBFspot.cpp:177:13: error: ‘writePacketHeader’ was not declared in this scope
writePacketHeader(buffer, 0x01, addr_unknown);
^~~~~~~~~~~~~~~~~
/home/pi/sbfspot_20210414/SBFspot/SBFspot/SBFspot.cpp:177:13: note: suggested alternative: ‘PacketHeader’
writePacketHeader(buffer, 0x01, addr_unknown);
^~~~~~~~~~~~~~~~~
PacketHeader
/home/pi/sbfspot_20210414/SBFspot/SBFspot/SBFspot.cpp:178:13: error: ‘writePacket’ was not declared in this scope
writePacket(buffer, 0x0E, 0xA0, 0x0100, anySUSyID, anySerial);
^~~~~~~~~~~
/home/pi/sbfspot_20210414/SBFspot/SBFspot/SBFspot.cpp:178:13: note: suggested alternative: ‘ethPacket’
writePacket(buffer, 0x0E, 0xA0, 0x0100, anySUSyID, anySerial);
^~~~~~~~~~~
ethPacket
/home/pi/sbfspot_20210414/SBFspot/SBFspot/SBFspot.cpp:179:13: error: ‘writeLong’ was not declared in this scope
writeLong(buffer, 0xFFFD040C);
^~~~~~~~~
/home/pi/sbfspot_20210414/SBFspot/SBFspot/SBFspot.cpp:179:13: note: suggested alternative: ‘write’
writeLong(buffer, 0xFFFD040C);
^~~~~~~~~
write
/home/pi/sbfspot_20210414/SBFspot/SBFspot/SBFspot.cpp:184:13: error: ‘writeArray’ was not declared in this scope
writeArray(buffer, pw, sizeof(pw));
^~~~~~~~~~
/home/pi/sbfspot_20210414/SBFspot/SBFspot/SBFspot.cpp:184:13: note: suggested alternative: ‘write’
writeArray(buffer, pw, sizeof(pw));
^~~~~~~~~~
write
/home/pi/sbfspot_20210414/SBFspot/SBFspot/SBFspot.cpp:185:13: error: ‘writePacketTrailer’ was not declared in this scope
writePacketTrailer(buffer);
^~~~~~~~~~~~~~~~~~
/home/pi/sbfspot_20210414/SBFspot/SBFspot/SBFspot.cpp:185:13: note: suggested alternative: ‘PacketHeader’
writePacketTrailer(buffer);
^~~~~~~~~~~~~~~~~~
PacketHeader
/home/pi/sbfspot_20210414/SBFspot/SBFspot/SBFspot.cpp:186:13: error: ‘writePacketLength’ was not declared in this scope
writePacketLength(buffer);
^~~~~~~~~~~~~~~~~
/home/pi/sbfspot_20210414/SBFspot/SBFspot/SBFspot.cpp:188:28: error: ‘buffer’ was not declared in this scope
while (!isCrcValid(buffer[packetposition-3], buffer[packetposition-2]));
^~~~~~
/home/pi/sbfspot_20210414/SBFspot/SBFspot/SBFspot.cpp:188:28: note: suggested alternative: ‘Buffer’
while (!isCrcValid(buffer[packetposition-3], buffer[packetposition-2]));
^~~~~~
Buffer
/home/pi/sbfspot_20210414/SBFspot/SBFspot/SBFspot.cpp:188:17: error: ‘isCrcValid’ was not declared in this scope
while (!isCrcValid(buffer[packetposition-3], buffer[packetposition-2]));
^~~~~~~~~~
make[2]: *** [CMakeFiles/SBFspot_qt.dir/build.make:206: CMakeFiles/SBFspot_qt.dir/SBFspot.cpp.o] Fehler 1
make[1]: *** [CMakeFiles/Makefile2:150: CMakeFiles/SBFspot_qt.dir/all] Fehler 2
make: *** [Makefile:152: all] Fehler 2
Damn, again Bluetooth. I disabled it for now.
Now it should compile. When you add "-v1" as parameter, you should see the exact OBIS data printed to console. I guess, that's a good place to verify. Please note, that now the MQTT_ItemFormat in the .cfg file is respected. So, you should set it to MSGPACK if not already done. Also update the git submodules :)
Thanks again for testing.
Compiled without problems.
Payload in Node Red looks like this:
Can you explain which elements of the objects represent which values? And inverter data is still not available through SBFspot_qt, right? At least I do not see anything else in MQTT Explorer ...
Regarding the key values, please have a look at #3 . Inverter data should be available. They will be auto-discovered.
$ ./SBFspot_qt -mqtt -v1
SBFspot V3.8.0
Yet another tool to read power production of SMA solar inverters
(c) 2012-2021, SBF (https://github.com/SBFspot/SBFspot)
Compiled for Linux (LE) 64 bit with SQLite support
Commandline Args: -mqtt
16:17:27.959 SmaManager.cpp:97 INFO| Found inverter at: 192.168.0.9
16:17:27.969 SmaManager.cpp:97 INFO| Found inverter at: 192.168.0.97
16:17:28.000 SmaInverter.cpp:175 INFO| Inverter 192.168.0.9, serial: 1980417100
16:17:28.000 SmaInverter.cpp:175 INFO| Inverter 192.168.0.97, serial: 3006932172
16:17:29.579 MqttExporter_qt.cpp:90 1| Publishing topic: sbfspot_1900042748/live, payload size: 82 bytes
16:17:30.500 MqttExporter_qt.cpp:90 1| Publishing topic: sbfspot_1900042748/live, payload size: 82 bytes
16:17:30.882 SmaManager.cpp:126 INFO| Polling inverters, timestamp: 1618496250
16:17:31.729 MqttExporter_qt.cpp:90 1| Publishing topic: sbfspot_1900042748/live, payload size: 82 bytes
16:17:31.884 SmaManager.cpp:144 1| Polling timed out
16:17:31.885 MqttExporter_qt.cpp:90 1| Publishing topic: sbfspot_1980417100/live, payload size: 103 bytes
16:17:31.885 MqttExporter_qt.cpp:90 1| Publishing topic: sbfspot_3006932172/live, payload size: 103 bytes
16:17:32.650 MqttExporter_qt.cpp:90 1| Publishing topic: sbfspot_1900042748/live, payload size: 82 bytes
Passing -v2 will also print the OBIS values.
Well, there is no inverter data, or am I missing something?
pi@raspberrypi:~/sbfspot_20210415_1/SBFspot/SBFspot/build $ ./SBFspot_qt -finq -nocsv -nosql -v1 -d5 -mqtt
time file:line v|
00:16:28.935 loguru.cpp:647 1| arguments: ./SBFspot_qt -finq -nocsv -nosql -v1 -d5 -mqtt
00:16:28.936 loguru.cpp:650 1| Current dir: /home/pi/sbfspot_20210415_1/SBFspot/SBFspot/build
00:16:28.936 loguru.cpp:652 1| stderr verbosity: 1
00:16:28.936 loguru.cpp:653 1| -----------------------------------
SBFspot V3.8.0
Yet another tool to read power production of SMA solar inverters
(c) 2012-2021, SBF (https://github.com/SBFspot/SBFspot)
Compiled for Linux (LE) 32 bit with SQLite support
Commandline Args: -finq -nocsv -nosql -d5 -mqtt
00:16:29.283 SmaManager.cpp:97 INFO| Found inverter at: 192.168.101.162
00:16:29.291 SmaManager.cpp:97 INFO| Found inverter at: 192.168.101.163
00:16:29.303 SmaInverter.cpp:175 INFO| Inverter 192.168.101.162, serial: XXX
There is nothing transmitted via MQTT with this most recent version, not even EM data. The version I compiled thursday morning still delivers data via MQTT, but not this version I compiled thursday evening.
Is this the end of your log? That's strange indeed.
Can you start with -v2 and send me the logs? Please also check your SBFspot.cfg.
Not the end of logs, but almost. This is what the relevant part looks like with -v1:
12:27:53.964 SmaManager.cpp:97 INFO| Found inverter at: 192.168.101.163
12:27:53.968 SmaManager.cpp:97 INFO| Found inverter at: 192.168.101.162
12:27:53.995 SmaInverter.cpp:175 INFO| Inverter 192.168.101.162, serial: XXX
12:28:00.584 SmaManager.cpp:126 INFO| Polling inverters, timestamp: 1618568880
12:28:01.534 SmaManager.cpp:144 1| Polling timed out
12:29:00.485 SmaManager.cpp:126 INFO| Polling inverters, timestamp: 1618568940
12:29:01.485 SmaManager.cpp:144 1| Polling timed out
12:30:00.486 SmaManager.cpp:126 INFO| Polling inverters, timestamp: 1618569000
12:30:01.485 SmaManager.cpp:144 1| Polling timed out
Result with -v2 is as expected, but still no MQTT traffic:
pi@raspberrypi:~/sbfspot_20210415_1/SBFspot/SBFspot/build $ ./SBFspot_qt -mqtt -v2
time file:line v|
12:21:17.259 loguru.cpp:647 1| arguments: ./SBFspot_qt -mqtt -v2
12:21:17.259 loguru.cpp:650 1| Current dir: /home/pi/sbfspot_20210415_1/SBFspot/SBFspot/build
12:21:17.259 loguru.cpp:652 1| stderr verbosity: 2
12:21:17.260 loguru.cpp:653 1| -----------------------------------
SBFspot V3.8.0
Yet another tool to read power production of SMA solar inverters
(c) 2012-2021, SBF (https://github.com/SBFspot/SBFspot)
Compiled for Linux (LE) 32 bit with SQLite support
Commandline Args: -mqtt
12:21:17.720 Ethernet_qt.cpp:72 2| Discard datagram from localhost
12:21:17.766 Ethernet_qt.cpp:77 2| Received discovery response datagram. size: 60
12:21:17.766 SmaManager.cpp:97 INFO| Found inverter at: 192.168.101.162
12:21:17.767 Ethernet_qt.cpp:56 2| Send datagram: 53...
12:21:17.777 Ethernet_qt.cpp:77 2| Received discovery response datagram. size: 60
12:21:17.777 SmaManager.cpp:97 INFO| Found inverter at: 192.168.101.163
12:21:17.777 Ethernet_qt.cpp:56 2| Send datagram: 53...
12:21:17.788 Ethernet_qt.cpp:82 2| Received datagram from: 192.168.101.162, size: 98 bytes
12:21:17.788 SmaInverter.cpp:175 INFO| Inverter 192.168.101.162, serial: XXX
12:21:18.577 SmaEnergyMeter.cpp:113 2| OBIS 0:1.4.0: 0
12:21:18.577 SmaEnergyMeter.cpp:113 2| OBIS 0:1.8.0: 15102857520
12:21:18.578 SmaEnergyMeter.cpp:113 2| OBIS 0:2.4.0: 6530
12:21:18.578 SmaEnergyMeter.cpp:113 2| OBIS 0:2.8.0: 24034861080
12:21:18.578 SmaEnergyMeter.cpp:113 2| OBIS 0:21.4.0: 0
12:21:18.578 SmaEnergyMeter.cpp:113 2| OBIS 0:21.8.0: 7041951720
12:21:18.578 SmaEnergyMeter.cpp:113 2| OBIS 0:22.4.0: 6873
12:21:18.578 SmaEnergyMeter.cpp:113 2| OBIS 0:22.8.0: 31670308440
12:21:18.578 SmaEnergyMeter.cpp:113 2| OBIS 0:31.4.0: 3186
12:21:18.578 SmaEnergyMeter.cpp:113 2| OBIS 0:32.4.0: 230146
12:21:18.578 SmaEnergyMeter.cpp:113 2| OBIS 0:41.4.0: 49
12:21:18.579 SmaEnergyMeter.cpp:113 2| OBIS 0:41.8.0: 7588043640
12:21:18.579 SmaEnergyMeter.cpp:113 2| OBIS 0:42.4.0: 0
12:21:18.579 SmaEnergyMeter.cpp:113 2| OBIS 0:42.8.0: 0
12:21:18.579 SmaEnergyMeter.cpp:113 2| OBIS 0:51.4.0: 49
12:21:18.579 SmaEnergyMeter.cpp:113 2| OBIS 0:52.4.0: 231474
12:21:18.579 SmaEnergyMeter.cpp:113 2| OBIS 0:61.4.0: 294
12:21:18.579 SmaEnergyMeter.cpp:113 2| OBIS 0:61.8.0: 8108308800
12:21:18.579 SmaEnergyMeter.cpp:113 2| OBIS 0:62.4.0: 0
12:21:18.579 SmaEnergyMeter.cpp:113 2| OBIS 0:62.8.0: 0
12:21:18.579 SmaEnergyMeter.cpp:113 2| OBIS 0:71.4.0: 181
12:21:18.580 SmaEnergyMeter.cpp:113 2| OBIS 0:72.4.0: 234832
12:21:18.721 Ethernet_qt.cpp:72 2| Discard datagram from localhost
12:21:18.748 Ethernet_qt.cpp:77 2| Received discovery response datagram. size: 60
12:21:18.750 Ethernet_qt.cpp:77 2| Received discovery response datagram. size: 60
12:21:19.579 SmaEnergyMeter.cpp:113 2| OBIS 0:1.4.0: 0
12:21:19.579 SmaEnergyMeter.cpp:113 2| OBIS 0:1.8.0: 15102857520
12:21:19.579 SmaEnergyMeter.cpp:113 2| OBIS 0:2.4.0: 6526
12:21:19.579 SmaEnergyMeter.cpp:113 2| OBIS 0:2.8.0: 24034861800
12:21:19.579 SmaEnergyMeter.cpp:113 2| OBIS 0:21.4.0: 0
12:21:19.579 SmaEnergyMeter.cpp:113 2| OBIS 0:21.8.0: 7041951720
12:21:19.579 SmaEnergyMeter.cpp:113 2| OBIS 0:22.4.0: 6866
12:21:19.580 SmaEnergyMeter.cpp:113 2| OBIS 0:22.8.0: 31670309160
12:21:19.580 SmaEnergyMeter.cpp:113 2| OBIS 0:31.4.0: 3187
12:21:19.580 SmaEnergyMeter.cpp:113 2| OBIS 0:32.4.0: 229949
12:21:19.580 SmaEnergyMeter.cpp:113 2| OBIS 0:41.4.0: 49
12:21:19.580 SmaEnergyMeter.cpp:113 2| OBIS 0:41.8.0: 7588043640
12:21:19.580 SmaEnergyMeter.cpp:113 2| OBIS 0:42.4.0: 0
12:21:19.580 SmaEnergyMeter.cpp:113 2| OBIS 0:42.8.0: 0
12:21:19.580 SmaEnergyMeter.cpp:113 2| OBIS 0:51.4.0: 49
12:21:19.580 SmaEnergyMeter.cpp:113 2| OBIS 0:52.4.0: 231589
12:21:19.581 SmaEnergyMeter.cpp:113 2| OBIS 0:61.4.0: 291
12:21:19.581 SmaEnergyMeter.cpp:113 2| OBIS 0:61.8.0: 8108308800
12:21:19.581 SmaEnergyMeter.cpp:113 2| OBIS 0:62.4.0: 0
12:21:19.581 SmaEnergyMeter.cpp:113 2| OBIS 0:62.8.0: 0
12:21:19.581 SmaEnergyMeter.cpp:113 2| OBIS 0:71.4.0: 180
12:21:19.581 SmaEnergyMeter.cpp:113 2| OBIS 0:72.4.0: 235218
12:21:19.721 Ethernet_qt.cpp:72 2| Discard datagram from localhost
12:21:19.749 Ethernet_qt.cpp:77 2| Received discovery response datagram. size: 60
12:21:19.751 Ethernet_qt.cpp:77 2| Received discovery response datagram. size: 60
12:21:20.575 SmaEnergyMeter.cpp:113 2| OBIS 0:1.4.0: 0
12:21:20.576 SmaEnergyMeter.cpp:113 2| OBIS 0:1.8.0: 15102857520
12:21:20.576 SmaEnergyMeter.cpp:113 2| OBIS 0:2.4.0: 6531
12:21:20.576 SmaEnergyMeter.cpp:113 2| OBIS 0:2.8.0: 24034862520
12:21:20.576 SmaEnergyMeter.cpp:113 2| OBIS 0:21.4.0: 0
12:21:20.576 SmaEnergyMeter.cpp:113 2| OBIS 0:21.8.0: 7041951720
12:21:20.576 SmaEnergyMeter.cpp:113 2| OBIS 0:22.4.0: 6864
12:21:20.576 SmaEnergyMeter.cpp:113 2| OBIS 0:22.8.0: 31670309520
12:21:20.577 SmaEnergyMeter.cpp:113 2| OBIS 0:31.4.0: 3185
12:21:20.577 SmaEnergyMeter.cpp:113 2| OBIS 0:32.4.0: 229921
12:21:20.577 SmaEnergyMeter.cpp:113 2| OBIS 0:41.4.0: 45
12:21:20.577 SmaEnergyMeter.cpp:113 2| OBIS 0:41.8.0: 7588043640
12:21:20.577 SmaEnergyMeter.cpp:113 2| OBIS 0:42.4.0: 0
12:21:20.577 SmaEnergyMeter.cpp:113 2| OBIS 0:42.8.0: 0
12:21:20.577 SmaEnergyMeter.cpp:113 2| OBIS 0:51.4.0: 48
12:21:20.577 SmaEnergyMeter.cpp:113 2| OBIS 0:52.4.0: 231602
12:21:20.578 SmaEnergyMeter.cpp:113 2| OBIS 0:61.4.0: 288
12:21:20.578 SmaEnergyMeter.cpp:113 2| OBIS 0:61.8.0: 8108308800
12:21:20.578 SmaEnergyMeter.cpp:113 2| OBIS 0:62.4.0: 0
12:21:20.578 SmaEnergyMeter.cpp:113 2| OBIS 0:62.8.0: 0
12:21:20.578 SmaEnergyMeter.cpp:113 2| OBIS 0:71.4.0: 178
12:21:20.578 SmaEnergyMeter.cpp:113 2| OBIS 0:72.4.0: 235003
12:21:20.722 Ethernet_qt.cpp:72 2| Discard datagram from localhost
SBFspot.cfg is always the same, I did not change anything.
Ok, i am more and more respecting the settings in SBFspot.cfg.
Please check the following settings: MQTT_ItemFormat=MSGPACK // This enables the MQTT MSGPACK export LiveInterval=60 // This defines the interval for polling the inverters (60 means every FULL minute) ArchiveInterval=300 // This defines the interval for exporting to SQL.
All of the settings are correct. I compiled again, and for whatever reason now there is MQTT traffic.
Inverter data is not available. No data for keys 1-7, 13 and 16.
Inverter data is not available.
Well, as mentioned before: the poll takes place every FULL interval you provided in the interval setting. So, for interval 60, you have to wait until a full minute is reached (e.g. 15:22:00). Also be aware, that the inverted won't be polled at night. Did you wait, until it reached a full interval?
No data for keys 1-7, 13 and 16.
Ehm, i could have defined plenty of keys. So, what data are you expecting?
Ok, i am more and more respecting the settings in SBFspot.cfg.
respecting or rather suspecting?
Actually respecting. When i started my fork, i did not read all settings for testing purposes. Now, i am adding back functionality piece by piece. Bad process, i know. But things are moving forward 👍
Continued from https://github.com/SBFspot/SBFspot/issues/50 ...
@mincequi Had to compile cmake because your CMakeLists.txt states
cmake_minimum_required(VERSION 3.10)
In the raspbian repo only version 3.6.2 is available (I am on jessie).Probably it is a naming problem? Because looking at my installed packages I see
So there is no package "msgpack" (but libmsgpack3) and also no package "libmosquittopp" (but libmosquittopp1).