nakla / sbfspot

Scripts for building and running the docker container
18 stars 10 forks source link

No data appearing in mapped folders on Docker host system #21

Closed daserra23 closed 7 months ago

daserra23 commented 8 months ago

I am attempting to move over to docker. I am using the following docker-compose.yml that I copied from this github.

`version: '3'

services: sbfspot: image: nakla/sbfspot:latest network_mode: host volumes:

And I am using the following SBFspot.cfg located in /etc/sbfspot/ on the host system. This config I copied from my old non-docker system that is still operational and functioning.

`################################################################################

____

/ _|| _ )| | _ _ | |_

_ | | | / | ' \ / | __|

_) | |) | |\ \ |) | () | |_

|/|/|_| |_/ ._/ \/ __|

|_|

SBFspot.cfg - Configuration file for SBFspot.exe

SBFspot - Yet another tool to read power production of SMA solar inverters

(c)2012-2020, SBF (https://github.com/SBFspot/SBFspot)

DISCLAIMER:

A user of SBFspot software acknowledges that he or she is receiving this

software on an "as is" basis and the user is not relying on the accuracy

or functionality of the software for any purpose. The user further

acknowledges that any use of this software will be at his own risk

and the copyright owner accepts no responsibility whatsoever arising from

the use or application of the software.

SMA and Speedwire are registered trademarks of SMA Solar Technology AG

################################################################################

SBFspot.cfg - Created Sun 31 May 10:57:48 BST 2020 with sbfspot-config V1.1.0

SMA Inverter's Speedwire IP address

If IP_Address is not set or is 0.0.0.0 SBFspot will try to detect the speedwire inverter by broadcast

If IP_Address is set to a valid IP, SBFspot will try to connect directly to that IP without broadcast detection

Multiple IP addresses can be provided (comma separated)

IP_Address=192.168.2.10

User password (default 0000)

Password=S**3

Plantname

Plantname=M**1

OutputPath (Place to store CSV files)

Windows: C:\Users\Public\SMAdata\%Y

Linux : /home/pi/smadata/%Y

%Y %m and %d will be expanded to Year Month and Day

OutputPath=/var/sbfspot/%Y

OutputPathEvents (Place to store CSV files for events)

If omitted, OutputPath is used

OutputPathEvents=/var/sbfspot/%Y/Events

Timezone

Select the right timezone in date_time_zonespec.csv

e.g. Timezone=Europe/Brussels

Timezone=Europe/London

Position of pv-plant http://itouchmap.com/latlong.html

Example for Ukkel, Belgium

Latitude=51.51 Longitude=-0.13

Locale

Translate Entries in CSV files

Supported locales: de-DE;en-US;fr-FR;nl-NL;es-ES;it-IT

Default en-US

Locale=en-US

DateTimeFormat (default %d/%m/%Y %H:%M:%S)

For details see strftime() function

http://www.cplusplus.com/reference/clibrary/ctime/strftime/

DateTimeFormat=

DateFormat (default %d/%m/%Y)

DateFormat=%d/%m/%Y

TimeFormat (default %H:%M:%S)

TimeFormat=%H:%M:%S

DecimalPoint (comma/point default comma)

DecimalPoint=comma

Calculate Missing SpotValues

If set to 1, values not provided by inverter will be calculated

eg: Pdc1 = Idc1 * Udc1

CalculateMissingSpotValues=0

SynchTime (0-30 - 0=disabled, 1=once a day (default), 7=once a week, 30=once a month)

If set to non-zero value, the plant time is synchronised with local host time

Some inverters don't have a real-time clock

SynchTime=7

SynchTimeLow (1-120 - default 1)

SynchTimeHigh (1200-3600 - default 3600)

Plant time is adjusted to local host time when SynchTime=1 and

time difference is between SynchTimeLow and SynchTimeHigh limits

SynchTimeLow=60 SynchTimeHigh=3600

SunRSOffset

Offset to start before sunrise and end after sunset (0-3600 - default 900 seconds)

SunRSOffset=900

With CSV_* settings you can define the CSV file format

CSV_Export (default 1 = Enabled)

Enables or disables the CSV Export functionality

CSV_Export=1

CSV_ExtendedHeader (default 1 = On)

Enables or disables the SMA extended header info (8 lines)

isep=;

Version CSV1|Tool SBFspot|Linebreaks CR/LF|Delimiter semicolon|Decimalpoint comma|Precision 3

etc...

This is usefull for manual data upload to pvoutput.org

CSV_ExtendedHeader=1

CSV_Header (default 1 = On)

Enables or disables the CSV data header info (1 line)

dd/MM/yyyy HH:mm:ss;kWh;kW

This is usefull for manual data upload to pvoutput.org

If CSV_ExtendedHeader is enabled, CSV_Header is also enabled

CSV_Header=1

CSV_SaveZeroPower (default 1 = On)

When enabled, daily csv files contain all data from 00:00 to 23:55

This is usefull for manual data upload to pvoutput.org

CSV_SaveZeroPower=1

CSV_Delimiter (comma/semicolon default semicolon)

CSV_Delimiter=semicolon

CSV_Spot_TimeSource (Inverter|Computer default Inverter)

CSV_Spot_TimeSource=inverter

CSV_Spot_WebboxHeader (Default 0 = Off)

When enabled, use Webbox style header (DcMs.Watt[A];DcMs.Watt[B]...)

CSV_Spot_WebboxHeader=0

SQL_Database=/var/sbfspot/SBFspot.db

End of Config

` I also issued the above commands on the host system.

sudo chown -R 5000:5000 /etc/sbfspot

sudo chown -R 5000:5000 /var/sbfspot

However no data appears on the host system in /var/sbfspot.

I still have the original system working so I am sure that side of things is functional.

I tried opening a shell into the container with

sudo docker exec -it 04fc71d59d09 sh

but this returns the following

Error response from daemon: Container 04fc71d59d09118ce6036a6452bb90419a8010272c5e5358758d757a01e5891c is restarting, wait until the container is running

No matter how many times I try.

I imagine I have made an error in either folder mirroring or the csv save location in the config file so the container keeps restarting but it is a bit beyond me.

nakla commented 8 months ago

your directory mappings in the docker compose file are wrong. - ~/sbfspot/etc:/etc/sbfspot - ~/sbfspot/data:/var/sbfspot you map the directories /home/USER/sbfspot/etc and /home/USER/sbfspot/data on the host system into the container. In your case, it should be

- /etc/sbfspot:/etc/sbfspot - /var/sbfspot:/var/sbfspot

Next steps: Disable Database and only use csv output - if that works, try it with the database, but for the first run, set the INIT_DBvariable to 1 (that creates a new database) after that run, you can set the variable back to 0, as you already did in your docker compose file.

daserra23 commented 7 months ago

Thanks for your reply and time, very much appreciated. I'm still having issues though.

I'm using this docker-compose.yml with the modifications you suggested.

`version: '3'

services: sbfspot: image: nakla/sbfspot:latest network_mode: host volumes:

`################################################################################

____

/ _|| _ )| | _ _ | |_

_ | | | / | ' \ / | __|

_) | |) | |\ \ |) | () | |_

|/|/|_| |_/ ._/ \/ __|

|_|

#

SBFspot.cfg - Configuration file for SBFspot.exe

SBFspot - Yet another tool to read power production of SMA solar inverters

(c)2012-2022, SBF

#

DISCLAIMER:

A user of SBFspot software acknowledges that he or she is receiving this

software on an "as is" basis and the user is not relying on the accuracy

or functionality of the software for any purpose. The user further

acknowledges that any use of this software will be at his own risk

and the copyright owner accepts no responsibility whatsoever arising from

the use or application of the software.

#

SMA and Speedwire are registered trademarks of SMA Solar Technology AG

################################################################################

SMA Inverter's Bluetooth address

Windows: sbfspot -scan

Linux : hcitool scan

IMPORTANT FOR SPEEDWIRE USERS: COMMENT OUT BTADDRESS (PUT # IN FRONT)

BTAddress=00:00:00:00:00:00

IP_Address=192.168.2.10

If multiple local BT adapters are available, select the one to use

LocalBTAddress=00:00:00:00:00:00

SMA Inverter's Speedwire IP address

If IP_Address is not set or is 0.0.0.0 SBFspot will try to detect the speedwire inverter by broadcast

If IP_Address is set to a valid IP, SBFspot will try to connect directly to that IP without broadcast detection

Multiple IP addresses can be provided (comma separated)

IP_Address=0.0.0.0

User password (default 0000)

Password=Sunnyboy23

MIS_Enabled (Multi Inverter Support: Default=0 Disabled)

+------------+-------+-------------+

| #Inverters | NetID | MIS_Enabled |

+------------+-------+-------------+

| 1 | 1 | Don't Care |

+------------+-------+-------------+

| 1 | >1 | 0 |

+------------+-------+-------------+

| >1 | >1 | 1 |

+------------+-------+-------------+

MIS_Enabled=0

Plantname

Plantname=CasaGlicinia

OutputPath (Place to store CSV files)

Windows: C:\Users\Public\SMAdata\%Y

Linux : /home/pi/smadata/%Y

%Y %m and %d will be expanded to Year Month and Day

OutputPath=/var/sbfspot/%Y

OutputPathEvents (Place to store CSV files for events)

If omitted, OutputPath is used

OutputPathEvents=/var/sbfspot/%Y/Events

Position of pv-plant https://www.gps-coordinates.net/maps

Example for Ukkel, Belgium

Latitude=51.51

Longitude=-0.13

Calculate Missing SpotValues

If set to 1, values not provided by inverter will be calculated

eg: Pdc1 = Idc1 * Udc1

CalculateMissingSpotValues=0

DateTimeFormat (default %d/%m/%Y %H:%M:%S)

For details see strftime() function

http://www.cplusplus.com/reference/clibrary/ctime/strftime/

DateTimeFormat=%d/%m/%Y %H:%M:%S

DateFormat (default %d/%m/%Y)

DateFormat=%d/%m/%Y

DecimalPoint (comma/point default comma)

DecimalPoint=comma

TimeFormat (default %H:%M:%S)

TimeFormat=%H:%M:%S

SynchTime (0-30 - 0=disabled, 1=once a day (default), 7=once a week, 30=once a month)

If set to non-zero value, the plant time is synchronised with local host time

Some inverters don't have a real-time clock

SynchTime=7

SynchTimeLow (1-120 - default 1)

SynchTimeHigh (1200-3600 - default 3600)

Plant time is adjusted to local host time when SynchTime>0 and

time difference is between SynchTimeLow and SynchTimeHigh limits

SynchTimeLow=60

SynchTimeHigh=3600

SunRSOffset

Offset to start before sunrise and end after sunset (0-3600 - default 900 seconds)

SunRSOffset=900

Locale

Translate Entries in CSV files

Supported locales: de-DE;en-US;fr-FR;nl-NL;es-ES;it-IT

Default en-US

Locale=en-US

Timezone

Select the right timezone in date_time_zonespec.csv

e.g. Timezone=Europe/Brussels

Timezone=Europe/Brussels

BTConnectRetries

Number of Bluetooth Connection attempts (1-15; Default=10)

BTConnectRetries=10

###########################

CSV Export Settings

###########################

With CSV_* settings you can define the CSV file format

CSV_Export (default 1 = Enabled)

Enables or disables the CSV Export functionality

CSV_Export=1

CSV_ExtendedHeader (default 1 = On)

Enables or disables the SMA extended header info (8 lines)

isep=;

Version CSV1|Tool SBFspot|Linebreaks CR/LF|Delimiter semicolon|Decimalpoint comma|Precision 3

etc...

This is usefull for manual data upload to pvoutput.org

CSV_ExtendedHeader=1

CSV_Header (default 1 = On)

Enables or disables the CSV data header info (1 line)

dd/MM/yyyy HH:mm:ss;kWh;kW

This is usefull for manual data upload to pvoutput.org

If CSV_ExtendedHeader is enabled, CSV_Header is also enabled

CSV_Header=1

CSV_SaveZeroPower (default 1 = On)

When enabled, daily csv files contain all data from 00:00 to 23:55

This is usefull for manual data upload to pvoutput.org

CSV_SaveZeroPower=1

CSV_Delimiter (comma/semicolon default semicolon)

CSV_Delimiter=semicolon

CSV_Spot_TimeSource (Inverter|Computer default Inverter)

CSV_Spot_TimeSource=Inverter

CSV_Spot_WebboxHeader (Default 0 = Off)

When enabled, use Webbox style header (DcMs.Watt[A];DcMs.Watt[B]...)

CSV_Spot_WebboxHeader=0

###########################

SQL DB Settings

###########################

SQLite

SQL_Database (Fullpath to SQLite DB)

Windows: C:\Users\Public\SMAdata\SBFspot.db

Linux : /home/pi/smadata/SBFspot.db

SQL_Database=/var.sbfspot/SBFspot.db

MySQL

SQL_Database=SBFspot

SQL_Hostname= or

SQL_Port=3306

SQL_Username=SBFspotUser

SQL_Password=SBFspotPassword

#########################

MQTT Settings

#########################

Full path to mosquitto_pub executable

Linux: sudo apt-get install mosquitto-clients

MQTT_Publisher=/usr/bin/mosquitto_pub

Windows: Download and install mosquitto clients from https://mosquitto.org/download/

Mosquitto 64-bit on Windows 64-bit

MQTT_Publisher=%ProgramW6432%\mosquitto\mosquitto_pub.exe

Mosquitto 32-bit on Windows 32-bit

MQTT_Publisher=%ProgramFiles%\mosquitto\mosquitto_pub.exe

IP address or DNS name of MQTT Broker

Don't use test broker for production environments

MQTT_Host=test.mosquitto.org

MQTT_Host=broker.hivemq.com

Port (Optional)

MQTT_Port=

Topic Name

When using public broker like test.mosquitto.org make sure to define a unique topic name

by appending the serial number of the inverter to avoid you receive data of another SBFspot user.

MQTTTopic=sbfspot{serial}

MQTTTopic=sbfspot{plantname}/sma_{serial}

Format of message items to be sent

JSON: MQTT_ItemFormat="{key}": {value}

TEXT: MQTT_ItemFormat={key}:{value}

XML: MQTT_ItemFormat=

MQTT_ItemFormat="{key}": {value}

MQTT_ItemDelimiter (none|blank|comma|semicolon default comma)

JSON: MQTT_ItemDelimiter=comma

TEXT: MQTT_ItemDelimiter=semicolon

XML : MQTT_ItemDelimiter=none

MQTT_ItemDelimiter=comma

Mandatory arguments for mosquitto_pub executable

JSON: MQTT_PublisherArgs=-h {host} -t {topic} -m "{{message}}"

TEXT: MQTT_PublisherArgs=-h {host} -t {topic} -m "{message}"

XML : MQTT_PublisherArgs=-h {host} -t {topic} -m "{message}"

MQTT_PublisherArgs=-h {host} -t {topic} -r -m "{{message}}"

Data to be published (comma delimited)

MQTT_Data=Timestamp,SunRise,SunSet,InvSerial,InvName,InvTime,InvStatus,InvTemperature,InvGridRelay,EToday,ETotal,PACTot,UDC,IDC,PDC

Possible keywords are (if supported by your inverter):

SBFspot Alias Code Description

====================================================================================

PrgVersion SBFspot Version

Plantname Plant name

Timestamp Current date/time

InvTime Inverter date/time

SunRise Sunrise time

SunSet Sunset time

InvSerial Serial Number

InvName NameplateLocation Device name

InvClass NameplateMainModel Device class

InvType NameplateModel Device type

InvSwVer NameplatePkgRev Software package

InvStatus OperationHealth Condition

InvTemperature CoolsysTmpNom Operating condition temperatures

InvGridRelay OperationGriSwStt Grid relay/contactor

ETotal MeteringTotWhOut Total yield

EToday MeteringDyWhOut Day yield

PACTot GridMsTotW Power

PDC1/PDC2/PDC DcMsWatt DC power input string 1/2/All

UDC1/UDC2/UDC DcMsVol DC voltage input string 1/2/All

IDC1/IDC2/IDC DcMsAmp DC current input string 1/2/All

PDCTot DcMsWatt DC power all strings

OperTm MeteringTotOpTms Operating time

FeedTm MeteringTotFeedTms Feed-in time

PAC1 GridMsWphsA Power L1

PAC2 GridMsWphsB Power L2

PAC3 GridMsWphsC Power L3

UAC1 GridMsPhVphsA Grid voltage phase L1

UAC2 GridMsPhVphsB Grid voltage phase L2

UAC3 GridMsPhVphsC Grid voltage phase L3

IAC1 GridMsAphsA_1 Grid current phase L1

IAC2 GridMsAphsB_1 Grid current phase L2

IAC3 GridMsAphsC_1 Grid current phase L3

GridFreq GridMsHz Grid frequency

BTSignal BTSignal Bluetooth signal strength

BatTmpVal BatTmpVal Battery temperature

BatVol BatVol Battery voltage

BatAmp BatAmp Battery current

BatChaStt BatChaStt Current battery charge status

InvWakeupTm Inverter Wakeup date/time

InvSleepTm Inverter Sleep date/time

MeteringWOut MeteringGridMsTotWOut Power sent towards the Grid (W)

MeteringWIn MeteringGridMsTotWIn Power drawn from the Grid (W)

MeteringWTot Power sent/drawn (negative when sending towards the net)

`

SBFspot.default.cfg SBFspotUpload.default.cfg

But there is no output at all in /var/sbfspot , no log, no database , no CSV.

Which leads me to think that the folder mapping is OK. My other "non-docker" system is able to read data from the inverter without problems.

daserra23 commented 7 months ago

Apologies for font size, not sure what happened there.

daserra23 commented 7 months ago

Both /etc/sbfspot and /var/sbfspot have read/write/execute as "anyone"

nakla commented 7 months ago

First, I hope every comment line in your SBFspot.cfg file begins with a hashtag - I can't see that in your posts. I don't know how you start the container, but there have to be output messages from the container. If not, something else is wrong. If the container starts and you initialize the Database, the container should correct your wrong entry SQL_Database=/var.sbfspot/SBFspot.db and outputs that process to you.

daserra23 commented 7 months ago

Hi sorry about the formatting, I was unable to edit it in this forum and however I paste it the formatting looks awful.

The cfg file does indeed have the hashtags, it is basically cut and pasted from the default cfg with my plantname, password and options just to use CSV.

I corrected the error in the SBFspot.cfg file, thanks for pointing that out.

I start the container with "sudo docker compose up -d" and there are no errors in the ssh window at that point.

`admin@domoticz2:/opt/sbfspot $ sudo docker compose up -d

[+] Running 1/1

✔ Container sbfspot-sbfspot-1 Started `

But still no database created but I might need to wait till daytime as the inverter goes to sleep at night.

daserra23 commented 7 months ago

I just noticed that a docker-compose.yml.swp file has been created in /opt/sbfspot

nakla commented 7 months ago

"But still no database created but I might need to wait till daytime as the inverter goes to sleep at night." no "`admin@domoticz2:/opt/sbfspot $ sudo docker compose up -d" running the container in the background means no logging to the console

daserra23 commented 7 months ago

`admin@domoticz2:/opt/sbfspot $ sudo docker compose up

[+] Running 1/1

✔ Container sbfspot-sbfspot-1 Created 0.4s

Attaching to sbfspot-sbfspot-1

sbfspot-sbfspot-1 | Wrong SQL_Database value in SBFspot.cfg. I change it to "/var/sbfspot/sbfspot.db"

sbfspot-sbfspot-1 | wal

sbfspot-sbfspot-1 exited with code 0 `

I double checked in SBFspot.cfg and the line is as suggested

SQL_Database=/var/sbfspot/sbfspot.db

nakla commented 7 months ago

If you start the container again, you get the same message?

daserra23 commented 7 months ago

This time it is...

admin@domoticz2:/opt/sbfspot $ sudo docker compose up [+] Running 1/1 ✔ Container sbfspot-sbfspot-1 Created 0.3s Attaching to sbfspot-sbfspot-1 sbfspot-sbfspot-1 | Error: near line 3: table Config already exists sbfspot-sbfspot-1 | wal sbfspot-sbfspot-1 | Error: near line 9: UNIQUE constraint failed: Config.Key sbfspot-sbfspot-1 | Error: near line 11: table Inverters already exists sbfspot-sbfspot-1 | Error: near line 28: table vwInverters already exists sbfspot-sbfspot-1 | Error: near line 39: table SpotData already exists sbfspot-sbfspot-1 | Error: near line 59: table vwSpotData already exists sbfspot-sbfspot-1 | Error: near line 99: table DayData already exists sbfspot-sbfspot-1 | Error: near line 108: table vwDayData already exists sbfspot-sbfspot-1 | Error: near line 117: table MonthData already exists sbfspot-sbfspot-1 | Error: near line 125: table vwMonthData already exists sbfspot-sbfspot-1 | Error: near line 132: table EventData already exists sbfspot-sbfspot-1 | Error: near line 148: table vwEventData already exists sbfspot-sbfspot-1 | Error: near line 159: table Consumption already exists sbfspot-sbfspot-1 | Error: near line 166: table vwConsumption already exists sbfspot-sbfspot-1 | Error: near line 176: table vwAvgConsumption already exists sbfspot-sbfspot-1 | Error: near line 184: table vwAvgSpotData already exists sbfspot-sbfspot-1 | Error: near line 206: table vwPvoData already exists sbfspot-sbfspot-1 | Error: near line 232: table SpotDataX already exists sbfspot-sbfspot-1 exited with code 0

daserra23 commented 7 months ago

Probably worth closing this, I moved over to "normal" SBFspot and all is working fine. Thanks for your time anyway.

nakla commented 7 months ago

To your 2nd last text, that message indicates, that the database is created, you just need to set the INIT_DBvariable back to 0. I will add this to the docu, I thought the docu was understandable enough. To your last post: I think that is a good decision