After update trying to convert database

PvdGulik commented 11 months ago

Describe the bug Updated the plugin, found some errors in the log.

2023-11-15 22:04:36.628 Error: GoodWe GW5000-DT: Call to function 'onStart' failed, exception details: 2023-11-15 22:04:36.631 Error: GoodWe GW5000-DT: Traceback (most recent call last): 2023-11-15 22:04:36.631 Error: GoodWe GW5000-DT: File "/home/pi/domoticz/plugins/domoticz-GoodWeSEMS/plugin.py", line 620, in onStart 2023-11-15 22:04:36.631 Error: GoodWe GW5000-DT: _plugin.onStart() 2023-11-15 22:04:36.631 Error: GoodWe GW5000-DT: File "/home/pi/domoticz/plugins/domoticz-GoodWeSEMS/plugin.py", line 337, in onStart 2023-11-15 22:04:36.631 Error: GoodWe GW5000-DT: self.startDeviceUpdateV2() 2023-11-15 22:04:36.631 Error: GoodWe GW5000-DT: File "/home/pi/domoticz/plugins/domoticz-GoodWeSEMS/plugin.py", line 150, in startDeviceUpdateV2 2023-11-15 22:04:36.631 Error: GoodWe GW5000-DT: self.updateDevices(DeviceData) 2023-11-15 22:04:36.631 Error: GoodWe GW5000-DT: File "/home/pi/domoticz/plugins/domoticz-GoodWeSEMS/plugin.py", line 158, in updateDevices 2023-11-15 22:04:36.631 Error: GoodWe GW5000-DT: self.createDevices(inverter["sn"]) 2023-11-15 22:04:36.631 Error: GoodWe GW5000-DT: File "/home/pi/domoticz/plugins/domoticz-GoodWeSEMS/plugin.py", line 217, in createDevices 2023-11-15 22:04:36.631 Error: GoodWe GW5000-DT: numDevs = len(Devices[serialNumber].Units) 2023-11-15 22:04:36.631 Error: GoodWe GW5000-DT: KeyError: '95000DTU193W0183'

Found out on this Github-page I have to convert the database. Followed the instructions, but on import I got a error...

To Reproduce Steps to reproduce the behavior: After convert the ID's

pi@Domoticz:~/domoticz $ sqlite3 domoticz.db SQLite version 3.34.1 2021-01-20 14:10:07 Enter ".help" for usage hints. sqlite> DROP TABLE IF EXISTS DeviceStatus; sqlite> .mode csv sqlite> .separator ";" sqlite> .import devicedata.csv ERROR: missing TABLE argument. Usage: .import FILE TABLE Import data from FILE into TABLE Options: --ascii Use \037 and \036 as column and row separators --csv Use , and \n as column and row separators --skip N Skip the first N rows of input -v "Verbose" - increase auxiliary output Notes:

If TABLE does not exist, it is created. The first row of input determines the column names.
If neither --csv or --ascii are used, the input mode is derived from the ".mode" output mode
If FILE begins with "|" then it is a command that generates the input text.

Expected behavior That import would work as supposed

Screenshots pi@Domoticz:~/domoticz $ sqlite3 domoticz.db SQLite version 3.34.1 2021-01-20 14:10:07 Enter ".help" for usage hints. sqlite> DROP TABLE IF EXISTS DeviceStatus; sqlite> .mode csv sqlite> .separator ";" sqlite> .import devicedata.csv ERROR: missing TABLE argument. Usage: .import FILE TABLE Import data from FILE into TABLE Options: --ascii Use \037 and \036 as column and row separators --csv Use , and \n as column and row separators --skip N Skip the first N rows of input -v "Verbose" - increase auxiliary output Notes:

If TABLE does not exist, it is created. The first row of input determines the column names.
If neither --csv or --ascii are used, the input mode is derived from the ".mode" output mode
If FILE begins with "|" then it is a command that generates the input text.

Domoticz environment (please complete the following information):

Domoticz V2023.2 (build 15650)
Python version '3.9.2',

Additional context Add any other context about the problem here.

JanJaapKo commented 11 months ago

Sorry missed the table name in the import. Wiki updated, please try the import steps again.

PvdGulik commented 11 months ago

Oke, that helped a bit.. but now following errors...

2023-11-15 22:43:27.834 Status: GoodWe GW5000-DT: Entering work loop. 2023-11-15 22:43:27.834 Status: GoodWe GW5000-DT: Started. 2023-11-15 22:43:27.840 Status: dzVents: Write file: /home/pi/domoticz/scripts/dzVents/generated_scripts/PVOutput Goodwe.lua 2023-11-15 22:43:30.407 Status: GoodWe GW5000-DT: Initialized version 4.0.0, author 'Jan-Jaap Kostelijk' 2023-11-15 22:43:30.432 Status: GoodWe GW5000-DT: Starting Goodwe SEMS API plugin, logging to file goodwe GoodWe GW5000-DT.log 2023-11-15 22:43:31.518 GoodWe GW5000-DT: Status of GoodWe inverter (SN: 95000DTU193W0183): '-1 offline' 2023-11-15 22:43:31.520 Error: GoodWe GW5000-DT: Update to 'UnitEx' failed to update any DeviceStatus records for key 21/95000DTU193W0183/4 2023-11-15 22:43:31.535 Error: GoodWe GW5000-DT: Call to function 'onStart' failed, exception details: 2023-11-15 22:43:31.541 Error: GoodWe GW5000-DT: Traceback (most recent call last): 2023-11-15 22:43:31.541 Error: GoodWe GW5000-DT: File "/home/pi/domoticz/plugins/domoticz-GoodWeSEMS/plugin.py", line 620, in onStart 2023-11-15 22:43:31.541 Error: GoodWe GW5000-DT: _plugin.onStart() 2023-11-15 22:43:31.541 Error: GoodWe GW5000-DT: File "/home/pi/domoticz/plugins/domoticz-GoodWeSEMS/plugin.py", line 337, in onStart 2023-11-15 22:43:31.541 Error: GoodWe GW5000-DT: self.startDeviceUpdateV2() 2023-11-15 22:43:31.542 Error: GoodWe GW5000-DT: File "/home/pi/domoticz/plugins/domoticz-GoodWeSEMS/plugin.py", line 150, in startDeviceUpdateV2 2023-11-15 22:43:31.542 Error: GoodWe GW5000-DT: self.updateDevices(DeviceData) 2023-11-15 22:43:31.542 Error: GoodWe GW5000-DT: File "/home/pi/domoticz/plugins/domoticz-GoodWeSEMS/plugin.py", line 182, in updateDevices 2023-11-15 22:43:31.542 Error: GoodWe GW5000-DT: newCounter = calculateNewEnergy(inverter["sn"], theInverter.inputPower1Unit, inputPower) 2023-11-15 22:43:31.542 Error: GoodWe GW5000-DT: File "/home/pi/domoticz/plugins/domoticz-GoodWeSEMS/plugin.py", line 605, in calculateNewEnergy 2023-11-15 22:43:31.542 Error: GoodWe GW5000-DT: lastUpdateDT = datetime.fromtimestamp(time.mktime(time.strptime(dt_string, dt_format))) 2023-11-15 22:43:31.542 Error: GoodWe GW5000-DT: File "/usr/lib/python3.9/_strptime.py", line 562, in _strptime_time 2023-11-15 22:43:31.542 Error: GoodWe GW5000-DT: tt = _strptime(data_string, format)[0] 2023-11-15 22:43:31.542 Error: GoodWe GW5000-DT: File "/usr/lib/python3.9/_strptime.py", line 349, in _strptime 2023-11-15 22:43:31.542 Error: GoodWe GW5000-DT: raise ValueError("time data %r does not match format %r" % 2023-11-15 22:43:31.542 Error: GoodWe GW5000-DT: ValueError: time data '15-11-2023 21:12' does not match format '%Y-%m-%d %H:%M:%S'

following with a lot of lines with:

2023-11-15 22:44:07.176 Error: Serious database error, problem getting ID from DeviceStatus! 2023-11-15 22:44:07.177 Error: Serious database error, problem getting ID from DeviceStatus! 2023-11-15 22:44:07.178 Error: Serious database error, problem getting ID from DeviceStatus! 2023-11-15 22:44:07.180 Error: Serious database error, problem getting ID from DeviceStatus! 2023-11-15 22:44:07.181 Error: Serious database error, problem getting ID from DeviceStatus! 2023-11-15 22:44:07.182 Error: Serious database error, problem getting ID from DeviceStatus! 2023-11-15 22:44:07.184 Error: Serious database error, problem getting ID from DeviceStatus! 2023-11-15 22:44:07.198 Error: Serious database error, problem getting ID from DeviceStatus! 2023-11-15 22:44:07.199 Error: Serious database error, problem getting ID from DeviceStatus! 2023-11-15 22:44:07.201 Error: Serious database error, problem getting ID from DeviceStatus! 2023-11-15 22:44:12.182 Error: Serious database error, problem getting ID from DeviceStatus! 2023-11-15 22:44:12.189 Error: Serious database error, problem getting ID from DeviceStatus! 2023-11-15 22:44:12.191 Error: Serious database error, problem getting ID from DeviceStatus!

PvdGulik commented 11 months ago

I restored my database with the old ID’s. But the new plugin is still installed. How to rollback to the previous plugin?

thanks!

JanJaapKo commented 11 months ago

not really sure how to properly do it, but a simple way is to download the previous versions of the files and place them on your PI (unfortunately I did make a version in GIT, that would have made this easier......)

JanJaapKo commented 11 months ago

ok, it looks like the import of the DeviceStatus was not successful. To check that the import was actually succesful, you can check, after the .import devicedata.csv DeviceStatus (so still in sqlite)

select * from DeviceStatus; Or select * from DeviceStatus where HardwareID=xx;

where you need to replace xx with the hardware number from your inverter (can be found on the Hardware page in Domoticz, in the first column under Idx). It should then show the devices for the inverter with the serial number. See below screenshot from my setup:

PvdGulik commented 11 months ago

I changed all the devices connected by hardware ID21 to the corresponding serialnr. of the GoodWe inverter. See included picture, and did the second part, importing the devicedata.csv back into the database.

After that, Domoticz does not start any more...

Tried your suggested checks, seems to be good..

After disabling the hardware Id with sudo sqlite3 -header -list domoticz.db "UPDATE Hardware set Enabled = 0 where ID = 21" Domoticz is running again, without the GoodWe hardware ;-(

Still not have a solution ..... please help

JanJaapKo commented 11 months ago

The devices seem to be present 2 times, that is not necesarily a good idea. But when it works..........

PvdGulik commented 11 months ago

The devices seem to be present 2 times, that is not necesarily a good idea. But when it works..........

No, it does not. Which ones should I remove?

JanJaapKo commented 11 months ago

Does not matter, it is only the definition, and they're equal.

PvdGulik commented 11 months ago

I'm back to the previous version. Cannot get it working...

MrHobbes74 commented 11 months ago

I just installed this plugin for the first time (no history with old devices) and I get the same error as the first post in this thread (keyerror). It seems that the creation devices when they do not yet exist is not working correctly!

PvdGulik commented 11 months ago

I'm still using the old version which still works..

JanJaapKo commented 11 months ago

OK, I see where it goes wrong for a clean install, preparing a fix. Thought I had this tested, but it apparently wasn't on a fully clean install....

I'm struggling with creating new devices (or units) in the plugin framework, awaiting advise from forum

MrHobbes74 commented 11 months ago

old version indeed works fine, which I have installed now.

JanJaapKo / domoticz-GoodWeSEMS

After update trying to convert database #25