Load initial data from cache or the Solcast API

stefanhintermayr commented 2 weeks ago

The problem

I had the Error:

„ Einrichtung fehlgeschlagen, wird erneut versucht: Failed to load initial data from cache or the Solcast API“

What version of Solcast Integration has the issue?

v4.1.3

What version of Home Assistant are you using?

2024.8.2

Diagnostics information

No response

Logs debug/info/error output

Einrichtung fehlgeschlagen, wird erneut versucht: Failed to load initial data from cache or the Solcast API

Additional information

I try, to reload. I try to deactivate and activate. I try to deinstall and install the integration. Everytimes the same error. I don‘t know with log files i have to send.

autoSteve commented 2 weeks ago

Without any logs this is going to be tricky to diagnose, but I'll try.

It is the Home Assistant log. Instructions for looking at these are included in the troubleshooting FAQ discussion. Even info level logs would be better than nothing.

Some further information that would be useful...

Are you upgrading from a prior version, or is this a new install having never run the application before?
Are your solar site(s) configured at solcast.com?
Have you read the readme?
Is this a HAOS install?

autoSteve commented 2 weeks ago

Those messages are indicating that initial setup of the integration is failing.

The sequence is:

Get the sites data from Solcast. This appears to have worked, but I do not know the sites configuration. It is likely your API key is correct.
Load saved data. If saved data does not exist then Solcast will be polled for history and forecast data. This is what is failing. The poll for forecast incorporates a retry mechanism that repeatedly tries to get data over a fifteen minute period. How long is the integration trying to set up for? Does it fail quickly, or does it take a long time?

stefanhintermayr commented 2 weeks ago

Are you upgrading from a previous version or is this a fresh install where the application has never been run?

I noticed it by accident as I don't have it in production yet, so the entities aren't controlling anything yet. Just FYI.

So I don't know exactly when it happened.

I did try a fresh install though.

Are your solar site(s) configured at solcast.com?

Yes, that actually worked.

Have you read the readme file?

Of course.

Is this a HAOS install?

Yes.

Get the site data from Solcast. This seems to have worked but I don't know the site configuration. Your API key is probably correct.

Yes, I checked it, and I also generated a new one and copied and pasted that as well.

Load saved data. If there is no saved data, Solcast will be queried for historical and forecast data. This is the error. The forecast query contains a retry mechanism that repeatedly tries to retrieve data over a 15 minute period. How long does it take to set up the integration? Does it fail quickly or does it take a long time?

Happens immediately, right after a restart of both the integration and HA.

If you tell me which log files you need and where I can find them, I'm happy to provide them.

autoSteve commented 2 weeks ago

Q: You asked for DEBUG logs to be provided when raising issues. How do I get these?

We can't usually do anything without them, and it's almost always the first question that will be asked: "Could you provide debug logs, please?"

When set for debug you see all kinds of fascinating stuff about what's going on under the covers. In configuration.yaml:

 logger:
   default: warn
   logs:
     custom_components.solcast_solar: debug

(Make the default info/warn/whatever, as we don't care. We just want the debug goodness.)

Reviewing logs is quite simple, but debug logs cannot be reviewed from the UI. The file homeassistant/home-assistant.log must be viewed. I find the handiest way is to use the less command from the HAOS command line to do this.

less /homeassistant/home-assistant.log then... /solcast_solar to search for and highlight lines relating to the integration &/solcast to only display lines containing 'solcast' PgUp/PgDn/Up/Down to hunt around. F key to follow live (capital F) Ctrl-C to exit follow mode

autoSteve commented 2 weeks ago

Yes, I checked it, and I also generated a new one and copied and pasted that as well.

~~This interests me. What did you copy/paste, and specifically to where? Screen shots if appropriate. (Redact any API keys, but rooftop site IDs are fine.)~~

~~Getting the sites data is an API call that the integration does, and nothing that you should have to do (apart from configuring your rooftop sites at solcast.com...~~

Sorry. You are talking about your API key. Regenerating a key will not help. Nor will other voodoo tricks.

stefanhintermayr commented 2 weeks ago

home-assistant_solcast_solar_2024-08-23T09-55-44.130Z.log Does this file help?

autoSteve commented 2 weeks ago

That helps immensely.

Thank you.

ERROR (MainThread) [custom_components.solcast_solar.solcastapi] The cached data in solcast.json is corrupt in load_saved_data()

There is a file, /config/solcast.json. Could you please examine its contents and post them here. cat /config/solcast.json

This file would appear to not be of JSON format, and that is causing the issue.

Depending on what you provide, it begs the question what should the integration really do in this situation? We fail hard right now, but should we remove this 'corrupt' file and start a fresh one?

(edit) I think we should not remove the file. Too risky given the past. Greater visibilty of a corruption in the UI is given in PR #146.

autoSteve commented 2 weeks ago

(waits patiently, with midnight on a Friday night fast approaching...)

stefanhintermayr commented 2 weeks ago

Sorry, it's Friday lunchtime here and the gardening is waiting... ;-)

Here are two files. I backed them up beforehand and then restarted the integration. And it's running again. I've attached the two files for analysis.

solocast-usage_old.json solocast_old.json

autoSteve commented 2 weeks ago

Wow.

The solcast_old.json is absolutely corrupted, and this has caused the issue. This is by no means a recent install, dating back to early June, so what happened?

What's your platform for HAOS? NUC, Pi?

stefanhintermayr commented 2 weeks ago

Hi, the Install was on a Synology with VM a HAOS, this is migrating to an Proxmox VM on a NUC by Backup and Restore on a new System.This could it be!?!?Am 23.08.2024 um 14:07 schrieb Steve Saunders @.***>: Wow. The solcast_old.json is absolutely corrupted, and this has caused the issue. This is by no means a recent install, dating back to early June, so what happened? What's your platform for HAOS? NUC, Pi?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

autoSteve commented 2 weeks ago

Closing. Not an issue with the integration, but interesting at least.

The cause of the corruption is unknown, but I am attributing it to the migration. This issue has resulted in two positive commits, so thank you.

Proxmox for the win. ❤️ I have Synology and love it, but would not go near Synology for hosting HA for quite a few reasons.

HA lives here on my three-node Proxmox cluster most happily.

BJReplay / ha-solcast-solar