CA-SK production parser down

[electricitymapsbot]

Description

This is an automatic error report generated for Canada Saskatchewan (CA-SK).

Issues:

• No recent data found for production parser
• No recent data found for consumption parser

Suggestions

• Try running the parser locally using the command poetry run test_parser CA-SK production
• Explore the runtime logs

You can see an overview of all parser issues here.

[kruschk]

Hi! I contributed a few times last year, and Hacktober has inspired me to get back to it (I'm a sucker for gamification :sweat_smile: ). I hope I'm still welcome to lend a hand!

I dug into this issue a bit, and it seems like the consumption URL has started returning "Data Unavailable" instead of the numeric value that (I believe) it used to. I figured this was likely just a transient thing, but then I noticed the JSON returned by the production URL contains a systemDemand key that may not have been present before, so perhaps this was an API change. If that's the case, could we use that value instead? For reference, here's the relevant SaskPower dashboard.

[electricitymapsbot]

Closing this as the electricity data is flowing again.

[kruschk]

I'm not sure why the bot closed this; the issue persists. I've noticed one other thing: the source is still reporting data from October 8, so something is likely wrong on SaskPower's end. Should I try contacting them to see if they're aware of the issue?

[VIKTORVAV99]

If the bot closed the issue we got data again (it will open a new issue if it stops working again).

If you want to reach out feel free to do so but it's nothing that's expected or required.

[kruschk]

I just thought it seemed odd because errors are still appearing in the runtime logs.

My last comment was wrong: the source is still updating because it now claims the data is "as of October 9, 2023", so I suppose it's just lagging by a few days. That said, the application shows data for October 10, so I'm not sure what's going on there. Looking at the parser logic, I don't see why this would be the case.

Despite the seemingly active production URL, the consumption URL is still responding with "Data Unavailable". Do you have any thoughts on using the SystemDemand key from the production URL's response instead?

[VIKTORVAV99]

CA-SK is a little special as the production data is only reported on a daily basis. So we also consume hourly load data in the hope that we can estimate the hourly production data.

This is the reason the zone don't show up in the hourly view on the map.

[kruschk]

What do you think of my proposed fix for the fetch_consumption function?

[kruschk]

By the way, I noticed on the Building a new parser wiki page that some new data structures have been introduced to help enforce the expected data format. Perhaps I could contribute by updating this parser to use them?

[VIKTORVAV99]

What do you think of my proposed fix for the fetch_consumption function?

Not sure this works, we specifically chose this endpoint because it updates during the day and isn't just a daily average.

[VIKTORVAV99]

By the way, I noticed on the Building a new parser wiki page that some new data structures have been introduced to help enforce the expected data format. Perhaps I could contribute by updating this parser to use them?

Feel free to do so! We plan on migrating all parsers and parser functions eventually.

[kruschk]

Not sure this works, we specifically chose this endpoint because it updates during the day and isn't just a daily average.

Oh, I see now!

Feel free to do so! We plan on migrating all parsers and parser functions eventually.

Great, I'll take a shot at it, and maybe I can work on some others afterward! I'll try to compile a list of parsers requiring migration in a new issue to help keep track. :slightly_smiling_face: