jobvk / Home-Assistant-Windcentrale

Provides Home Assistant sensors for multiple wind turbines from the Windcentrale
https://jobvankoeveringe.com/projects/windcentrale/
Apache License 2.0
14 stars 3 forks source link

API Deprecated | No Data #26

Closed jobvk closed 2 years ago

jobvk commented 2 years ago

I received an email on June 28, 2022 at 09:15:04 CEST

That the app and the associated APIs will cease to exist from June 30th. This will mean that this plugin can no longer retrieve data.

I am working on updating the plugin with data from the new platform. But that won't be done before June 30. The new platform uses AWS. And the API's use now a bearer token for authorization, where they had no authorization in the past.

But I'm working on it if you have any questions in between, feel free to ask me below or via Discord. Version 0.2.0 will then be a working version with the new platform.

Email: image

Simanias commented 2 years ago

To bad they change everything, I’m very happy with the integration. If there’s anything I can help with please let me know.

gerard76 commented 2 years ago

There is an open API as discussed here https://steenderen.net/blog/winddelen-opbrengst-in-home-assistant/

jobvk commented 2 years ago

There is an open API as discussed here https://steenderen.net/blog/winddelen-opbrengst-in-home-assistant/

@gerard76 I am currently already using the API discussed there for this custom component. But according to the company De Windcentrale, this API will stop shortly. That's why I'm in the process of converting this custom component to the new platform that uses the AWS platform with bearer tokens.

gerard76 commented 2 years ago

@gerard76 I am currently already using the API discussed there for this custom component. But according to the company De Windcentrale, this API will stop shortly. That's why I'm in the process of converting this custom component to the new platform that uses the AWS platform with bearer tokens.

Ah ok, sorry, could have seen that. Thanks for putting your time and effort in it.

Simanias commented 2 years ago

@jobvk strange, for me the API is still working. I'm receiving the 'correct' information in Home Assistant......

jobvk commented 2 years ago

@jobvk strange, for me the API is still working. I'm receiving the 'correct' information in Home Assistant......

@Simanias Yeah the app stort working for me. But the API's all seem to work just fine. I will continue developing with a new version because I don't know how long these API's will work.

Simanias commented 2 years ago

The API just stopped working 2 hours ago. Waiting for your new integration for now.

gerard76 commented 2 years ago

The API just stopped working 2 hours ago. Waiting for your new integration for now.

Works for me? https://zep-api.windcentrale.nl/production/201/live

jobvk commented 2 years ago

The API just stopped working 2 hours ago. Waiting for your new integration for now.

Yes it looked like they stopped all the old API's. Unfortunately I don't have time this weekend. I'll be free next week and hope to have it fully working again.

jobvk commented 2 years ago

For some odd reason, the old APIs are working again. Unfortunately, I couldn't finish it this week. Next two weeks I will be on vacation and after that I will continue with updating the custom component.

jobvk commented 2 years ago

Time for a update on how it's going

Credentials API

The API is working. Unfortunately, the token only works for 60 minutes. Therefore, the custom component requests a new token every 55 minutes.

Live API

The API is working. The sensor Run Percentage is no longer there. The sensor Max Power is renamed to Power Percentage. The sensor kWh is renamed to Production This Year. The sensor Hours Run Total is add it will show how many hours the windturbine has been running in total.

Production History API

I haven't started this yet because I'm not quite sure how I'm going to do this yet.

There are many APIs. For example, if you look at the chart for years, you will see 3 years. With a graph for the total windturbine and one for your wind shares. These are all 2 API's, if you then want to add 3 more years, it will become 4 API's.

And that was just the years. If I add months, weeks, days and hours. Then it adds up very quickly.

I'm currently talking to @balk77 about a possible HACS Lovelace card that might have a filter and buttons on it, so you can go through it just like on the website without having to load everything at once.

If anyone else has an idea how this can be done more efficiently, I'd like to hear your suggestion.

Notifications API

The API is working, only waiting for some more notifications, so I can make sure the filter works again.

Energy Management

The production sensor that is currently used sensor.name_day_production no longer works because the API only updates after the day has already passed. So I created a new sensor and called it Energy Management. The Energy Management sensor copy's the value of the Production This Year sensor and divides it by total shares and multiply it by the number of shares you own.

Example: 1.957.764 kWh / 5721 * 9 = 3.079,86575773466 kWh

I'm currently testing to see if it matches the charts from the website.

Ideas

If anyone else an idea on something that's missing or a new feature. I'd like to hear your suggestion.

vressel commented 2 years ago

It seems that the API is no longer delivering actual data, timestamp is stuck on: "timestamp":"2022-08-01T13:13:30"

Hel1Pink commented 2 years ago

@jobvk: I did use de old wincentrale api (zep-api) with and without domoticz. But it stopped working at 1-8-2022. I did find some API-links to the new live production, but I am not Authorized. How do I get a token? And how can I get the new api's to work? I wil be happy if I can get de API working even if the token only works for 60 minutes. Maybe you kan give me also the URL's of the API's you have found?

jobvk commented 2 years ago

@Hel1Pink I’m still testing. You can expect a release next week. For the live data en the news data. Production history will follow later.

Xorfor commented 2 years ago

If possible, I want to be a beta tester?

jobvk commented 2 years ago

@Simanias @gerard76 @Xorfor @vressel @Hel1Pink If you want you can test the new beta. See if everything works well. It's not perfect yet, but I want to see if the live sensors are working. The history and news sensors will come back later. If something goes wrong or something is not right like missing shares or windturbines, please let us know. https://github.com/jobvk/Home-Assistant-Windcentrale/releases/tag/0.2.0-Beta

Xorfor commented 2 years ago

I just installed the new beta. As far as I can check, the data is correct for at least one windmill. Because of no wind, the other windmill doesn't give data. Good work! It is stupid, that they don't make the kWh available per hour or live per day 😢. But I really like to have that data to calculate my total energy consumption/production per day. Even if it is one day too late 😄 .

gerard76 commented 2 years ago

Works like a charm!

It gave unique names to sensors I already had. All values seem to check out.

I am not sure .energy_management is a good name for my share production. I would have picked:

over

but other then that I am very happy to get everything up and running again! Thanks!

vressel commented 2 years ago

Thanks! Beta now running for several hours, and looking fine, only Last Update gives a message: invalid timestamp.

Op 16 aug. 2022, om 17:19 heeft Job van Koeveringe @.***> het volgende geschreven:

@Simanias https://github.com/Simanias @gerard76 https://github.com/gerard76 @Xorfor https://github.com/Xorfor @vressel https://github.com/vressel @Hel1Pink https://github.com/Hel1Pink If you want you can test the new beta. See if everything works well. It's not perfect yet, but I want to see if the live sensors are working. The history and news sensors will come back later. If something goes wrong or something is not right like missing shares or windturbines, please let us know. https://github.com/jobvk/Home-Assistant-Windcentrale/releases/tag/0.2.0-Beta https://github.com/jobvk/Home-Assistant-Windcentrale/releases/tag/0.2.0-Beta — Reply to this email directly, view it on GitHub https://github.com/jobvk/Home-Assistant-Windcentrale/issues/26#issuecomment-1216783059, or unsubscribe https://github.com/notifications/unsubscribe-auth/AWZGDYYX6262TBXC4S2YKCLVZOWPXANCNFSM52CCG6CA. You are receiving this because you were mentioned.

Hel1Pink commented 2 years ago

Thanks. It is running from yesterdag evening. Is working fine and de data seems OK for my windmill parts

Xorfor commented 2 years ago

I have shares of 2 windmills, De Grote Geert and De Trouwe Wachter. Is there a reason why the data of sensor.de_grote_geert is updated every 30 seconds, but sensor.de_trouwe_wachter only every 10 minutes?

rob-on-git commented 2 years ago

Het rode hert looks fine with the beta version. Still trying to mimic the data I had before when is used the old API.

Keep up the good work!

rob-on-git commented 2 years ago

Made a template sensor to mimic the run-percentage for this year: {{ (states('sensor.het_rode_hert_hours_run_this_year') | float / ((as_timestamp(now()) - as_timestamp(now().year ~ "-01-01")) / 3600) * 100 )| round(2)}} Might be useful

steenderen commented 2 years ago

Hi, just installed it for the replacement of my scripts you guys found at https://steenderen.net/blog/winddelen-opbrengst-in-home-assistant/ which is indeed also not working anymore. Everything seems to work fine in the beta version. Is there a difference between the Energy Management sensor and the Production This Year?

gerard76 commented 2 years ago

Is there a difference between the Energy Management sensor and the Production This Year?

Energy Management is your production (total / total_shares * your shares) and Production this Year is total for the mill.

balk77 commented 2 years ago

also running fine for me. And I managed to make some graphs:

Screenshot 2022-08-20 at 21 06 27

It consumes the cumulative yearly kWh production into a Utility Meter sensor, into a hourly, daily, weekly and monthly buckets. It then takes the data from the long term statistics database and plots with Apexcharts. Explained here.

Xorfor commented 2 years ago

Hmmm. Today no data from De Grote Geert:

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 514, in async_update_ha_state await self.async_device_update() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 709, in async_device_update raise exc File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/config/custom_components/windcentrale/sensor.py", line 158, in update self._state = self._windturbine.live_data[self._sensor] KeyError: 'wind_power'

Xorfor commented 2 years ago

For some reason I get data again from De Grote Geert since last night. I guess this windmill didn't deliver data (again) yesterday.

jobvk commented 2 years ago

Made a template sensor to mimic the run-percentage for this year: {{ (states('sensor.het_rode_hert_hours_run_this_year') | float / ((as_timestamp(now()) - as_timestamp(now().year ~ "-01-01")) / 3600) * 100 )| round(2)}} Might be useful

@rob-on-git I added your formula to version 0.2.1

rob-on-git commented 2 years ago

Energy Management

The production sensor that is currently used sensor.name_day_production no longer works because the API only updates after the day has already passed. So I created a new sensor and called it Energy Management. The Energy Management sensor copy's the value of the Production This Year sensor and divides it by total shares and multiply it by the number of shares you own.

Example: 1.957.764 kWh / 5721 * 9 = 3.079,86575773466 kWh

I'm currently testing to see if it matches the charts from the website.

Ideas

If anyone else an idea on something that's missing or a new feature. I'd like to hear your suggestion.

Hi Job,

The energy-management works fine, but i would suggest you do a 'round(3)' on it, as a Wh is already detailed, and more digits is a bit useless, i would say.

Also, I would suggest you rename it to : [mill]_energy_shares (and optional, the [mill]_production_this_year to [mill]_energy)

Xorfor commented 2 years ago

Thanks to the ideas from @balk77, I get almost the same values per day as in the Windcentrale portal .

Hel1Pink commented 2 years ago

@jobvk I want to thank you for the good work. I do not use Home Assistant, but your scripts helped me to access the new windcentrale API. I made my own python script to put data of Livewind and History directly in a MariaDB database. I use Grafana to visualize the grafics.

balk77 commented 2 years ago

I have noticed a weird irregularity earlier this week. The sensor reported minus 17 thousand kWh production. Next datapoint it was corrected again. This offsets the utility meter sensor quite a bit.

Did anyone else experience this? Would it be possible to build in a filter to remove these big jumps? I did not find this spike on mijn windcentrale.

Screenshot_20220825-184617

Edit: There was a peak but not that large: Screenshot_20220825-185547

rob-on-git commented 2 years ago

Maybe another usefull item. Some people want to know how much production is already realized in %. (ofcourse, you can see it through the graphs, but a percentage is easier to read.

I've got this template inserted:

    [windmill]_produced_this_year:
      friendly_name: [Windmill] Year production prognoses
      unit_of_measurement: '%'
      value_template: >
        {{ (states('sensor.[windmill]_production_this_year') | float / 3324000 * 100 ) | round(1) 
}}

Where the value is the total amount of shares from a windmill * 500 (500kW ~ one share)

It is not totally accurate, cause not every windmill has 500kW/share, but it is close enough i think. And within your code I did see already the amount of shares defined for every windmill.

rob-on-git commented 2 years ago

I have noticed a weird irregularity earlier this week. The sensor reported minus 17 thousand kWh production. Next datapoint it was corrected again. This offsets the utility meter sensor quite a bit.

Did anyone else experience this? Would it be possible to build in a filter to remove these big jumps? I did not find this spike on mijn windcentrale.

Edit: There was a peak but not that large:

To avoid those spikes, i usually make an extra sensor with something like this:

[windmill]_prod:
 value_template: >-
          {% if float([windmill]_production) > 100000 %}
            unknown
          {% else %}
            {{ [windmill]_production }}
          {% endif %}

Where, ofcourse the 10000 is the max value you think should be allowed...

balk77 commented 2 years ago

value_template: >- {% if float([windmill]_production) > 100000 %} unknown {% else %} {{ [windmill]_production }} {% endif %}

Thanks for the suggestion! You have put me on the right track; I have configured a filter sensor instead. Advantage: easier to configure and it passes the old value if filter was activated.

- platform: filter
  name: "het rode hert energy management filtered"
  entity_id: sensor.het_rode_hert_energy_management
  filters:
    - filter: outlier
      window_size: 4
      radius: 4.0
      precision: 3

My maximum daily production is approximately 6.5 kWh. A jump of 4 kWh would indicate an outlier.

Xorfor commented 2 years ago

Suggestion: add the total number and the number of owned shares of a windmill as entities. This allows us to do calculations without 'hardcoded' values.

rob-on-git commented 2 years ago

Suggestion: add the total number and the number of owned shares of a windmill as entities. This allows us to do calculations without 'hardcoded' values.

The amount of shares is already there, as attribute of the windmill entitity. This can already be used for calculation. The total amount is not there (i believe, not available through API, hence harcoded, but maybe job can export those from hardcoded to the attribute).

@Xorfor : I created a pull request, which also will contain the total shares as attribute of the windmill (similar as amount of your own shares)

jobvk commented 2 years ago

Energy Management

The production sensor that is currently used sensor.name_day_production no longer works because the API only updates after the day has already passed. So I created a new sensor and called it Energy Management. The Energy Management sensor copy's the value of the Production This Year sensor and divides it by total shares and multiply it by the number of shares you own. Example: 1.957.764 kWh / 5721 * 9 = 3.079,86575773466 kWh I'm currently testing to see if it matches the charts from the website.

Ideas

If anyone else an idea on something that's missing or a new feature. I'd like to hear your suggestion.

Hi Job,

The energy-management works fine, but i would suggest you do a 'round(3)' on it, as a Wh is already detailed, and more digits is a bit useless, i would say.

Also, I would suggest you rename it to : [mill]_energy_shares (and optional, the [mill]_production_this_year to [mill]_energy)

@rob-on-git Name changes will be available in version 0.2.2

jobvk commented 2 years ago

mber and the number of owned shares of a windmill as entities. This allows us to do cal @Xorfor I created a discussions page where you van share your ideas https://github.com/jobvk/Home-Assistant-Windcentrale/discussions/categories/ideas

jobvk commented 2 years ago

I have noticed a weird irregularity earlier this week. The sensor reported minus 17 thousand kWh production. Next datapoint it was corrected again. This offsets the utility meter sensor quite a bit. Did anyone else experience this? Would it be possible to build in a filter to remove these big jumps? I did not find this spike on mijn windcentrale. Edit: There was a peak but not that large:

To avoid those spikes, i usually make an extra sensor with something like this:

[windmill]_prod:
 value_template: >-
          {% if float([windmill]_production) > 100000 %}
            unknown
          {% else %}
            {{ [windmill]_production }}
          {% endif %}

Where, ofcourse the 10000 is the max value you think should be allowed...

@rob-on-git I'm looking in to it. maybe you added it to the discussions page where you van share your ideas. https://github.com/jobvk/Home-Assistant-Windcentrale/discussions/categories/ideas

jobvk commented 2 years ago

Many thanks to everyone who helped fixing this issue

I am closing this issue now as the original issue has been resolved for the majority. If you want to talk about the wind power plant in general, be sure to check out the github discussions general page. https://github.com/jobvk/Home-Assistant-Windcentrale/discussions/categories/general

If you have any ideas that should be added, please post them on the github discussions ideas page. https://github.com/jobvk/Home-Assistant-Windcentrale/discussions/categories/ideas

And if you still run into an issue. You can always create a new issue. https://github.com/jobvk/Home-Assistant-Windcentrale/issues/new/choose