search

ssenart / home-assistant-gazpar

GrDF Gazpar meter integration for Home Assistant
MIT License
43 stars 9 forks source link

[Issue] No data update - GrDF web site is half broken - Download button does not work anymore #28

Closed Patbonamy closed 1 year ago

Patbonamy commented 1 year ago

Hi, since 8 days the integration not updated the gazpar data. Thanks for a help. Pat

ssenart commented 1 year ago

Hello, I can't help. The integration relies on GrDF site (Mon espace), and data are not updated since 22/11.

Patbonamy commented 1 year ago

Ok, thanks Have a good evening. Regards Pat

hayvan96 commented 1 year ago

Hi, it seems API have changed since 29/11/2022? https://sites.grdf.fr/web/portail-api-grdf-adict/documentation_fonctionnelle https://sites.grdf.fr/web/portail-api-grdf-adict/documentation-api-grdf-adict-production

Patbonamy commented 1 year ago

Here is the problem: https://sites.grdf.fr/web/portail-api-grdf-adict/indisponibilites

I think...

hayvan96 commented 1 year ago

You're right. This being said, beyond the temporary glitches listed there, a change of API is also mentioned. I don't know if the script is impacted or not

tikismoke commented 1 year ago

Same for me since same date 😭

olivierboudet commented 1 year ago

There is an other API which is working fine :

https://monespace.grdf.fr/api/e-conso/pce/consommation/informatives?dateDebut=2019-12-01&dateFin=2022-12-01&pceList%5B%5D=xxxxxxxx

Would it be possible to use it instead of the telecharger API ?

Thanks

ssenart commented 1 year ago

Hello, thanks for that, I will deliver a fix shortly (if this new URL works fine).

ssenart commented 1 year ago

It works but the format is not the same. We have in return a Json document instead of an Excel file. This Json file only contains daily data. I have to rebuild the weekly and monthly data where they were immediatly available before. There's quite some work to adapt to this URL... not straigthforward.

The Integration mimics the "Télécharger" button on the GrDF web site. And this button does not work any more (you can try yourself on the site). We have a 404 error => GrDF site is still half broken. It happens once a year and it takes times for them to recover it.

It seems they are aware of that issue. I see the message after trying to download: image I hope they are working on it and we will have shortly a solution.

Please, be patient !

Patbonamy commented 1 year ago

There is an other API which is working fine :

https://monespace.grdf.fr/api/e-conso/pce/consommation/informatives?dateDebut=2019-12-01&dateFin=2022-12-01&pceList%5B%5D=xxxxxxxx

Would it be possible to use it instead of the telecharger API ?

Thanks

Not working for me... json file empty...

ssenart commented 1 year ago

Things are getting tough. GrDF have changed something in their login process... my account is now locked.

Does someone have a JSON sample of the data got by the URL: https://monespace.grdf.fr/api/e-conso/pce/consommation/informatives?dateDebut=2019-12-01&dateFin=2022-12-01&pceList%5B%5D=xxxxxxxx

I will try to progress in the dev based on that sample (waiting for my account being unlocked).

helepeut commented 1 year ago

Unfortunately, I'm unable to login too. The site is asking for email and password, try to connect. Error regarding the captcha which appears. Clicking on it, disappearing, Error again regarding the captcha which appears then. I'm going crazy :')

Patbonamy commented 1 year ago

Unfortunately, I'm unable to login too. The site is asking for email and password, try to connect. Error regarding the captcha which appears. Clicking on it, disappearing, Error again regarding the captcha which appears then. I'm going crazy :')

Same for me...

BolAuxNiaises commented 1 year ago

Things are getting tough. GrDF have changed something in their login process... my account is now locked.

Does someone have a JSON sample of the data got by the URL: https://monespace.grdf.fr/api/e-conso/pce/consommation/informatives?dateDebut=2019-12-01&dateFin=2022-12-01&pceList%5B%5D=xxxxxxxx

I will try to progress in the dev based on that sample (waiting for my account being unlocked).

I don't have data until the end of 2020, the 2019 start date causes an http 400 error. With https://monespace.grdf.fr/api/e-conso/pce/consommation/informatives?dateDebut=2020-12-01&dateFin=2022-12-01&pceList%5B%5D=09xxxxxxxxxxxx

We get this response: https://codeshare.io/YLMEkN

Patbonamy commented 1 year ago

I found a way to connect again. Click on "mot de passe perdu" After I put the same password and ok to connect. But no json file for me.

nmorel73 commented 1 year ago

it is working for me today with Dec 1st to Dec 2nd. please find attached the file retuened by the URL if it can help. I replace only my Gazpar Id. nmo.txt

Nmo.

hirooekimae commented 1 year ago

Here is what I got, using this call :

https://monespace.grdf.fr/api/e-conso/pce/consommation/informatives?dateDebut=2022-11-01&dateFin=2022-11-11&pceList%5B%5D=xxxxx

{"xxxxx":{"idPce":"xxxxx","releves":[{"dateDebutReleve":"2022-10-31T06:00:00+00:00","dateFinReleve":"2022-11-01T06:00:00+00:00","journeeGaziere":"2022-10-31","indexDebut":21179,"indexFin":21183,"volumeBrutConsomme":4,"energieConsomme":45,"pcs":null,"volumeConverti":null,"pta":null,"natureReleve":"Informative Journalier","qualificationReleve":"Mesur\u00e9","status":null,"coeffConversion":11.130000000000001,"frequenceReleve":null,"temperature":null},{"dateDebutReleve":"2022-11-01T06:00:00+00:00","dateFinReleve":"2022-11-02T06:00:00+00:00","journeeGaziere":"2022-11-01","indexDebut":21183,"indexFin":21188,"volumeBrutConsomme":5,"energieConsomme":54,"pcs":null,"volumeConverti":null,"pta":null,"natureReleve":"Informative Journalier","qualificationReleve":"Mesur\u00e9","status":null,"coeffConversion":11.130000000000001,"frequenceReleve":null,"temperature":null},{"dateDebutReleve":"2022-11-02T06:00:00+00:00","dateFinReleve":"2022-11-03T06:00:00+00:00","journeeGaziere":"2022-11-02","indexDebut":21188,"indexFin":21195,"volumeBrutConsomme":8,"energieConsomme":85,"pcs":null,"volumeConverti":null,"pta":null,"natureReleve":"Informative Journalier","qualificationReleve":"Mesur\u00e9","status":null,"coeffConversion":11.130000000000001,"frequenceReleve":null,"temperature":null},{"dateDebutReleve":"2022-11-03T06:00:00+00:00","dateFinReleve":"2022-11-04T06:00:00+00:00","journeeGaziere":"2022-11-03","indexDebut":21195,"indexFin":21205,"volumeBrutConsomme":10,"energieConsomme":108,"pcs":null,"volumeConverti":null,"pta":null,"natureReleve":"Informative Journalier","qualificationReleve":"Mesur\u00e9","status":null,"coeffConversion":11.130000000000001,"frequenceReleve":null,"temperature":null},{"dateDebutReleve":"2022-11-04T06:00:00+00:00","dateFinReleve":"2022-11-05T06:00:00+00:00","journeeGaziere":"2022-11-04","indexDebut":21205,"indexFin":21216,"volumeBrutConsomme":11,"energieConsomme":124,"pcs":null,"volumeConverti":null,"pta":null,"natureReleve":"Informative Journalier","qualificationReleve":"Mesur\u00e9","status":null,"coeffConversion":11.130000000000001,"frequenceReleve":null,"temperature":null},{"dateDebutReleve":"2022-11-05T06:00:00+00:00","dateFinReleve":"2022-11-06T06:00:00+00:00","journeeGaziere":"2022-11-05","indexDebut":21216,"indexFin":21221,"volumeBrutConsomme":5,"energieConsomme":55,"pcs":null,"volumeConverti":null,"pta":null,"natureReleve":"Informative Journalier","qualificationReleve":"Mesur\u00e9","status":null,"coeffConversion":11.130000000000001,"frequenceReleve":null,"temperature":null},{"dateDebutReleve":"2022-11-06T06:00:00+00:00","dateFinReleve":"2022-11-07T06:00:00+00:00","journeeGaziere":"2022-11-06","indexDebut":21221,"indexFin":21233,"volumeBrutConsomme":12,"energieConsomme":129,"pcs":null,"volumeConverti":null,"pta":null,"natureReleve":"Informative Journalier","qualificationReleve":"Mesur\u00e9","status":null,"coeffConversion":11.130000000000001,"frequenceReleve":null,"temperature":null},{"dateDebutReleve":"2022-11-07T06:00:00+00:00","dateFinReleve":"2022-11-08T06:00:00+00:00","journeeGaziere":"2022-11-07","indexDebut":21233,"indexFin":21241,"volumeBrutConsomme":8,"energieConsomme":93,"pcs":null,"volumeConverti":null,"pta":null,"natureReleve":"Informative Journalier","qualificationReleve":"Mesur\u00e9","status":null,"coeffConversion":11.130000000000001,"frequenceReleve":null,"temperature":null},{"dateDebutReleve":"2022-11-08T06:00:00+00:00","dateFinReleve":"2022-11-09T06:00:00+00:00","journeeGaziere":"2022-11-08","indexDebut":21241,"indexFin":21247,"volumeBrutConsomme":6,"energieConsomme":66,"pcs":null,"volumeConverti":null,"pta":null,"natureReleve":"Informative Journalier","qualificationReleve":"Mesur\u00e9","status":null,"coeffConversion":11.130000000000001,"frequenceReleve":null,"temperature":null},{"dateDebutReleve":"2022-11-09T06:00:00+00:00","dateFinReleve":"2022-11-10T06:00:00+00:00","journeeGaziere":"2022-11-09","indexDebut":21247,"indexFin":21255,"volumeBrutConsomme":8,"energieConsomme":94,"pcs":null,"volumeConverti":null,"pta":null,"natureReleve":"Informative Journalier","qualificationReleve":"Mesur\u00e9","status":null,"coeffConversion":11.130000000000001,"frequenceReleve":null,"temperature":null},{"dateDebutReleve":"2022-11-10T06:00:00+00:00","dateFinReleve":"2022-11-11T06:00:00+00:00","journeeGaziere":"2022-11-10","indexDebut":21255,"indexFin":21264,"volumeBrutConsomme":8,"energieConsomme":91,"pcs":null,"volumeConverti":null,"pta":null,"natureReleve":"Informative Journalier","qualificationReleve":"Mesur\u00e9","status":null,"coeffConversion":11.130000000000001,"frequenceReleve":null,"temperature":null}],"frequence":null}}

ssenart commented 1 year ago

Thanks all for your help.

It permits to progress and I published a first alpha version (1.3.4-alpha.1).

It is now based 100% on URL https://monespace.grdf.fr/api/e-conso/pce/consommation/informatives?dateDebut=2022-11-01&dateFin=2022-11-11&pceList%5B%5D=xxxxx with the Json document.

Currently, only the daily consumption is available. I have to go further and compute weekly, monthly and yearly data...

Patbonamy commented 1 year ago

Thanks Ssenart. It works for the daily update, but I haven't the two days before the current day, I remember it was the same before.

ssenart commented 1 year ago

Yes, at least it should be the same as the GrDF site.

ssenart commented 1 year ago

I've just published a second alpha version (1.3.4-alpha.2) that brings back weekly, monthly and yearly consumptions.

hirooekimae commented 1 year ago

Thanks. A few problems in months names. They all appear with a 'D'. Capture d’écran 2022-12-04 à 21 20 09

ssenart commented 1 year ago

Yes, thanks, fixed in 1.3.4-alpha.3. Temperatures are now retrieved correctly and displayed in the card (from card version 1.3.8)

hirooekimae commented 1 year ago

Thanks, it works perfectly.

Patbonamy commented 1 year ago

Thanks, it works perfectly.

Same for me. You're a master!

nmorel73 commented 1 year ago

nice! it is working perfectly ! Thank you !

olivierboudet commented 1 year ago

Hello, I have an issue wihtout 1.3.4.alpha-3 : The value of the sensor sensor.gazpar is correct:

$ select * from states where entity_id like 'sensor.gazpar%';
10574556||sensor.gazpar|1889.42||||2022-12-06 22:17:41.634044||01GKMR88G25S1XPTMTWAQT6RP9||10574177|115565|0|
10574906||sensor.gazpar|1889.42|||2022-12-06 22:17:41.634044|2022-12-07 00:17:27.584934||01GKMZ3J109RRR7YB9W50TTY8E||10574556|115620|0|
10575068||sensor.gazpar|1889.42|||2022-12-06 22:17:41.634044|2022-12-07 01:17:54.896964||01GKN2J8AG64H38F8H4HTTXSGY||10574906|115647|0|
10575473||sensor.gazpar|1889.42|||2022-12-06 22:17:41.634044|2022-12-07 03:17:36.532845||01GKN9DDMMST9NM9YWJX0XABQ1||10575068|115696|0|
10575732||sensor.gazpar|1889.42|||2022-12-06 22:17:41.634044|2022-12-07 04:17:03.603414||01GKNCT93KVYZPSTTAZHK8K33P||10575473|115721|0|
10575735||sensor.gazpar|1889.42|||2022-12-06 22:17:41.634044|2022-12-07 04:17:45.042871||01GKNCVHJJQ4N75PMSK0SG1YJ7||10575732|115722|0|
10576110||sensor.gazpar|1889.42|||2022-12-06 22:17:41.634044|2022-12-07 06:17:19.596302||01GKNKPFZCF6WMNBEE7ST8DA6S||10575735|115774|0|
10576383||sensor.gazpar|1889.42|||2022-12-06 22:17:41.634044|2022-12-07 07:17:09.460640||01GKNQ41PM03XTWGTC7VF3XGYT||10576110|115819|0|
10576705||sensor.gazpar|unknown||||2022-12-07 07:47:58.813868||01GKNRWFPXZXC8HPFRAKCJVY89|||115825|0|
10576776||sensor.gazpar|1889.42||||2022-12-07 07:48:10.398154||01GKNRWV0YYE7Y7JNFWCTQTMX6||10576705|115826|0|

But the computation of daily/weekly/monthly/yearly values are incorrect:

$  select * from states where entity_id like 'sensor.monthly_gas_energy';
10471939||sensor.monthly_gas_energy|223.60||||2022-11-21 15:53:59.079239||01GJDEAWH73V1H4B4RZ55EZZXM|||87832|0|
10478207||sensor.monthly_gas_energy|257.14||||2022-11-22 15:53:58.566277||01GJG0QK16QXE7N9QPJTWZ07HX||10471939|87832|0|
10485557||sensor.monthly_gas_energy|268.32||||2022-11-23 14:53:57.907204||01GJJFPDRKM63JBTZQS2R0E0D2||10478207|87832|0|
10491252||sensor.monthly_gas_energy|301.86||||2022-11-24 15:53:58.861441||01GJN5H1AD9CBW8QYGS9HF2X3H||10485557|87832|0|
10533268||sensor.monthly_gas_energy|301.86||||2022-11-30 08:59:06.779738||01GK3W5PMVYD16W89WJXX9VGP0|||87832|0|
10535989||sensor.monthly_gas_energy|301.86||||2022-11-30 15:43:05.637227||01GK4K9DD586E58HAEEBJR05J2|||87832|0|
10539349||sensor.monthly_gas_energy|301.86||||2022-11-30 22:07:49.876282||01GK599WKM74G3YM71NXXZAM25|||87832|0|
10539593||sensor.monthly_gas_energy|301.86||||2022-11-30 22:16:45.899500||01GK59T82BH9G6TQEEQV7Y558F|||87832|0|
10539892||sensor.monthly_gas_energy|301.86||||2022-11-30 22:22:37.161882||01GK5A4Z39PXD9FD81SP9MN1KE|||87832|0|
10540394||sensor.monthly_gas_energy|0||||2022-12-01 00:00:00.075820||01GK5FQ92BGFSZMHAC3GBQ1RR3||10539892|111821|0|
10541688||sensor.monthly_gas_energy|0||||2022-12-01 07:07:11.107279||01GK685FC3155WR359AH43V7FY|||111821|0|
10550555||sensor.monthly_gas_energy|0||||2022-12-02 17:47:07.482982||01GK9Z5YTTQGE82W40KATJE7J1|||111821|0|
10551142||sensor.monthly_gas_energy|0||||2022-12-02 18:57:00.513576||01GKA35XK15RR23Z6NPFGJW08D|||111821|0|
10554784||sensor.monthly_gas_energy|0||||2022-12-03 11:46:37.780515||01GKBWYK2MDGBFV50BPVVZTYHA|||111821|0|
10560402||sensor.monthly_gas_energy|0||||2022-12-04 13:45:10.684430||01GKEP4C8WK0YGWVPTGF6MJC7X|||111821|0|
10561311||sensor.monthly_gas_energy|0||||2022-12-04 17:25:11.579859||01GKF2Q7RVZCWDNVQYX93JPW5M|||111821|0|
10561576||sensor.monthly_gas_energy|0||||2022-12-04 17:26:39.572570||01GKF2SXPMJNMN03RBTVQQVDM5|||111821|0|
10561856||sensor.monthly_gas_energy|0||||2022-12-04 17:33:54.861838||01GKF376SDPMRYA1Y43F7S2121|||111821|0|
10562147||sensor.monthly_gas_energy|0||||2022-12-04 17:55:41.629858||01GKF4F2XXZD9Z9TQD9XR316F5|||111821|0|
10562446||sensor.monthly_gas_energy|0||||2022-12-04 17:59:32.825979||01GKF4P4PSZNXET5YDBH8Z8HVF|||111821|0|
10562792||sensor.monthly_gas_energy|0||||2022-12-04 18:09:42.338567||01GKF58QY2R9M8PZCGVNGSVCAX|||111821|0|
10563046||sensor.monthly_gas_energy|0||||2022-12-04 18:13:20.882520||01GKF5FDBJ6Q9NB11SSTZGW2K7|||111821|0|
10563314||sensor.monthly_gas_energy|0||||2022-12-04 18:17:38.015636||01GKF5Q8EZ5F3DXP0QND07Q71X|||111821|0|
10563988||sensor.monthly_gas_energy|0||||2022-12-04 20:35:55.366119||01GKFDMFB62QFVYP9Q76JGFN0G|||111821|0|
10564287||sensor.monthly_gas_energy|0||||2022-12-04 20:39:57.702924||01GKFDVW06T807Y6QVBG9H5WF9|||111821|0|
10564545||sensor.monthly_gas_energy|0||||2022-12-04 20:42:50.295429||01GKFE14HQPPCMRGBEM0XV13ET|||111821|0|
10574186||sensor.monthly_gas_energy|5668.26||||2022-12-06 21:17:03.566656||01GKMMS7PE1VVDVJ4RZJFJKC1G|||111821|0|
10574562||sensor.monthly_gas_energy|7557.68||||2022-12-06 22:17:41.637359||01GKMR88G5BBGW9CT0MVYFT34X||10574186|111821|0|
10576558||sensor.monthly_gas_energy|7557.68||||2022-12-07 07:47:58.843651||01GKNRWFQV3PF00RQAX7R69Y7Y|||111821|0|
10576782||sensor.monthly_gas_energy|9447.10||||2022-12-07 07:48:10.414554||01GKNRWV1E9RCTVAR1CW1MMBAK||10576558|111821|0|

Since December 4th, after upgraded to 1.3.4 alpha version, value of sensor.gazpar is added to the monthly value each time.

This is my config: 

template:
  - sensor:
    - name: gas_volume
      unit_of_measurement: 'mÂł'
      state: >
        {{ state_attr('sensor.gazpar', 'daily')[0]['end_index_m3'] | float(0) }}
      icon: mdi:fire
      device_class: gas
      state_class: total_increasing
    - name: gas_energy
      unit_of_measurement: 'kWh'      
      state: >   
        {{ states('sensor.gazpar') | float(0) }}
      icon: mdi:fire
      device_class: gas
      state_class: total_increasing
utility_meter:
  total_gas_volume:
    source: sensor.gas_volume
  daily_gas_volume:
    source: sensor.gas_volume    
    cycle: daily
  weekly_gas_volume:
    source: sensor.gas_volume    
    cycle: weekly 
  monthly_gas_volume:
    source: sensor.gas_volume    
    cycle: monthly
  yearly_gas_volume:
    source: sensor.gas_volume    
    cycle: yearly
ssenart commented 1 year ago

It is the expected behaviour. The monthly_gas_energy is the accumulation of all the daily consumption (so sensor.gas_energy) until the end of the month when this counter is reset to zero.

utility_meter is not part of gazpar integration. It is documented here: https://www.home-assistant.io/integrations/utility_meter/

dcaman commented 1 year ago

It worked for me only few days; no more data since this Sunday. I'm using the 1.3.4 alpha3 version. Any tip or advice ?

image

ssenart commented 1 year ago

What about data on GrDF site (Mon Espace) ? The integration depends on their data ?

dcaman commented 1 year ago

You're right, nothing there also... Don't know how often Gazpar send data; was thinking it was every 24h00.

image

ssenart commented 1 year ago

Usually every day with a delay of 2 days.

Since 3 weeks, it is erratic.

hayvan96 commented 1 year ago

Hi I've upgrade to 1.3.4.a4 and I'm a bit surprised. The Kwh shown in HA Energy dashboard is way to high. It says 8747,55 kWh for today

My daily use shoud be around 100 kwH, see below. Which sensor should I add to HA Energy? all seem to show the same result.

This is the content of the sensor (I've removed the first lines)

`unit_of_measurement: kWh friendly_name: Gazpar icon: mdi:fire device_class: energy state_class: total_increasing errorMessages: hourly: daily:

weekly:

monthly:

yearly:

ssenart commented 1 year ago

Hello, HA computes the today consumption with the difference of the total meter of today (given by the state value of sensor.gazpar) and the last meter value found in its database.

For example, if you stop HA for 1 month, the day you restart it, you will have your 1 month consumption aggregated on the today consumption.

I don't see any other reason why HA should show you a consumption that high.

In your energy dashboard, you can directly setup in gas section with sensor.gazpar. But I think it is a good practice to have one level of indirection for the naming using a template as below:

template:
    - name: gas_energy
      unit_of_measurement: 'kWh'      
      state: >   
        {{ states('sensor.gazpar') }}
      icon: mdi:fire
      device_class: energy
      state_class: total_increasing

And you use sensor.gas_energy in your Energy Dashboard.

Later, if you want to use another sensor than sensor.gazpar for gas source, you only have to change this template, with no impact in HA database and data history.

hayvan96 commented 1 year ago

Even after having uninstalled/reinstalled the component, I still have the same behaviour. Two data per day (00-01, 02-03, so in the middle of the night, when I used to have only on data point around 1800). And when I look at the week view, it shows 13760 kwH every day, and today, it has even doubled. It looks like it's cumulating all values instead of computing deltas.

I used to have this component: https://github.com/cyprieng/gazpar-home-assistant which uses the same name. EVen though I've uninstalled repository, component and files from that addon, could it be the culprit ? My case seems isolated.

ssenart commented 1 year ago

Yes, indeed, that may explain. If the index_kwh is not computed the same way, and the final index is not the same, what you observe is the diff between the two indexes.

Wait for GrDF data to be recovered. You will see there is no problem and can observe your daily consumption as expected (except the 3/12 when you switch from the old sensor implementation to the new one).

hayvan96 commented 1 year ago

OK, I've purged the SQL database of all gazpar lines, I'll see how it goes in a couple of hours -- probable a local artefact and nothing linked to this addon ;) EDIT: nope, no change, I still receive two updates early in the morning (midnight, 2am), like 5000 kWh and 8000 kwh two hours later. I have removed all statistics linked to gas in the database, yesterday it was initialised at 70 000 kWh, I would have thought I would received correct values after that.

cannabarman commented 1 year ago

Hi @hayvan96, Same problem here, 2 measurements in the early morning and then nothing for the entire day. Have you found a solution ?

PhilippeAB commented 1 year ago

Hi all, GRDF website is recovering data but the data in Gazpar don't match the data from GRDF website. Yesterday I got 3 data point with 111, 122, 122 kWh on this day for about 355kWh. Today I got 4 data points (2:14 = 100kWh, 4:14 = 222kWh, 8:15 = 78kWh, 16:30=67) for a total of 467 kWh. This does not reflect my gaz usage. sensor:

ssenart commented 1 year ago

Hi PhilippeAB, for this recovery, GrDF has published successively each missing days since the december 4th until last sunday. Those data points you had should correspond exactly to those publications. I invite you to go in development tools, in States tab as below: image You can see your consumption as sent by GrDF. Also, you can use the corresponding Lovelace Card to display it nicely: https://github.com/ssenart/lovelace-gazpar-card

PhilippeAB commented 1 year ago

Your right I didn't realize they was catching up the data processing. Today it work perfectly Thanks @ssenart