barisdemirdelen / homeassistant-greenchoice

This is a Home Assistant custom component that connects to the Greenchoice API
MIT License
31 stars 6 forks source link

HTTP Error: 500 Server Error: Internal Server Error for url: https://mijn.greenchoice.nl/microbus/request #11

Closed FlexxFR closed 2 months ago

FlexxFR commented 2 months ago

Since 2 days the intergration stopped working. Tried to reinstall but didn;t work. Is this a known issue?

szamfirov commented 2 months ago

The Greenchoice API endpoint(s) have been changed. Here's a fix which would help: https://github.com/barisdemirdelen/homeassistant-greenchoice/pull/12

FlexxFR commented 2 months ago

Great work. Will try to test this.

barisdemirdelen commented 2 months ago

Fixed with #12

FlexxFR commented 2 months ago

I'm sorry, but it is still not working for me. I tried uninstalling/installing multiple times.

It does "something" as it "used some off my personal info in a URL request.

HTTP Error: 404 Client Error: Not Found for url: https://mijn.greenchoice.nl/api/v2/Rates/2177418?AgreementIdElectricity=2776016&AgreementIdGas=2776016&HouseNumber=<Correct Number>&ReferenceIdElectricity=179687&ReferenceIdGas=&ZipCode=<Correct ZIP> When I browse to the URL directly: {"type":"https://tools.ietf.org/html/rfc9110#section-15.5.5","title":"Not Found","status":404,"traceId":"00-8ae3c3ee9dd6ab574bf5fb6900a2f3fe-c5c3e1715bdc8eb6-00"}

barisdemirdelen commented 2 months ago

Ah, interesting. You probably have electricity but no gas connected right?

barisdemirdelen commented 2 months ago

@FlexxFR can you check master branch to see if it works for you now?

FlexxFR commented 2 months ago

Yes, you are correct, no gas, only electricity. I just replayed the files in my custom components folder and rebooted HA. The referenceIdGas part is gone now, so I think it is working with the new code.

Still no success. URL is now:

HTTP Error: 404 Client Error: Not Found for url: https://mijn.greenchoice.nl/api/v2/Rates/2177418?AgreementIdElectricity=2776016&AgreementIdGas=2776016&HouseNumber=XX&ZipCode=XXXXXX&ReferenceIdElectricity=179687

Is there a way I can find this API url? How do you retrieve this? I also tried to remove the other IdGas part, but that does not work either.

barisdemirdelen commented 2 months ago

You can open your developer console in the browser by pressing F12, clicking network mode to see the calls made, and visit to https://mijn.greenchoice.nl/contract

There are a lot of requests but one of them should be a url similar to this url. If you can tell me what the url is for you we can maybe debug it easier. I am not sure why it wouldnt work still, this url looks correct to me. Especially if it doesnt even work after removing "&AgreementIdGas=2776016"

FlexxFR commented 2 months ago

it appears to use this URL but with the same error, 404 not found. Both references to Gas are not present in the URL. Strange thing is that the website appears to work as normal.

Request URL: https://mijn.greenchoice.nl/api/v2/Rates/2177418?AgreementIdElectricity=2776016&HouseNumber=<XX>&ReferenceIdElectricity=179687&ZipCode=<XXXX> Request Method: GET Status Code: 404 Not Found Remote Address: 109.200.196.199:443 Referrer Policy: strict-origin-when-cross-origin

From the main contract page this is the only field that is questionable: image

barisdemirdelen commented 2 months ago

Right, so website tries this url and gets a 404, so it infers you have an unknown tariff and doesnt display any cost information?

Another possibility is that since you have an unknown tariff it looks at another api to display the cost information, if its displayed still somehow.

szamfirov commented 2 months ago

Today, I'm getting the same 404s from the "Rates" API endpoint even though I have both electricity and gas 😩 I was so happy it worked for a while... It has something to do with the "variabel" vs "vast" (aka "isVariabel": false) - the 404 error causes no prices to be displayed in the website as well.

szamfirov commented 2 months ago

I may have just stumbled upon a gem 🙂

Does this URL work for you both?

https://mijn.greenchoice.nl/api/tariffs

I'm wondering if we can fetch the tariffs from there (from the "huidig" section) as a fallback in case we run into a 404 error with the Rates API endpoint 🤔

Here's what I'm getting from "/api/tariffs":

Screenshot 2024-09-03 at 20 13 14

FlexxFR commented 2 months ago

Yes, I have the same values.

It also shows the new tariff that start from oct 1. (forecast). Unfortunately, the "terugleververgoeding" is still not in use (0), but maybe it will in the future. Currently these values are not displayed on the website either.

These urls have also interesting info, but I'm unable to find API calls in the F12 overview (could be the lack of my skills :)) https://mijn.greenchoice.nl/meterstanden-overzicht https://mijn.greenchoice.nl/verbruik

barisdemirdelen commented 2 months ago

Nice. I have uploaded a new branch called model_validation. Would you be so kind to check that and see if it works for you?

I was working on some api validation inside our codebase, since it isnt at all documented. I have added the fallback /api/tariff

If you let me know if it works or not i can continue with it

FlexxFR commented 2 months ago

What I did: I downloaded all from: https://github.com/barisdemirdelen/homeassistant-greenchoice/tree/model_validation

copied all files from: https://github.com/barisdemirdelen/homeassistant-greenchoice/tree/model_validation/custom_components/greenchoice into my custom components folder restart HA (no uninstall/reinstall of the component, not sure what that does)

I now get the error:

Logger: homeassistant.config Source: config.py:1298 First occurred: 22:12:16 (1 occurrences) Last logged: 22:12:16

Platform error: sensor - Requirements for greenchoice not found: ['pydantic>=2.8.0,<3.0.0'].

barisdemirdelen commented 2 months ago

Unfortunate, it seems home assistant doesnt like pydantic v2. I have removed the dependency, it should now work without it.

szamfirov commented 2 months ago

Great work @barisdemirdelen !

Made a small proposal here https://github.com/barisdemirdelen/homeassistant-greenchoice/pull/14 because historically I've had multiple contacts with Greenchoice and the "Profiles" endpoint returns a list of profiles. This results in the first profile being picked up from that list as opposed to the latest one.

I can still see pydantic in that branch so not sure if it's going to work when installed in HA. The rest of the implementation works great when I tested by hand.

barisdemirdelen commented 2 months ago

Nice! Thank you!

Now the only horrible part is, we have switched to mostly v2 api requests, but we still do a microbus init just to get the referenceId for Gas and Electricity 😆 This is what the website does as well, its a bit of shame.

FlexxFR commented 2 months ago

Nice, I replaced the files with the most recent from https://github.com/barisdemirdelen/homeassistant-greenchoice/tree/model_validation/custom_components/greenchoice

and it appears to work, values are available, no obvious errors in the logs :)