Linky Integration: cannot connect with my valid credentials

mixoum commented 4 years ago

The problem

Cannot connect on linky integration with my valid credentials (but working on https://mon-compte-particulier.enedis.fr/). I have the issue: " Erreur inconnue: merci de réessayer plus tard (pas entre 23h et 2h)". I try to add it since since few weeks. It works fine on Enedis and EDF website. it does not work either with the intergation in configuration.yaml file or directly in the lovelace UI.

Environment

Home Assistant Core release with the issue: 107.1
Last working Home Assistant Core release (if known): NA
Operating environment (Home Assistant/Supervised/Docker/venv): Docker On Asustor NAS
Integration causing this issue: Linky
Link to integration documentation on our website: https://www.home-assistant.io/integrations/linky/

Problem-relevant `configuration.yaml`

linky:
  - username: 'xxx'
    password: 'xxx'

Traceback/Error logs

 Logger: homeassistant.components.linky.config_flow

No Data
Source: components/linky/config_flow.py:82
Integration: linky (documentation, issues)
First occurred: 11:20:41 (2 occurrences)
Last logged: 11:39:41

Additional information

probot-home-assistant[bot] commented 4 years ago

Hey there @Quentame, mind taking a look at this issue as its been labeled with a integration (linky) you are listed as a codeowner for? Thanks!

Quentame commented 4 years ago

Hi !

Are you using a personal or professional account ?

At which time of day are you trying to add the integration ?

Thanks

mixoum commented 4 years ago

Hi,

I use a personnal account. Whatever the time of the day I try to connect I have the same message.

Rgds

Antexa commented 4 years ago

Hi,

Same behavior for me aswell (personnal account)

👍

Quentame commented 4 years ago

Can you try to set the timeout to 30 ?

Do you have special network going on over your NAS ? Special DSN resolver ? AdGuard/Pi-hole ? Sub network ?

@Antexa may you specify your operating environment ?

I've just set up my Linky on a HA testing set up, working perfectly.

Thanks

Quentame commented 4 years ago

Are your accounts new ?

There is an issue on the underlying library side that may be linked to your issue : Pirionfr/pyLinky#28

The maintainer seems to be very responsive, you may check this and help him.

mixoum commented 4 years ago

I have access to enedis account since 6 months but I have my data since 1 or 2 months maximum. I tried to connect through the configuartion.yaml file and to add timeout: 30. Unfortunately, it changed nothing. The lovelace integration UI says " New devices discovered. We have discovered new devices on your network. Check it out". But no sensor has been created. I have to add my credentials and I still have the error message.

let us know if you find a solution to this. You are right, it's probably link to the issue you mentionned in the above message.

nephthys commented 4 years ago

I have the same problem with incorrect credentials (new personal account).

If I try the dependency pylinky with the command pylinky -u USERNAME -p PASSWORD, I get the same error: "Login error: Please check your username / password".

Looking in the pylinky/client.py file, there is this variable LOGIN_URL = "https://espace-client-connexion.enedis.fr/auth/UI/Login"

This URL no longer exists so there is a (logical) error. Hopefully the dependency (pyLinky) can be fixed in the future ;)

Quentame commented 4 years ago

Okay so the issue is comming from the underlying library.

So we should all follow this issue Pirionfr/pyLinky#28 The maintainer is responsive and I hope a solution to come soon.

Closing this issue.

Chuckame commented 4 years ago

Why closing this issue because, even if the maintainer is responsive, the pylinky will be updated when the bug will be fixed, so we can keep this issue open to avoid multiple issues reported for the same problem???

Oscar7601 commented 4 years ago

Hello I am facing the same problem with valid credentials not being accepted to connect. Home Assistant is running in Docker on an Unraid server. No noticeable difference if I remove my PiHole from the network.

Chuckame commented 4 years ago

@Quentame please reopen to track the pylinky upgrade

Th0mC commented 4 years ago

The issue seems to have been corrected (pyLinky v0.5.0) but the way to get credentials is a bit more complex : "In order to use the library, you need an account on https://datahub-enedis.fr/. You need to create a Data Connect Application and as of May 2020 you will get your client_id by mail and your client_secret by SMS after a week or so... With these credentials, you can only access the Sandbox environment... You need to sign a contract and probably wait more to get Production credentials..."

peioe commented 4 years ago

Has anyone tried it with this method ? Does it work for you @Th0mC ?

jluc2808 commented 4 years ago

hello, i'm not a HA user but i could give you some info which is similar with other integration or other box. Enedis have implemented the API to let everyone play with the sandbox environment, but only professionnal could access to the production environment. i will explain: end-user needs to give acknowledge (in french consentement) directly inside his own enedis accompt, this acknowledge is converted in a code send to a professionnal entity which have already signed a contract with Enedis. This contract is needed to allow the professionnal structure to recieve the client_id and client_secret for the real environment (the production one).

So no end-user could use the API authorization (Oauth2), only company is allowed to perform the complete job to recieve access_token and refresh_token.

i did that way for lifedomus box.

I have my own company, i have signed a contract with enedis, publish a web server to establish the dialog with end-user, gave enedis access to test the mecanism, then recieve the 2 client_id and client_secret. I have published a little Html page which forward the user into enedis step (consentement), store on a public server an application to recieve the code generated by enedis (on uri_redirect URL), this URL finalize and use this code to get the token (access_token and refresh_token), then forward the 2 token to the end-user application.

Then and only then end-user application could call enedis API with access_token to request for linky data.

peioe commented 4 years ago

Quelle bande de pénibles ...

Th0mC commented 4 years ago

@jluc2808 Any opportunity to use your html page to get our credentials?

jluc2808 commented 4 years ago

yes that need to be tested and surely adapted to send token to HA , as i don't know what is the standard dialog to send to HA , i need some help , perhaps it's easier to gon this discussion with my direct email. jean-luc.berno@wanadoo.fr

jluc2808 commented 4 years ago

just to say , my program is free (and will stay free), it could be used to dialog with any box, with a little modification according to the way of incoming info for the box. I think that will not be a big change, but as i said i not familiar with HA and don't know how to send HTTP or ? info to the right devices in HA. (I know how to do that with domoticz, lifedomus, but not HA )

so if @Th0mC or @peioe or anybody else could give me some help for this part i'm open to make the modification and publish the new page for HA.

peioe commented 4 years ago

To be honest, if Enedis requires an outside service ran by a company to integrate into HA, I think I'm just going to give up and not do it. Maybe I will reconsider in the future if someone gets it working but right now I think what they are doing is not ok and I'd rather not use it at all.

Hopefully you can find someone else to test it, good luck.

Th0mC commented 4 years ago

Just received my clientID and my clientSecret for the sandbox. @jluc2808 I'm ready to help but honestly I don't really know how? I'm a newbie on HA (have juste installed one instance locally). What I understood is that HA is using pyLinky to get info from Enedis. The previous version, with email/password. So first thing would be to make evolve HA to be compatible with the new version of pyLinky. Then get your help to get cientid/clientsecret for my own "prod" data ;). Right?

Pirionfr commented 4 years ago

maybe @NabuCasa can help us on this.

jluc2808 commented 4 years ago

thanks for the reply , how could contact @NabuCasa ?

Pirionfr commented 4 years ago

https://www.nabucasa.com/support/

Quentame commented 4 years ago

Wait wait wait, Nabu-casa does not provide support at all.

See https://www.home-assistant.io/ in the footer Contact (no support!)

Only support here are volunteer, code mainteners or the Discord https://discord.gg/NAKujAp

Quentame commented 4 years ago

Hi again !

So, I've started to work on the new integration : https://github.com/hacf-fr/home-assistant-core/tree/linky/new-api

But I was facing problems with OAuth authentication system.

And sorry I've started on 21th may, but then I realized I have to ask Enedis to create a sandbox app that need about a week to "validate", 5 days after that, my app was validated. I've work on it on 28th of may, but guess what ? I needed to modify my sandbox app and have to be validated again. From this time I didn't work in it ... (don't even know if my app was validated again).

So in https://github.com/hacf-fr/home-assistant-core/tree/linky/new-api :

Once you added your client ID and SECRET to the config.yaml, you can try to add the Linky integration again, but when it should open the OAuth Linky screen, it opens a blank page, close itself and reset HA to HA_URL/lovelace/default_view.

Stuck here for now.

I am working on Home Assistant during most of people sleep or watch Netflix, as a volunteer for this. I am also coding for a company during the day. So I would like to do a lot of things for Home Assistant, but unfortunatly time can't be paused. I was on other integration during this time, sorry.

I will check again during this week or weekend, but don't expect something before 0.112.0 or 0.113.0 !

Thanks for you understanding.

Quentame commented 4 years ago

Reopening the issue to follonw the progress of the fix.

But can't guarantee the integration will work with professionnal accounts.

jluc2808 commented 4 years ago

@quentame , what you discribe with your last post is probably with sandbox environment. Carefull In real production the API auth doesn't react the same, 1 - you need to recieve a new client_id, client_secret which are generated by enedis only if you have already signed a contrat (a real one with company name, kbis, .....) . 2 - you have to create a web page that redirect in real time the linky's owner to the "consentement" URL page, where linky's owner have a checkbox wich allow the access of his data for someone else than himself, this step is manual and couldn't be bypassed and pre-answered. This step is controled by enedis before you get the new secret/id. 3 - the "consentement" URL with the akcnowledge of the user return a code to the redirect_uri specified in the contrat (not the one in the request), this code is used with client_id, client_secret to request to enedis the tokens (access_token, refresh_token, and usage_point_id). 4 - after reception of theses data (tokens, usage_point_id) you have to send these 3 variables to the end_user program. They are used (access_token, usage_point_id) in every request to ask contents of the linky (consumption or production) 5 - As the access_token is valid only for 3h30, after this time, you have to renew the access_token using the refresh_token, so send a request to the program which hold "client_id, client_secret", with refresh_token, access_token and usage_point_id to renew the step 3 (now with refresh rather than with code)

sorry if that description is a headhake but it is now what is needed.

my program takes in charge 1 to 5 step , and let to end_user the charge of contents requests and ask renew token.

Quentame commented 4 years ago

1 - I have it 2 - Yes, but if every user create its account and create an app, I don't think we need it (as Netatmo, Spotify, Somfy ...) 3 - I dont think I understand well 4 - This is done by the library, thanks to Pirionfr/pyLinky#29 5 - Also the library

When you say "my program" it's because you alreay integrate pyLinky 0.5.0 with an other program ?

We can chat on this with Discord if you want and can, today or tomorrow after 18h. I have to same pseudo.

jluc2808 commented 4 years ago

yes better to tchat, i don'know discord , this need an acompt ? otherwise i have zoom, whatsapp, skype, teamviewer ?

jluc2808 commented 4 years ago

@quentame , ok i havn't seen you gave me a link to discord , i'm now registered a connected if you want to chat , my name jluc2808

jluc2808 commented 4 years ago

1 - in production ? 2 - no end_user couldn't create there own app , or need to hijack your own client_id and client_secret , the redirect_uri will still stay the same and manual step with Enedis account is still necessary 3 - as this part is critical , we have to discust 4 - i don't think so with the new api and the redirect_uri 5 - each user need to store his own token , this done by the library ?

Quentame commented 4 years ago

Now I am having Missing code or state parameter in http://localhost:8123/auth/external/callback?code=400&error=Invalid_request&error_description=missing_parameter

Quentame commented 4 years ago

OK guys, thanks to our long video call with @jluc2808, it will be faster than expected ! :tada:

We are working together to bring you a fix on this.

And I think it will be an opportunity to add some new features (let's see what happen next :wink:) ! It can be on 0.112.0, we will need testers on this when at beta stage.

big-muscle commented 4 years ago

Hi !

Thank you for this info ! I was starting to despair ! I would be interested in integrating my Linky electric meter into Home Assistant, but I haven't been able to do it for 2 months ;)

rParslow commented 4 years ago

OK guys, thanks to our long video call with @jluc2808, it will be faster than expected ! 🎉

We are working together to bring you a fix on this.

And I think it will be an opportunity to add some new features (let's see what happen next 😉) ! It can be on 0.112.0, we will need testers on this when at beta stage.

Will be OK to Beta test :)

br0c commented 4 years ago

OK guys, thanks to our long video call with @jluc2808, it will be faster than expected !

We are working together to bring you a fix on this.

And I think it will be an opportunity to add some new features (let's see what happen next ) ! It can be on 0.112.0, we will need testers on this when at beta stage.

Happy to beta test when ready as well, just tried to set it up which brought me up to this opened issue.

rParslow commented 4 years ago

Happy to beta test when ready as well, just tried to set it up which brought me up to this opened issue.

MeToo !

Pol2Tls commented 4 years ago

Happy to beta test when ready as well, just tried to set it up which brought me up to this opened issue.

I can test too !

diyanei commented 4 years ago

I might be able to help and give it a shot but I may need to get Enedis in the loop, and that is not the easy part :-)

Quentame commented 4 years ago

Sorry guys but not available on 0.112 it's harder than expected.

utkiupe commented 3 years ago

I'll be happy to test too. Still not sure why Enedis does not want to share our own data with us directly and require us to work with a service provider! I am quite pissed off actually!

welcoMattic commented 3 years ago

I'm available to test if is necessary. I've got a Linky and an account on https://mon-compte-particulier.enedis.fr I requested an account on https://datahub-enedis.fr/data-connect/ too, I'm waiting for my credentials

plum59 commented 3 years ago

Hello i have an old account in enedis (since 2 years) and i experiment same issu since: 09/07/2020. I use custom linky components from pirion. But i had try to install linky components from official ha repos. And when it Ask to me to provide my user/password during install process, a login error message appear.

yjajkiew commented 3 years ago

Unfortunately I don't have much time to help develop this fix, but I have just received my Enedis Data Connect identifier (sandbox client Id and client secret) and I'm willing to share it with a confirmed developer if necessary in order to help with this problem.

jluc2808 commented 3 years ago

just to make some precision on the Enedis data connect way of use

identifier:
everybody could ask for an identifier to get a client_id / secret_id to access Enedis data connect "sandbox", this open and no restriction is applied to this "test" environment.

it's complitly different if you want to access the production environment, You have to prove that you'are a society, sign a contract beetween the society and Enedis, and write a program tested by Enedis (before the delivery of client_id / secret_id), which point to Enedis "consentement" page where the end-user (the one which own the linky) accept that Enedis give you access to His data. This step couldn't be bypassed and is within a live interaction of the owner (this not the case with sandbox).

So, some (Oauth) programs tested with sandbox aren't the same that the program which are used in a production environment.

As i have made this TRIP (and it is a hard TRIP), i just want to point that everyone who got client_id and secret_id under sandbox environment, couldn't transform this step directly in a production environment.

SeByDocKy commented 3 years ago

just to make some precision on the Enedis data connect way of use

identifier: everybody could ask for an identifier to get a client_id / secret_id to access Enedis data connect "sandbox", this open and no restriction is applied to this "test" environment.

it's complitly different if you want to access the production environment, You have to prove that you'are a society, sign a contract beetween the society and Enedis, and write a program tested by Enedis (before the delivery of client_id / secret_id), which point to Enedis "consentement" page where the end-user (the one which own the linky) accept that Enedis give you access to His data. This step couldn't be bypassed and is within a live interaction of the owner (this not the case with sandbox).

So, some (Oauth) programs tested with sandbox aren't the same that the program which are used in a production environment.

As i have made this TRIP (and it is a hard TRIP), i just want to point that everyone who got client_id and secret_id under sandbox environment, couldn't transform this step directly in a production environment.

Where to ask this new type of account ? I can't see anything in my enedis account ?

jluc2808 commented 3 years ago

just to make some precision on the Enedis data connect way of use

identifier: everybody could ask for an identifier to get a client_id / secret_id to access Enedis data connect "sandbox", this open and no restriction is applied to this "test" environment.

it's complitly different if you want to access the production environment, You have to prove that you'are a society, sign a contract beetween the society and Enedis, and write a program tested by Enedis (before the delivery of client_id / secret_id), which point to Enedis "consentement" page where the end-user (the one which own the linky) accept that Enedis give you access to His data. This step couldn't be bypassed and is within a live interaction of the owner (this not the case with sandbox). So, some (Oauth) programs tested with sandbox aren't the same that the program which are used in a production environment. As i have made this TRIP (and it is a hard TRIP), i just want to point that everyone who got client_id and secret_id under sandbox environment, couldn't transform this step directly in a production environment.

Where to ask this new type of account ? I can't see anything in my enedis account ?

it's not under enedis account , it's under https://datahub-enedis.fr/data-connect/

SeByDocKy commented 3 years ago

Thanks .... for me it's a real scandal !!!!! To sell our data like this and more over to ban our own data from our collect system. Not sure it's constitutional..... I am pretty sure as a client of this new datahub than you can collect data even with unchecking the data transmission in your enedis account.

jluc2808 commented 3 years ago

Thanks .... for me it's a real scandal !!!!! To sell our data like this and more over to ban our own data from our collect system. Not sure it's constitutional..... I am pretty sure as a client of this new datahub than you can collect data even with unchecking the data transmission in your enedis account.

yes with a wired teleinfo

home-assistant / core