PipedreamHQ / pipedream

Connect APIs, remarkably fast. Free for developers.
https://pipedream.com
Other
9.01k stars 5.28k forks source link

[APP] Enedis #4112

Open LucBerge opened 2 years ago

LucBerge commented 2 years ago

Name of app / service Enedis

Link to developer documentation Data-connect Official documentation

Is lack of support preventing you from building workflows, or do you have a workaround? No

Are there specific actions, or triggers, you'd like to see for this app? Please let us know here or use the Action and Trigger issue templates to open requests for each!

Action:

dylburger commented 2 years ago

Thanks for the request! Did you know that anyone can author and publish their own Pipedream actions and sources? Would you be interested in helping out with that? We're happy to answer any questions or support you in any way.

If you don't have the time, would you be willing to reach out to the app provider to see if they're interested in building these? When we work with an app partner, it improves the quality of the integration, and it helps when this kind of request comes from a customer like you.

dylburger commented 2 years ago

Thanks for the request! We've added this to our integrations backlog, and we'll update this ticket with progress.

sergioeliot2039 commented 2 years ago

Marking app integration for now as blocked because the API is documented around the a node.js SDK. Perhaps the Pipedream team can add separately from the app submit form, which doesn't allow custom code.

LucBerge commented 2 years ago

@sergioeliot2039 I found the full API doc but you need an account. Also, for the authorization request, you need an application string when requesting a new token. This string is obtained by asking to Enedis. Maybe you should do it, you are in better position to explain an defend the project. I did it too, I'll keep you in touch.

Edit : Do not close this issue for inactivity. The application creation is pending on Enedis side.

LucBerge commented 2 years ago

You must be able to authenticate via the Authorize API.

@sergioeliot2039 Is it now possible to create the app ? Thanks

Regarding this repo, the response_type value must be code and duration can be up to P3Y (3 years).

dannyroosevelt commented 2 years ago

Thanks @LucBerge I can take a look at this, but we'll need to provide a redirect_uri and acquire a client_secret for the Pipedream OAuth app as well (which should not be pasted here on this issue).

dannyroosevelt commented 2 years ago

FYI I also just submitted a request to create an OAuth app.

LucBerge commented 2 years ago

APP UPDATE

Great work have been made thanks to @dannyroosevelt. Integrating this app is a mess of administrative tasks !

The APP is currently in sandbox mode which means the data we can access are fake. The purpose is to validate the code while wainting for the APP to go production mode.

To go production mode pipedream need to fill this form in order to be granted to access real data. As a US based company, you must select Autre for the field Type entité. In step 1 of 3, they are asking for any document which can be used as the ID of the company (I guess the idea is to prove your company exists and being able to find you for any problem)

ACTIONS UPDATE

I've done the components in local. I'm wainting for the App to work in sandbox mode to test it and make a PR.

BUG

For now, I have a troubles to connect my account in sandbox mode:

On my workflow, when I try to connect my account (1), a connection window is open (2). The redirection to pipedream is not working. Pipedream support already have acces to my workflow : https://pipedream.com/@LucBerge/test-enedis-p_q6CnmyB

The url is the following https://gw.hml.api.enedis.fr/dataconnect/v1/oauth2/authorize?client_id=c8a45b5b-789c-4987-8d85-fb00a427bf08&duration=P6M&redirect_uri=https%3A%2F%2Fapi.pipedream.com%2Fconnect%2Foauth%2Foa_gK1i37%2Fcallback&response_type=code&scope=&state=dV9yOWhiYlZnfDdkMGIzMWRlMGYwOTcxMmFkOTU3ZDhhZTRiYTEzYzdl

If I copy the link and open it in a regular browser tab, the redirection works but I have the following pipedream error:

Pipedream failed to connect to your account.

Please reach out to integration-errors@pipedream.com and provide this error ID: e913d677f0634b1db10dc7aea4c314ed

Strange behaviour going on here

LucBerge commented 2 years ago

BUG

A different bug occurs with Microsoft Edge: The main page is redirected while the small one no. Here is a video

The error code is e35a01029c004a36a00a4e603501d2f2

LucBerge commented 1 year ago

Hello, any news about this bug?

dannyroosevelt commented 1 year ago

Hey @LucBerge apologies for the delay getting back to you. Troubleshooting anything with the Enedis app is difficult since their servers are only available 9-5 Paris time, and the majority of our team is in US Pacific time, so we cannot test or debug anywhere close to our working hours.

FYI I am still going back and forth with their team re: approving our app.

LucBerge commented 1 year ago

I don't understand why you say their servers are only available 9-5. Where did you find this info ? I don't have troubles to connect (at 1am) with a python script I've made (see slack)

dannyroosevelt commented 1 year ago

Maybe this was only re: creating the application? But when I test their Sandbox Authorization endpoint right now I get a 503 Service Unavailable error.

Screenshot by Dropbox Capture

Screenshot by Dropbox Capture

LucBerge commented 1 year ago

Mmm ok, I didn't know that. Maybe you can setup a pipedream workflow triggered on cronjob to test and debug the api calls.

Will it help you if I do it on my workflow ?

dannyroosevelt commented 1 year ago

That's a good idea, but I really need to be able to debug while testing. By the way I made a slight change to the OAuth Authorization URL yesterday (but I just can't validate that it made any difference...)

dannyroosevelt commented 1 year ago

@LucBerge unfortunately we're blocked on this again -- they approved our corporate verification, but they require we sign a contract to proceed and they are not able to share in English, which is a blocker for us right now. Screen Shot 2022-12-02 at 9 33 32 AM

sergioeliot2039 commented 1 year ago

@dannyroosevelt jumping in with some input and digress on related language barrier when it comes to Project App Store

Perhaps this is a case where it could be helpful to reach out for a "sworn translation" of the contract. for this it is needed to reach a French to English translator with a government issued certification on translation document for legal purposes (a "sworn translator"). I totally get this is an unusual Pipedream activity but could be helpful for the matter of make sure the contract is signed while the appropriate understanding of its content. Not very time efficient, but better than Pipedream being on the need to have legal representation in France to sign that.

Now, I recently requested API access to a service that is based in France too. I sent then over request in French and they denied, and... wait for it, invited to create an account in another service! 😅

image

"we are sorry, but this decision is final, we invite you to create an account in another service..."

That said, I know there is Google Translation to get what the contract says in English, but maybe I can give faith on what is says (and probably @LucBerge too ), if that's of helpful. My two cents

LucBerge commented 1 year ago

they require we sign a contract to proceed and they are not able to share in English

...

That said, I know there is Google Translation to get what the contract says in English, but maybe I can give faith on what is says (and probably @LucBerge too ), if that's of helpful. My two cents

If I can help you in anyway, just ask. I can translate !

LucBerge commented 1 year ago

@dannyroosevelt I'm able to retrieve daily consumption with sandbox endpoint on my workflow: https://pipedream.com/@LucBerge/test-enedis-p_q6CnmyB.

To make it work correctly, you will need to change some params in the requests:

In step 1

In step 2

In step 3

Once step1, 2 and 3 are complete, the Bearer token received in step 3 is available 3 years as requested in step 1. It means the following step 4 can be done as much as you need without requesting a new token.

dannyroosevelt commented 1 year ago

Thanks for the prototype!

  1. The Authorization URL is not responding right now -- can you try on your side? See below screenshot
  2. I just made an update to the app configuration on our end, which will permit you to enter your own Client ID and Secret. This will help bypass the current contractual obligations that are slowing down the app release for the time being, since I think your OAuth app has been approved for production, is that right?

Screenshot by Dropbox Capture

LucBerge commented 1 year ago
  1. Not responding either
  2. No my app cannot go production because I cannot sign the contract as a single individual. Thank you, I'll try tomorrow. If it's working I'll PR the actions
LucBerge commented 1 year ago

When I try to connect my Enedis account with my client_id and client_secret a window pops-up (while it should not with your bypass) with a JSON response

dannyroosevelt commented 1 year ago

Sorry, I don't quite follow -- are you saying the Firefox window in the bottom of your screenshot opened automatically? You did not open that window?

LucBerge commented 1 year ago

Yes, the window in the bottom opened automatically when clicking continue

dannyroosevelt commented 1 year ago

Interesting. This is what I see when I test in Firefox: Screenshot by Dropbox Capture

So you do not get that standard OAuth popup window?

LucBerge commented 1 year ago

I have this too now, because the server is off by this time

Edit: I haven't changed the redirection_uri...

LucBerge commented 1 year ago

Note that they just released a new API. The endpoints we are using are still available but will expire on the 30/04/2023.

Something interesting: Regarding this page and this one which (has been updated), we no longer need the authorization code in sandbox

Les applications actuelles en bac à sable ne sont pas utilisables avec les nouvelles API, il faut donc en créer une nouvelle

Current applications in sandbox mode are not usable with the new API, you need to create a new one

dannyroosevelt commented 1 year ago

Now that we've modified the app's configuration to use your own client ID and client secret, there shouldn't be any changes required for Pipedream in terms of creating a new application.

Let me know when you want to update the Authorization and Access Token URLs and we can do that.

LucBerge commented 1 year ago

@dannyroosevelt Yes you have some changes to do:

In sandbox mode: You don't need to call the authorization endpoint anymore (sandbox mode only). You need to change the token request:

POST https://ext.hml.api.enedis.fr/oauth2/v3/token

Params

redirect_uri: https://api.pipedream.com/connect/oauth/oa_gK1i37/callback

Body (application/x-www-form-urlencoded)

grant_type: client_credentials
client_id: <clinet_id>
secret_id: <secret_id>

I've done an other workflow to test the new API. I wasn't able to test it because my new app will be created tomorrow

dannyroosevelt commented 1 year ago

Got it, thanks. I just made the changes to the OAuth configuration on our end -- let me know how that works for you when you get your new app configured!

LucBerge commented 1 year ago

Not working. A secure window is opened with the following url:

https://api.pipedream.com/connect/oauth/oa_gK1i37?app_id=app_13GhwZ&&&connect_start_ms=1670913562005&cfmap[client_id]=5750d3d9-2542-4a55-a5b5-10ee0f095350&cfmap[client_secret]=mysecretkey

The response is as follow:

Pipedream failed to connect to your account. Please reach out to integration-errors@pipedream.com and provide this error ID: f7a6234f230f4b0bb06f21e4c14a3145

I noticed a weird thing in the url: Between the first and second parameter, there is &&&. I'm not sure it is the problem

dannyroosevelt commented 1 year ago

I fixed the issue that returned that specific error ID. To move forward, I just have a couple questions re: structuring the test request that is currently configured to https://ext.hml.api.enedis.fr/metering_data_dcmp/v5/daily_consumption_max_power, which requires 3 params — start, end, usage_points_id.

Do you know if there is a way to programmatically retrieve the usage_points_id? They used to return that from the Access Token Request but I guess they removed that field. That’s why the auth flow was breaking for you.

LucBerge commented 1 year ago

Do you know if there is a way to programmatically retrieve the usage_points_id?

With the new API, yes: https://datahub-enedis.fr/data-connect/ressources/production/ But it's only available in production on the Authorize request. In sandbox mode we can put whatever we want.

Obtenir les numéros de PRM du client

Une fois que le client a accepté de partager ses données, le navigateur le redirigera vers la page de votre choix, celle indiquée comme page de redirection au moment de la création de votre application. Dans l’URL se trouve les numéros de PRM que vous devez récupérer pour la suite des opérations.

Le state est renvoyé avec la même valeur que vous aviez envoyée. Un code d’autorisation est également transmis : vous pouvez ignorer cette valeur.

Exemple de réponse: https://redirect.com/redirect?state=XYZ&usage_points_id=xxxxx,yyyyyy&code=134567281

dannyroosevelt commented 1 year ago

Ok, feel free to try connecting the app again, and let me know if you're successful.

LucBerge commented 1 year ago

It's working ! PR for actions coming soon.

Would it make sense to have one enedis app for the sandbox mode and one for production ?

LucBerge commented 1 year ago

@dannyroosevelt Btw, new endpoints are available 24/7 now.., Better late than never.

Do you need help to translate the contract to go production?

dannyroosevelt commented 1 year ago

Would it make sense to have one enedis app for the sandbox mode and one for production ?

Maybe it can be handled within a single app -- what's the difference, just the host?

--

Do you need help to translate the contract to go production?

I'm checking with the team on how to handle next steps on the contract

LucBerge commented 1 year ago

what's the difference, just the host?

No.

Sandbox:

Production:


There are differences for the actions as well:

Sandbox:

Production:


Actions in sandbox mode are done but I can't verify two of them. Two endpoints return an error 500 : Technical error. Please try later. since three days...

I'm waiting for a 200 status before PR

LucBerge commented 1 year ago

Ok they told me by email that the usage point I use (my house) does not have production enabled so it's normal... I can use the following virtual usage point instead: 12655648759651

My actions are working fine, PR coming


@dannyroosevelt Should we have two apps? One for the sandbox, one for the production?

LucBerge commented 1 year ago

@vunguyenhung @dannyroosevelt The merge automatically closed this issue, while it should not be closed. The app is running in sandbox mode not production yet.


How is the contract translation going on?

vunguyenhung commented 1 year ago

Thank you for informing us @LucBerge! I've reopened the issue, though I will also move this in the Blocked column until this is deployed to prod

LucBerge commented 1 year ago

Thank you for reopening the issue.

Why do you consider it as blocked?

dannyroosevelt commented 1 year ago

@LucBerge unfortunately the contract is still on hold due to the language barrier.

LucBerge commented 1 year ago

@LucBerge unfortunately the contract is still on hold due to the language barrier.

@dannyroosevelt How can I help ?

dannyroosevelt commented 1 year ago

It's on the backlog to address but we're a small team and are focused on other priorities right now.

LucBerge commented 1 year ago

Sure, I understand

sergio-eliot-rodriguez commented 7 months ago

https://pipedream.com/apps/enedis

LucBerge commented 7 months ago

@sergio-eliot-rodriguez why did you move it from Blocked to Done? Why is the issue closed?

sergio-eliot-rodriguez commented 7 months ago

@LucBerge , I see that Enedis has been integrated -- https://pipedream.com/apps/enedis was the wrong application integrated?

LucBerge commented 7 months ago

@LucBerge , I see that Enedis has been integrated -- https://pipedream.com/apps/enedis was the wrong application integrated?

This ticket is not only about app integration. The app is in sandbox mode and need to go to production. In order to go to production, pipedream need to sign a contract with Enedis. This is the reason why it is tagged as blocked.

Please read the full thread.