502 - Web server received an invalid response while acting as a gateway or proxy server.

[modemgeek]

On one particular webapp, we started getting this 502 error when going to the /letsencrypt/ We haven't made any changes to it. I have also tried upgrade to 0.9.6 (from 0.9.5) but same issue happens. Once you are on /letsencrypt and click Next it hangs for a bit and then the 502 error pops up. Any idea of what is causing this behavior or where I can look for errors?

I can also browse to /letsencrypt/Home/Install and select a hostname. When I click the button request and install certificate, I get the same 502 error.

[modemgeek]

**Update: it looks like it is happening to our webapps in the south region only

[ncote]

Same here. Just noticed it a few minutes ago.

[tdoumas]

We have the same problem here. North Europe. Maybe something changed in Let's Encrypt API. I have found a related question in ServerFault https://serverfault.com/questions/986517/error-creating-letsencrypt-certificate-from-azure-web-site-extension

No solution for the moment

[sjkp]

Hi guys - i will take a look tonight.

[modemgeek]

Thanks. I also wonder if it has anything to do with LE current status

October 3, 2019 01:57 UTC[Investigating] We are investigating user reports for occasional timeouts when accessing the /directory endpoint.

[ak23young]

Hey all, just wanted to add in I'm also receiving the 502 error on my webapp hosted in Central US

[sjkp]

Quick update - I won't be able to release a fix tonight. I moving to a new ACME library that supports the LetsEncrypt V2 API - which is kinda required as the old V1 api is being shutdown first of 1. november and the staging environment is already closed (which could be why you see the errors, the extension at least wont work at all right now with staging certs). I will hopefully be able to release it tomorrow, I just have to do some more testing.

[ncote]

Thank you so much.

On Thu, Oct 3, 2019 at 5:20 PM Simon J.K. Pedersen notifications@github.com wrote:

Quick update - I won't be able to release a fix tonight. I moving to a new ACME library that supports the LetsEncrypt V2 API - which is kinda requires as the old V1 api is being shutdown first of 1. november and the staging environment is already closed (which could be why you see the errors, the extension at least wont work at all right now with staging certs). I will hopefully be able to release it tomorrow, I just have to do some more testing.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/sjkp/letsencrypt-siteextension/issues/331?email_source=notifications&email_token=AAV4CBHZ5TGXKCASBS3YHOTQMZO2ZA5CNFSM4I4YPFD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAJUKCY#issuecomment-538133771, or mute the thread https://github.com/notifications/unsubscribe-auth/AAV4CBASESEUBMYWJL4G2ULQMZO2ZANCNFSM4I4YPFDQ .

[imadsani]

Glad we're not the only ones experiencing this issue, had me worried for a minute.

Region: East Asia

[VicSmith]

Noting the same problem today in the Central US Azure Region.

[tiltsoftware]

Just following up to see if this is close to being resolved?

Thanks so much for your work with this, saved me a lot of time over the last few years.

[sjkp]

You should move to version 1.0.1 then should should be able to renew your certs. If you have any issues please report them.

[VexedSyd]

Good day, I have updated the extension to latest version (clean install) and I am still getting the 502 error. I am not sure if this issue is related to v2 API because I am getting the error before requesting the certificate on the Authentication Settings page, as soon as I click on the Next button.

[modemgeek]

Same here. Upgraded to 1.0.1. Did a restart. Still getting the 502 error

[VicSmith]

Same here. Brand new fresh install of App, then extension. Still getting the same 502 error.

[tiltsoftware]

I'm still also seeing the same behavior with 1.0.1 with the 502 error. Thanks for working on this.

I have about 50 domains on a site so in the meantime I am using this tool to manually create the certificates. Much faster than any other manual process I have found so far and it has bought me a few days. Just in case it helps someone else. https://zerossl.com/free-ssl/

[shane-hall]

I'm getting the same error in Australia South-East. I tried updating the plug-in as well as removing and replacing. I see an error in the Web Job console that might be related;

The configuration is not properly set for the Microsoft Azure WebJobs Dashboard. In your Microsoft Azure Website configuration you must set a connection string named AzureWebJobsDashboard by using the following format DefaultEndpointsProtocol=https;AccountName=NAME;AccountKey=KEY pointing to the Microsoft Azure Storage account where the Microsoft Azure WebJobs Runtime logs are stored.

Please visit the article about configuring connection strings for more information on how you can configure connection strings in your Microsoft Azure Website.

[bclevering]

The same problem here... The POST to /letsencrypt is failing with a HTTP 502 Bad Gateway response on all websites hosted in West Europe. (i don't have any other)

• version 1.0.1

[fredrik-stigsson]

I have the same problem with version 1.0.1. I can click play on the extension but when I try to click on next it takes some minutes and fails with "502 - Web server received an invalid response while acting as a gateway or proxy server."

North Europe

[rutgervanwilligen]

Got the same issue today (West-Europe), but it might be unrelated to the Site Extension version. Yesterday, I successfully renewed a certificate using version 0.9.6. Today, in a different app service, I got a 502 both when using version 0.9.6 and after upgrading to 1.0.1.

[DanielHosseini]

@sjkp, I'm also facing the same issue with 1.0.1. Performing the post to /letsencrypt gives a 502 bad gateway error. Any update?

[sjkp]

Does anyone have a site that they don't mind me getting access to, so I can see this bad gateway error (because it doesn't happen on my sites, so I'm guessing MS is rolling out a patch to their scale units that breaks stuff, but I need to be on one of the updated scale units to reproduce it). You can write me on mail@sjkp.dk

[Jjarrard]

Brilliant extension, and has saved me time and money over the years, just wanted to mention I had the same 502 bad gateway, Central US. Thanks for your hard work!

[sjkp]

If you have the settings stored in app settings already you can skip the first next button, by just going to letsencrypt/home/install If you dont have the app settings setup, then you need to do that without the help from the extension, until I figure out what is prevent it from allowing next to be clicked on the first page.

[stevet26]

Thanks for looking into it and posting a work around. I have added the app setting manually and I can now see the Request and Install Certificate page with my domain in the hostname box.

Sadly, I hit the 502 error again when clicking the "Request and Install Certificate" button.

Let me know if there is anything I can send you to help with this...

[sjkp]

You dont need to provide me access to any other web apps, I have one of my own that is broken now too. I think this is related to the upgrade that is currently rolling out to all sites to support .net core 3.0 :/

I will work on a work around tonight - as I don't expect MS to come with a swift solution to this.

[modemgeek]

Does anyone have a site that they don't mind me getting access to, so I can see this bad gateway error (because it doesn't happen on my sites, so I'm guessing MS is rolling out a patch to their scale units that breaks stuff, but I need to be on one of the updated scale units to reproduce it). You can write me on mail@sjkp.dk

I opened a ticket with MS and they said they are not seeing any errors on their end and they haven't done any updates on their stamps. At first it was limited to our south region webapps, but now I'm seeing it on east region webapps as well. We have a dev webapp that I could give you access to, however the dev webapp is in the west region and not having this issue.

[m4nthys]

If you have the settings stored in app settings already you can skip the first next button, by just going to letsencrypt/home/install If you dont have the app settings setup, then you need to do that without the help from the extension, until I figure out what is prevent it from allowing next to be clicked on the first page.

Hi, could you please send the name of the attributes so I can add it manually in settings?

Thanks a lot! Diego

[sjkp]

@modemgeek i call that BS, because - i tested a fresh web app created friday it worked fine with my updates when I did the final testing yesterday. Then I create a new web app today, after the new bug reports and the exact same extension code on a new web app with the exact same settings, and it fails with this 502 error. I can see that it does throw an internal 500 from the IIS site, when I enable failed request tracing, but there are no error details in that error, and I'm assuming that the 502 is because of timeout between the load balancers that sit in front of Web Apps and the kudu iis process after it failed.

Even better the same error can be reproduced by going to a web job in the kudu portal and attempting to replay an old execution. That also causes a POST request to the kudu site, which run forever and eventually returns a 502.

[modemgeek]

@modemgeek i call that BS, because - i tested a fresh web app created friday it worked fine with my updates when I did the final testing yesterday. Then I create a new web app today, after the new bug reports and the exact same extension code on a new web app with the exact same settings, and it fails with this 502 error. I can see that it does throw an internal 500 from the IIS site, when I enable failed request tracing, but there are no error details in that error, and I'm assuming that the 502 is because of timeout between the load balancers that sit in front of Web Apps and the kudu iis process after it failed. Even better the same error can be reproduced by going to a web job in the kudu portal and attempting to replay an old execution. That also causes a POST request to the kudu site, which run forever and eventually returns a 502.

I don't disagree with you. Trying to convince Microsoft it's an issue on their end or something they caused is extremely difficult. I've been on this path with them many times.

[bistok]

I’m getting this on normal azure sites that don’t use the extension but have heavy load the users are reporting getting the 502 using the site .

On Oct 8, 2019, at 10:58 AM, Simon J.K. Pedersen notifications@github.com wrote:



@modemgeekhttps://github.com/modemgeek i call that BS, because - i tested a fresh web app created friday it worked fine with my updates when I did the final testing yesterday. Then I create a new web app today, after the new bug reports and the exact same extension code on a new web app with the exact same settings, and it fails with this 502 error. I can see that it does throw an internal 500 from the IIS site, when I enable failed request tracing, but there are no error details in that error, and I'm assuming that the 502 is because of timeout between the load balancers that sit in front of Web Apps and the kudu iis process after it failed.

Even better the same error can be reproduced by going to a web job in the kudu portal and attempting to replay an old execution. That also causes a POST request to the kudu site, which run forever and eventually returns a 502.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/sjkp/letsencrypt-siteextension/issues/331?email_source=notifications&email_token=ABYWIVXKQCKRN2NJJXGCF63QNSU3TA5CNFSM4I4YPFD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAUV7AY#issuecomment-539582339, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABYWIVVGA6HMXZJG7476G73QNSU3TANCNFSM4I4YPFDQ.

[tuin007]

Any news about this error? I still get this error when trying to add a domain in let encrypt

[sjkp]

Nothing yet - and I cant fix it myself, I hope Microsoft will return with an answer.

[shanselman]

@sjkp what's up? We are rolling out 3.0 all over and it should be done this week I think. Your extension requires it? (I care because I have 19 sites using it) http://aspnetcoreon.azurewebsites.net/

We appreciate you @sjkp

I'll ask someone on the App Service team.

[modemgeek]

@sjkp what's up? We are rolling out 3.0 all over and it should be done this week I think. Your extension requires it? (I care because I have 19 sites using it) http://aspnetcoreon.azurewebsites.net/ We appreciate you @sjkp I'll ask someone on the App Service team.

Scott, what are the chances Microsoft will include a built in service that lets us use free LE certificates?

[shanselman]

@modemgeek I'll let you know ASAP as that happens.

For now I have engaged engineers on the App Service team and I'll report back soon!

[DylanTusler]

Hey ho, just chiming in to say I also am having this issue with 1.0.1 in Australia South East. This is, by the way, an amazing little tool. Thanks for keeping it running as long as you have.

For what it's worth, my webjob appears to be stuck with an error same as reported above: "The configuration is not properly set for the Microsoft Azure WebJobs Dashboard. In your Microsoft Azure Website configuration you must set a connection string named AzureWebJobsDashboard by using the following format DefaultEndpointsProtocol=https;AccountName=NAME;AccountKey=KEY pointing to the Microsoft Azure Storage account where the Microsoft Azure WebJobs Runtime logs are stored."

though when I check my config it appears to be correct.

[DylanTusler]

I've managed to get my webjobs working in my instance by reinstating the webjobs connection strings. For some reason they are not present in my sites. THIS DOES NOT FIX THE TOOL UI but will fix situations where your certificates are not renewing because the webjob has stopped working.

Do this by selecting an appropriate storage account in Azure portal to use for the webjob (Select "Storage Accounts" from menu bar then choose a storage account in the appropriate subscription/location), then navigate to the "Access Keys" blade, and copy one of the connection strings onto the clipboard.

Now return to the App Service (or website) that you are trying to re-establish SSL for ("App Services" on the left menu, then select an app.) Once there, open the Configuration blade under Settings and add two new connection strings, one called AzureWebJobsDashboard and one called AzureWebJobsStorage. I used the same connection string for both.

Now you can go to the Webjobs blade (also under Settings) and restart the webjob.

That fixed it for me.

[milan-stojanovic]

I've managed to get my webjobs working in my instance by reinstating the webjobs connection strings. For some reason they are not present in my sites. THIS DOES NOT FIX THE TOOL UI but will fix situations where your certificates are not renewing because the webjob has stopped working.

Do this by selecting an appropriate storage account in Azure portal to use for the webjob (Select "Storage Accounts" from menu bar then choose a storage account in the appropriate subscription/location), then navigate to the "Access Keys" blade, and copy one of the connection strings onto the clipboard.

Now return to the App Service (or website) that you are trying to re-establish SSL for ("App Services" on the left menu, then select an app.) Once there, open the Configuration blade under Settings and add two new connection strings, one called AzureWebJobsDashboard and one called AzureWebJobsStorage. I used the same connection string for both.

Now you can go to the Webjobs blade (also under Settings) and restart the webjob.

That fixed it for me.

Yep, that fixed it for me. Thanks! WebJob now works properly.

[Samelandslaget]

I've managed to get my webjobs working in my instance by reinstating the webjobs connection strings. For some reason they are not present in my sites. THIS DOES NOT FIX THE TOOL UI but will fix situations where your certificates are not renewing because the webjob has stopped working.

Do this by selecting an appropriate storage account in Azure portal to use for the webjob (Select "Storage Accounts" from menu bar then choose a storage account in the appropriate subscription/location), then navigate to the "Access Keys" blade, and copy one of the connection strings onto the clipboard.

Now return to the App Service (or website) that you are trying to re-establish SSL for ("App Services" on the left menu, then select an app.) Once there, open the Configuration blade under Settings and add two new connection strings, one called AzureWebJobsDashboard and one called AzureWebJobsStorage. I used the same connection string for both.

Now you can go to the Webjobs blade (also under Settings) and restart the webjob.

That fixed it for me.

This worked for me too. Adding the connection string settings and restarting (actually stopping and starting because there was no Restart button) the web job resulted in the certificate for my site being renewed immediately. (It would've expired in 4 days.) Thank you so much!

[fredrik-stigsson]

I decided to test another Let's encrypt implementation (App Service Acmebot) and it worked. Found it from https://letsencrypt.org/docs/client-options/ and wrote a blog post about the implementation: https://www.annytab.com/setup-lets-encrypt-for-azure-web-apps/

I really hope that this extension starts to work, but the implementation above is an alternative.

[EricHerlitz]

This worked for me too. Adding the connection string settings and restarting (actually stopping and starting because there was no Restart button) the web job resulted in the certificate for my site being renewed immediately. (It would've expired in 4 days.) Thank you so much!

While this may work to renew certificates creating new ones still doesn't work.

I'd also like to know (and this should be specified in the documentation) if the ConnectionStrings are set as appsettings or as connectionstrings.

[modemgeek]

I decided to test another Let's encrypt implementation (App Service Acmebot) and it worked. Found it from https://letsencrypt.org/docs/client-options/ and wrote a blog post about the implementation: https://www.annytab.com/setup-lets-encrypt-for-azure-web-apps/

I really hope that this extension starts to work, but the implementation above is an alternative.

i'm testing this out as well. However, the challenge file gets placed in /site/.well-known folder. The webapp extension places it in /site/wwwroot/.well-known. The problem I have is that the former is not a accessible via HTTP. I also noticed that it created another virtual directory called /.well-known which maps to /site/.well-known and it causes a run time error.

[Petryxasport]

I still have the same issue with .Net V4.7 and with extension: Azure Let's Encrypt (No Web Jobs) v. 1.0.1 as well

[sjkp]

@EricHerlitz you can infact install certificates on a new site too. You dont have to use the UI to setup the app settings, you can add them all manually and then just run the web job as you describe.

The app settings you would need to add are the following:

Key	Value
letsencrypt:Tenant	The tenant name e.g. myazuretenant.onmicrosoft.com
letsencrypt:SubscriptionId	The subscription id
letsencrypt:ClientId	The value of the clientid of the service principal
letsencrypt:ClientSecret	The secret for the service principal
letsencrypt:ResourceGroupName	The name of the resource group this web app belongs to
letsencrypt:ServicePlanResourceGroupName	The name of the resource group with the app service plan that hosts the web app, if the app service plan is in the same plan as the web app, then this property is optional.
letsencrypt:AcmeBaseUri	The url to Let's Encrypt servers e.g. https://acme-v02.api.letsencrypt.org/directory or https://acme-staging-v02.api.letsencrypt.org/directory (defaults to this)
letsencrypt:Email	The Email used for registering with Let's Encrypt
letsencrypt:Hostnames	Comma separated list of custom hostnames (externally hosted setup with CNames), that should automatically be configured for the site.

[EricHerlitz]

@EricHerlitz you can infact install certificates on a new site too. You dont have to use the UI to setup the app settings, you can add them all manually and then just run the web job as you describe.

That I know, and I have those setup already but the web jobs still returns a 502 even if I go directly to /letsencrypt/home/install in kudo

[joshdickerson92]

Firstly, thanks @sjkp this really is a great extension!

I'm having the same issue as above when creating a new certificate and tried the above, however my web job is stuck in "Pending Restart" with the error below:

Any ideas anyone? Thanks in advance!

The configuration is not properly set for the Microsoft Azure WebJobs Dashboard. A connection string named AzureWebJobsDashboard is not well-formed. In your Microsoft Azure Website configuration you must set a connection string named AzureWebJobsDashboard by using the following format DefaultEndpointsProtocol=https;AccountName=NAME;AccountKey=KEY pointing to the Microsoft Azure Storage account where the Microsoft Azure WebJobs Runtime logs are stored.

Please visit the article about configuring connection strings for more information on how you can configure connection strings in your Microsoft Azure Website.

[CarlosAndreuLlaneras]

I decided to test another Let's encrypt implementation (App Service Acmebot) and it worked. Found it from https://letsencrypt.org/docs/client-options/ and wrote a blog post about the implementation: https://www.annytab.com/setup-lets-encrypt-for-azure-web-apps/

I really hope that this extension starts to work, but the implementation above is an alternative.

Thanks @fredrik-annytab, it worked for me. Fortunately, right now my web is ssl secured again. Anyway, I will be watching this issue in order to use Azure Let's Ecript Extension again.

[RobDeVoer]

This worked for me too. Adding the connection string settings and restarting (actually stopping and starting because there was no Restart button) the web job resulted in the certificate for my site being renewed immediately. (It would've expired in 4 days.) Thank you so much!

While this may work to renew certificates creating new ones still doesn't work.

I'd also like to know (and this should be specified in the documentation) if the ConnectionStrings are set as appsettings or as connectionstrings.

@EricHerlitz Set them as connection strings with type other

[m4nthys]

@EricHerlitz you can infact install certificates on a new site too. You dont have to use the UI to setup the app settings, you can add them all manually and then just run the web job as you describe.

The app settings you would need to add are the following:

Key Value letsencrypt:Tenant The tenant name e.g. myazuretenant.onmicrosoft.com letsencrypt:SubscriptionId The subscription id letsencrypt:ClientId The value of the clientid of the service principal letsencrypt:ClientSecret The secret for the service principal letsencrypt:ResourceGroupName The name of the resource group this web app belongs to letsencrypt:ServicePlanResourceGroupName The name of the resource group with the app service plan that hosts the web app, if the app service plan is in the same plan as the web app, then this property is optional. letsencrypt:AcmeBaseUri The url to Let's Encrypt servers e.g. https://acme-v02.api.letsencrypt.org/directory or https://acme-staging-v02.api.letsencrypt.org/directory (defaults to this) letsencrypt:Email The Email used for registering with Let's Encrypt letsencrypt:Hostnames Comma separated list of custom hostnames (externally hosted setup with CNames), that should automatically be configured for the site.

I've added all the app settings manually and got the same error after clicking "Request and install certificate" in the following URL: https://icollectprod.scm.azurewebsites.net/letsencrypt/home/install

=/