Durable Function on App Service Linux returns HTTP based APIs that query the orchestrator function status. (Expected HTTPS based APIs)

[fasigpt]

Description

When the durable function is hosted on App Service LINUX it returns HTTP based APIS to query the status. However, on windows ASP it works fine and returns HTTPS-based API.

We are referring to REST command APIS returned to query the function status

https://docs.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-overview?tabs=csharp#async-http

Expected behavior

Actual behavior

Relevant source code snippets

public static class Function1
    {
        [FunctionName("Function1")]
        public static async Task<List<string>> RunOrchestrator(
            [OrchestrationTrigger] IDurableOrchestrationContext context)
        {
            var outputs = new List<string>();

            // Replace "hello" with the name of your Durable Activity Function.
            outputs.Add(await context.CallActivityAsync<string>("Function1_Hello", "Tokyo"));
            outputs.Add(await context.CallActivityAsync<string>("Function1_Hello", "Seattle"));
            outputs.Add(await context.CallActivityAsync<string>("Function1_Hello", "London"));

            // returns ["Hello Tokyo!", "Hello Seattle!", "Hello London!"]
            return outputs;
        }

        [FunctionName("Function1_Hello")]
        public static string SayHello([ActivityTrigger] string name, ILogger log)
        {
            log.LogInformation($"Saying hello to {name}.");
            return $"Hello {name}!";
        }

        [FunctionName("Function1_HttpStart")]
        public static async Task<HttpResponseMessage> HttpStart(
            [HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequestMessage req,
            [DurableClient] IDurableOrchestrationClient starter,
            ILogger log)
        {
            // Function input comes from the request content.
            string instanceId = await starter.StartNewAsync("Function1", null);

            log.LogInformation($"Started orchestration with ID = '{instanceId}'.");

            return starter.CreateCheckStatusResponse(req, instanceId);
        }
    }

Known workarounds

The issue can be reproduced locally as well. However on local issue can be fixed with forcing "host start --useHttps"

App Details

• Durable Functions extension version (e.g. v1.8.3): 2.1.1
• Azure Functions runtime version (1.0 or 2.0): 3
• Programming language used: c#

[ConnorMcMahon]

I believe I know what is going on here.

On Azure App Service, we terminate SSL requests on the front end, so apps have to look at the X-Forwarded headers to determine whether the request was HTTP or not. You can read about this here.

If your application is using the CORS or Authentication/Authorization feature, we use an intermediate container that proxies requests to your application. There is currently a bug where this container is consuming the X-Forwarded headers, meaning the function app can't use those to determine whether the request is HTTPS or not. This bug has been fixed and will be rolled out in the next App Service update in the next month or two.

You can test whether this is the case by removing all of the Allowed Origins under the CORS plane. These allowed origins should not be required, as they are legacy settings for portal compatibility that should no longer be necessary.

[fasigpt]

@ConnorMcMahon CORS blade is empty. Still get the HTTP URIS

[ConnorMcMahon]

Interesting.

@fasigpt, I have validated that your app is not running the intermediate container, that is not the problem. I am curious if the Functions host is not properly looking at these X-Forwarded headers. If that is the case, this is an issue we should file against the azure-functions-host repository, as opposed to here.

We can investigate this on our end, but if you have the time/resources, you could validate this by looking at the web request of an HTTP trigger to your application and validate whether it looks like HTTP or HTTPs.

[fasigpt]

thanks @ConnorMcMahon looks like the x-fwd header is HTTPS. I checked the KUDU and also loged the header

[wflemingnz-retired]

Hi

@fasigpt (who is a MS tech) logged this issue on my behalf following a support request.

Following the above advice I have also tried with no CORS Allowed Origins, but it did not make a difference.

A couple of other points which may be relevant:

1) I am 99% sure that the correct URLs were being returned around a week ago.

2) While @fasigpt is seeing this incorrect behaviour under a Consumption plan, I am only seeing it under a Premium Plan. If I deploy the same functions to a Consumption plan, it works as expected.

3) Looking at the source code, CreateCheckStatusResponse calls GetClientResponseLinks which looks like it just takes the first part of request.RequestUri and appends the other URL endings to it.

I have put a log message inside my trigger to output req.RequestUri and when called via https:

• under Consumption plan it shows https
• under Premium plan it shows http

So coming back to @ConnorMcMahon comment that SSL requests are terminated on the front end - does this only apply for Premium plans? And if it does, given the source code mentioned above, how can it determine that https was in effect?

Thanks

[wflemingnz-retired]

@ConnorMcMahon do you have any progress update on this? Thanks.

[ConnorMcMahon]

@wflemingnz,

It turns out for Linux Consumption, the Azure Functions host already has middleware to handle reading the X-Forwarded-Proto header. They do not have anything similar for Linux Dedicated or Linux Premium offerings.

Thankfully, there is a workaround. Just set the app setting ASPNETCORE_FORWARDEDHEADERS_ENABLED to True, and that worked for my application.

I am going to open up a bug against the Azure Functions host, and close this issue, as this is a broader Azure Functions problem rather than Durable Functions.

[ConnorMcMahon]

Filed issue here: https://github.com/Azure/azure-functions-host/issues/6602.

[wflemingnz-retired]

Thanks very much, I used the workaround and it is now returning https URLs.

[daharmon]

I ran into this issue as well and this helped. Can this workaround get added to the microsoft docs page for durable functions? I spent countless hours trying debug and find a solution to this problem.

https://docs.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-create-first-csharp?pivots=code-editor-visualstudio

[ConnorMcMahon]

While this is a bug for Azure Functions, it is unlikely that they will be able to change their default behavior based on the initial conversations there.

I think we can work around this by adding some logic to check for X-Forwarded-Proto headers in HttpApiHandler.GetClientResponseLinks, and it should be straightforward to fix.

[nnekaede]

@ConnorMcMahon Hello. I am also facing this issue. Is there a solution yet?

I am trying to trigger a python based Azure function through logic app, but I am getting this error. I did try to add app setting ASPNETCORE_FORWARDEDHEADERS_ENABLED = True but nothing changed. Also I do have HTTPS Only set to On; however, I am unable to turn it off.

[esttenorio]

ASPNETCORE_FORWARDEDHEADERS_ENABLED=true worked for me but it wasn't obvious until I reached this stackoverflow thread: https://stackoverflow.com/questions/64615864/301-permanent-redirect-when-using-managedserviceidentity-from-logic-app-to-azure

This is still an issue with Linux Function Apps, please add it to the documentation or fix it.