Azure / azure-functions-dotnet-worker

Azure Functions out-of-process .NET language worker
MIT License
428 stars 182 forks source link

The gRPC channel URI 'http://:' could not be parsed. #2121

Closed wheelie33 closed 10 months ago

wheelie33 commented 11 months ago

Azure function worked yesterday with zero issues. I made a small change to add a new model field to my function code, published, and now I only get this exception.

Investigative information

Please provide the following:

Works fine when running locally but not after any publish. Worked yesterday

Repro steps

I replay a message from my queue and I see the invocation error in the logs.

Expected behavior

When a new message is queued, the function should invoke without error and process the message like it did yesterday.

Actual behavior

https://gist.github.com/wheelie33/10625a57637aca4de1ba9da219f63404

Related information

host.json:

{
    "version": "2.0",
  "logging": {
    "logLevel": { "default": "Information" },
    "fileLoggingMode": "always",
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true,
        "excludedTypes": "Request"
      },
      "enableLiveMetricsFilters": true
    }
  }
}

program.cs

using AzureMessageFunctionsEf.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;

using var loggerFactory = LoggerFactory.Create(builder =>
{
    builder.AddConsole();
});

var logger = loggerFactory.CreateLogger<Program>();

try
{
    var config = new ConfigurationBuilder()
            .AddJsonFile("local.settings.json", optional: true)
            .AddEnvironmentVariables()
            .Build();

    var connString = config.GetConnectionString("Default");

    var host = new HostBuilder()
        .ConfigureAppConfiguration(builder => {
            builder.AddEnvironmentVariables();
            builder.AddJsonFile("local.settings.json", optional: true);

        })
            .ConfigureServices(services =>
            {

                services.AddDbContext<DatabaseContext>(options => options.UseSqlServer(connString, options =>
                {
                    options.EnableRetryOnFailure();
                }));
            })
        .ConfigureFunctionsWorkerDefaults()
        .Build();

    await host.RunAsync();
}
catch(Exception ex)
{
    logger.LogCritical("JW: " + ex.Message);
}

Packages:

<ItemGroup>
    <PackageReference Include="Azure.Communication.Email" Version="1.0.1" />
    <PackageReference Include="Azure.Communication.Sms" Version="1.0.1" />
    <PackageReference Include="Azure.Messaging.ServiceBus" Version="7.17.0" />
    <PackageReference Include="Azure.Storage.Blobs" Version="12.19.1" />
    <PackageReference Include="Microsoft.Azure.Functions.Extensions" Version="1.1.0" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker" Version="1.20.0" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.ServiceBus" Version="5.14.1" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Sdk" Version="1.16.2" />
    <PackageReference Include="Microsoft.EntityFrameworkCore" Version="7.0.13" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="7.0.13" />
  </ItemGroup>
  <ItemGroup>

Azure Configuration


[
  {
    "name": "APPLICATIONINSIGHTS_CONNECTION_STRING",
    "value": "InstrumentationKey=06acf3bf-efda-4b61-ba72-760374208516;IngestionEndpoint=https://southcentralus-3.in.applicationinsights.azure.com/;LiveEndpoint=https://southcentralus.livediagnostics.monitor.azure.com/",
    "slotSetting": false
  },
  {
    "name": "AzureWebJobs.Function1.Disabled",
    "value": "0",
    "slotSetting": false
  },
  {
    "name": "AzureWebJobsStorage",
    "value": "DefaultEndpointsProtocol=https;AccountName=scmessageworkerfunctions;AccountKey={REDACTED};EndpointSuffix=core.windows.net",
    "slotSetting": false
  },
  {
    "name": "BlobConnectionString",
    "value": "DefaultEndpointsProtocol=https;AccountName=scoutconnectstore;AccountKey={REDACTED};EndpointSuffix=core.windows.net",
    "slotSetting": false
  },
  {
    "name": "CommunicationConnectionString",
    "value": "endpoint=https://sc-production-comm-service.unitedstates.communication.azure.com/;accesskey={REDACTED}",
    "slotSetting": false
  },
  {
    "name": "FromEmail",
    "value": "{REDACTED}",
    "slotSetting": false
  },
  {
    "name": "FromNumber",
    "value": "{REDACTED}",
    "slotSetting": false
  },
  {
    "name": "FUNCTIONS_EXTENSION_VERSION",
    "value": "~4",
    "slotSetting": false
  },
  {
    "name": "FUNCTIONS_WORKER_RUNTIME",
    "value": "dotnet-isolated",
    "slotSetting": false
  },
  {
    "name": "SCM_DO_BUILD_DURING_DEPLOYMENT",
    "value": "0",
    "slotSetting": false
  },
  {
    "name": "ServiceBusConnectionString",
    "value": "Endpoint=sb://scoutconnect.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey={REDACTED}",
    "slotSetting": false
  },
  {
    "name": "WEBSITE_USE_PLACEHOLDER_DOTNETISOLATED",
    "value": "1",
    "slotSetting": false
  }
]```
wheelie33 commented 11 months ago

Bump for visibility...

wheelie33 commented 11 months ago

Bump again.

wheelie33 commented 10 months ago

Can anyone help here? I even created a new Azure resource, brand new project from fully updated Visual Studio. Deploying has the same issue. This is getting critical.

kshyju commented 10 months ago

@wheelie33 Do you see any errors in your AppInsights logs? Can you confirm you have an app setting called ServiceBusConnectionString in azure with the service bus connection string as the value of it?

mikkovor commented 10 months ago

I was having the exact same problem with pretty much identical settings and packages. I updated Microsoft.Azure.Functions.Worker.Sdk from 1.6.2 to 1.6.3 and that maybe was the fix that worked for me. 1.6.3 is broken when using Xunit as described here: https://github.com/Azure/azure-functions-dotnet-worker/issues/2114

1.6.4-preview1 is what I ended up with for now.

Could be a coincidence since i did a few other things as well but worth a try

EDIT. Still getting that same error at times, so this doesn't fix it.

wheelie33 commented 10 months ago

Can anyone help here? I even created a new Azure resource, brand new project from fully updated Visual Studio. Deploying has the same issue. This is getting critical.

Please see the link to the error gist from my original post.

https://gist.github.com/wheelie33/10625a57637aca4de1ba9da219f63404

wheelie33 commented 10 months ago

ServiceBusConnectionString

Yes, please read my original post.

Error Gist, references to Azure config, etc. all in the OP

kshyju commented 10 months ago

We're currently investigating the issue. In the meantime, if you are experiencing this issue only in Azure, please add/update the below app setting entry under Settings -> Environment variables -> App settings

Name

WEBSITE_USE_PLACEHOLDER_DOTNETISOLATED

Value

0

wheelie33 commented 10 months ago

We're currently investigating the issue. In the meantime, if you are experiencing this issue only in Azure, please add/update the below app setting entry under Settings -> Environment variables -> App settings

Name

WEBSITE_USE_PLACEHOLDER_DOTNETISOLATED

Value

0

Changing that value from 1 (it's default) to 0 seems to have made it work. Thank you.

Qualizorg commented 10 months ago

.NET 6 Isolated Process Azure Function, upgraded Azure Functions Worker + Worker SDK nuget packages to latest version. Working now.

Wangor commented 10 months ago

Any updates on this?

Wangor commented 10 months ago

Still not working?

dzejsien commented 9 months ago

it still doesn't work for us - do you have updates? @kshyju we did

harrisonbs commented 9 months ago

I'm getting the same error message with my solution running locally (I haven't gotten to deploying yet). .Net 8 isolated model.

host.Run()

is throwing

System.InvalidOperationException: "The gRPC channel URI 'http://:' could not be parsed."

circus-dan commented 9 months ago

The problem seems to be in the config file of the DependencyInjection project. It looks like the expected keys don't exist or don't have a value.

bug_dependency_injection

kshyju commented 9 months ago

@circus-dan Would you mind sharing the repro steps where you got into that situation? Do you have a minimal repro app we can take a look at?

harrisonbs commented 9 months ago

I have info on my issue here

https://stackoverflow.com/questions/77837962/i-am-getting-an-error-when-attempting-to-start-my-azure-function-app-net-8-iso/77838423

I seem to have narrowed it down to adding the configuration (IConfiguration) as a Singleton...

harrisonbs commented 9 months ago

@kshyju - in my case, I solely have a ServiceBus function. What is the function above doing? Is this related to hosting the Function App, or to setting up the endpoint for an HTTP triggered one? Because if it is the second, that would be a bug if there are no functions using that trigger method and therefore HOST and PORT are not set.

Qualizorg commented 9 months ago

I do have same issue nevertheless with .NET 6 azure functions app. When I use dotnet publish command and I create an artifact, it weights 16MB, I publish the artifact, I do get 500 gRPC channel URI error, nevertheless when I publish the artifact from the outputs from dotnet build that weights 100MB+ I do not get any errors. Guidance would be appreciated.

Mathih13 commented 9 months ago

I'll post my solution here in case anyone stumble on this looking for answers. I was having an issue with including IConfiguration as a service, effectively overwriting the default Config dependency injection. This didn't seem to be a problem before doing an upgrade to NET 8.

I moved the config import of "local.settings.json" from within ConfigureServices to ConfigureAppConfiguration where it should ideally add on to the config instead of overwriting. That got me past the gRPC error.

Before

var configuration = new ConfigurationBuilder()
              .SetBasePath(Directory.GetCurrentDirectory())
              .AddJsonFile("local.settings.json", optional: true, reloadOnChange: true)
              .Build();

var host = new HostBuilder()
    .ConfigureFunctionsWebApplication()
    .ConfigureServices(s =>
    {
        s.AddSingleton<IConfiguration>(configuration);
        s.AddScoped<ITestService, TestService>();
    }) ...

After

var configuration = new ConfigurationBuilder()
              .SetBasePath(Directory.GetCurrentDirectory())
              .AddJsonFile("local.settings.json", optional: true, reloadOnChange: true)
              .Build();

var host = new HostBuilder()
    .ConfigureFunctionsWebApplication()
    .ConfigureServices(s =>
    {
        s.AddScoped<ITestService, TestService>();
    })
    .ConfigureAppConfiguration(s => s.AddConfiguration(configuration))
    .Build();
milen-denev commented 8 months ago

I got the same error, this is not fixed.

4nderss commented 8 months ago

Same issue appears during local debug after upgrading to net 8.

<PackageReference Include="Azure.Data.Tables" Version="12.8.2" />
<PackageReference Include="Azure.Extensions.AspNetCore.Configuration.Secrets" Version="1.3.0" />
<PackageReference Include="Azure.Storage.Blobs" Version="12.19.1" />
<PackageReference Include="Azure.Storage.Queues" Version="12.17.1" />
<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.ServiceBus" Version="5.16.0" />
<PackageReference Include="Microsoft.Extensions.Caching.StackExchangeRedis" Version="8.0.1" />
<PackageReference Include="Microsoft.Extensions.Configuration" Version="8.0.0" />
<PackageReference Include="Microsoft.Azure.Functions.Worker" Version="1.21.0" />
<PackageReference Include="Microsoft.Azure.Functions.Worker.Sdk" Version="1.16.4" />
<PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="8.0.0" />
<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Storage" Version="6.3.0" />
<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Timer" Version="4.3.0" />
<PackageReference Include="System.Linq.Async" Version="6.0.1" />
var host = new HostBuilder()
       .ConfigureFunctionsWorkerDefaults()
       .ConfigureAppConfiguration((hostingContext, config) =>
       {
           var builder = WebApplication.CreateBuilder();
           config.AddJsonFile(
               "appsettings.json", optional: false, reloadOnChange: false);
           config.AddJsonFile(
               "appsettings.local.json", optional: true, reloadOnChange: true);
           config.AddEnvironmentVariables();
           config.AddAzureKeyVault(
               new Uri(builder.Configuration["Configuration:KeyVaultUrl"]),
               new DefaultAzureCredential());
       })
       .ConfigureServices(builder => ...omitted DI)
       .Build();
host.Run();
rohncarlson commented 8 months ago

This is not really a fix but may be a workaround to help other get by, I had the same issue after installing .NET 8 core along side of Visual Studio 2022 to allow us to migrate our Azure functions from .NET 7 isolated to .NET 8 isolated. Uninstalling and reinstalling did not help and it even broke the .NET 7 function app . I was finally able to get .NET 7 working again by adding .NET Framework 4.8.1. development tools in Visual Studio. After installing, .NET 7 was working again but .NET 8 through the error, "There is no Functions runtime available that matches the version specified in the project" even though .NET 8 was still installed. To solve this issue and get .NET 8 working, I had to go to Tools -> Options -> Projects & Solutions -> Azure functions. Then check for updates and install updates.

harrisonbs commented 8 months ago

This is not really a fix but may be a workaround to help other get by, I had the same issue after installing .NET 8 core along side of Visual Studio 2022 to allow us to migrate our Azure functions from .NET 7 isolated to .NET 8 isolated. Uninstalling and reinstalling did not help and it even broke the .NET 7 function app . I was finally able to get .NET 7 working again by adding .NET Framework 4.8.1. development tools in Visual Studio. After installing, .NET 7 was working again but .NET 8 through the error, "There is no Functions runtime available that matches the version specified in the project" even though .NET 8 was still installed. To solve this issue and get .NET 8 working, I had to go to Tools -> Options -> Projects & Solutions -> Azure functions. Then check for updates and install updates.

That is more around updating the templates etc... The underlying issue here surely is that adding IConfiguration to Function Apps was triggering an issue with DI.

danelec-hpm commented 8 months ago

Same issue with my .Net 8 Azure Function. Adding the AddSingleton line breaks the Azure Function with the error "System.InvalidOperationException: 'The gRPC channel URI 'http://:' could not be parsed.'":

using Microsoft.Azure.Functions.Worker;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
var host = new HostBuilder()
    .ConfigureFunctionsWebApplication()
    .ConfigureServices(services =>
    {
        services.AddHttpClient<UpdateStatus>();
        services.AddApplicationInsightsTelemetryWorkerService();
        services.ConfigureFunctionsApplicationInsights();
        services.AddSingleton<IConfiguration>(new ConfigurationBuilder()
            .SetBasePath(Directory.GetCurrentDirectory())
            .AddJsonFile("appsettings.json")
            .AddEnvironmentVariables()
            .Build());
    })
    .Build();

host.Run();
JoostVanVelthoven commented 8 months ago

is it possible to reopen this ticket? Otherwise i think it will not be fixed.

matejkohut commented 8 months ago

Hello. Another use case I ran into: Trying to write an integration test and running the code from the test methods. This shouldn't be closed.

JackDevAU commented 8 months ago

+1 getting this issue when running locally

danelec-hpm commented 8 months ago

I tried the fix from @Mathih13 where you move the services.Add line out of the ConfigureServices and instead use .ConfigureAppConfiguration(s => s.AddConfiguration(new ConfigurationBuilder(..... and that works.

H-Finch-404 commented 8 months ago

I tried the fix from @Mathih13 where you move the services.Add line out of the ConfigureServices and instead use .ConfigureAppConfiguration(s => s.AddConfiguration(new ConfigurationBuilder(..... and that works.

This workaround does not work for me

djimenez1enviroingenieria1com commented 8 months ago

We are facing the same issue, and we are consulting with our Microsoft sponsor. Currently, we have opted to directly remove the 'WEBSITE_USE_PLACEHOLDER_DOTNETISOLATED' environment variable, in addition to the configuration changes mentioned in this same thread. With that, the error is reduced but sporadically, we still see the error in Application Insights without deploying or making modifications. We are going to try escalating the issue because it is causing us message losses and consequent loss of transactions (lost money). If we continue to experience the inconvenience, we may consider switching cloud service providers.

Wangor commented 7 months ago

This is not resolved at all, no proper solution and no fix. Azure Functions got really BS...

teknofobia commented 7 months ago

I'm having this issue, but only when I deploy to Azure. Running locally, it works fine. This is with a brand new .Net 8 function app. Setting WEBSITE_USE_PLACEHOLDER_DOTNETISOLATED to 0 did not fix it.

Here is my host building for comparison

public static IHostBuilder InitializeHostBuilder()
{
    return new HostBuilder()
        .ConfigureFunctionsWebApplication()
        .ConfigureAppConfiguration((context, config) =>
        {
            config
                .SetBasePath(context.HostingEnvironment.ContentRootPath)
                .AddJsonFile("local.settings.json", optional: true, reloadOnChange: true)
                .AddAppSettingsFile(context)
                .AddAppSettingsFile(context, useEnvironment: true)
                .AddEnvironmentVariables();

            config.ConfigureApplication(context.HostingEnvironment);
        })
        .ConfigureServices((ctx, services) =>
        {
            services.AddApplicationInsightsTelemetryWorkerService();
            services.ConfigureFunctionsApplicationInsights();

            services.RegisterFunctions(ctx.Configuration, ctx.HostingEnvironment)
                .RegisterInfrastructure(ctx.Configuration)
                .RegisterApplication();
        });
}

EDIT:

I just found a solution to this issue.

In my case, I have a private vNet. The storage account is in one subnet, and the function app has one subnet for incoming requests and through vNet integration another subnet for outgoing requests, all within the same vNet. The function app is also set to "Enabled with access restrictions" for Public Access, with disallow all as the default rule. (This is so we can allow the tags AzureDevOps and AzureCloud access to the scm site for deployments) The vNet has a private dns zone that maps the storage account name to the private subnet ip address. Access keys are disabled on my storage account.

First, I went into the storage account and granted Storage Account Data Owner via RBAC to the managed identity that I am using with my function app. This can be either System or User Assigned. I did this based on an article I found that requires this level of access for a function app.

If you are using a blob-triggered function, you will also have to add Storage Account Contributor and Storage Queue Data Contributor roles

Then, I went into the function app and deleted the Configuration Setting AzureWebJobsStorage. Per this article

If you are using a system-assigned Managed Identity, you need to add the following configuration setting:

If you are using a User Assigned Managed Identity, you need to add the following configuration settings:

Also, in the Storage Account, on the Networking Blade, (because my storage account is in a private vNet), I had to grant access from the subnet where the Private link resides for my Function app (the subnet assigned via vNet Integration within the function app)

GrimRob commented 6 months ago

This is not resolved at all, no proper solution and no fix. Azure Functions got really BS...

I'll just stick to dotnet6 until they fix it properly. I can't even run on localhost.

wheelie33 commented 6 months ago

This is not resolved at all, no proper solution and no fix. Azure Functions got really BS...

I'll just stick to dotnet6 until they fix it properly. I can't even run on localhost.

FYI: They are removing support for 6 in a few months. They want version 8

GrimRob commented 6 months ago

This is not resolved at all, no proper solution and no fix. Azure Functions got really BS...

I'll just stick to dotnet6 until they fix it properly. I can't even run on localhost.

FYI: They are removing support for 6 in a few months. They want version 8

Yes. I will try in a couple of months to see if things have improved. I have dotnet8 in a branch ready to merge once I get it working.

jangix commented 6 months ago

Even though the problem is already closed I would like to add my solution because although I had tried all the solutions proposed in this thread I had not obtained any positive results, taking a long time to solve by myself.

In my case, the function app v4 was to run as an isolated process as a Docker linux-x64 container (in debug with VS22 it was started without errors).

What I was doing wrong was the sdk image I was using in the DockerFile, I was using: FROM mcr.microsoft.com/azure-functions/dotnet-isolated:4-dotnet-isolated7.0 instead of: FROM mcr.microsoft.com/azure-functions/dotnet-isolated:4-dotnet-isolated7.0-appservice

If you need to launch the function app locally instead, you must install Azure Functions Core Tools, then publish the function app and run the following command in the publish folder: func start --dotnet-isolated

Faccio presente che è meglio specificare anche queste variabili d'ambiente:

ENV FUNCTIONS_WORKER_RUNTIME=dotnet-isolated \
 WEBSITE_USE_PLACEHOLDER_DOTNETISOLATED=0 \
 linuxFxVersion=DOTNET-ISOLATED|7.0

I hope this helps other people.

marcroussy commented 5 months ago

I have the same issue running from Visual Studio, but if I run it via func start --dotnet-isolated from the command line then it works. Makes it difficult to debug obviously, but I can at least get around it for now.

Update: the problem on my side was that somehow azure development tools had been uninstalled from visual studio, re-installing it and doing what @rohncarlson suggested fixed it for me.

PlehXP commented 4 days ago

The grpc host and port are provided by command line. So the fix for me was to add

.AddCommandLine(Environment.GetCommandLineArgs())

When creating your ConfigurationBuilder Hope this helps.

MichaelBakkerDP commented 4 days ago

I had the same problem (debugging locally) and read this thread with dread. Then i closed the solution, removed all bin and debug folders and restarted. A default tactic that i always use when reaching the end of communications like these. This solved my problem so i continued working, with yet another uncomfortable feeling added to the dark cloud that hangs over me.

stokara commented 3 days ago

This was driving me crazy. I had intermittent problems of the program not starting - getting stuck when reading the config. And then had this issue with grpc. Removing the ConfigurationBuilder has solved the problem completely and I have verified that the HostBuilder does read the local.setting.json.

-var configuration = new ConfigurationBuilder()
-    .SetBasePath(AppContext.BaseDirectory)
-    .AddJsonFile("local.settings.json", optional: true, reloadOnChange: true)
-    .AddEnvironmentVariables()
-    .Build();
var host = new HostBuilder()
    .ConfigureFunctionsWebApplication()
    .ConfigureServices(services =>
    {
        services.AddLogging();
-        services.AddSingleton<IConfiguration>(configuration);_
        services.AddApplicationInsightsTelemetryWorkerService();
        services.ConfigureFunctionsApplicationInsights();
        services.ConfigureMdbLoaderServices();
    })
    .Build();
TheGissel commented 1 day ago

I had the same problem (debugging locally) and read this thread with dread. Then i closed the solution, removed all bin and debug folders and restarted. A default tactic that i always use when reaching the end of communications like these. This solved my problem so i continued working, with yet another uncomfortable feeling added to the dark cloud that hangs over me.

This exact problem came out of the blue, but closing the solution, deleting bin and obj folders and then starting again solved it for me