search

microsoft / durabletask-mssql

Microsoft SQL storage provider for Durable Functions and the Durable Task Framework
MIT License
87 stars 32 forks source link

Platform Not Supported Exception for Microsoft.Data.SqlClient #40

Closed marcd123 closed 3 years ago

marcd123 commented 3 years ago

I'm running a brand new Python Durable Function app locally with the SQL Server Durable Task extension. I have provided the Connection String to my Azure SQL server in a local.settings.json environment variable and am referencing that variable in my host.json configuration (all shown below).

When I run func start I get the following error towards the bottom saying that Microsoft.Data.SqlClient is not supported on this platform:

Found Python version 3.9.5 (python3).

Azure Functions Core Tools
Core Tools Version:       3.0.3477 Commit hash: 5fbb9a76fc00e4168f2cc90d6ff0afe5373afc6d  (64-bit)
Function Runtime Version: 3.0.15584.0

[2021-07-22T00:12:50.711Z] Cannot create directory for shared memory usage: /dev/shm/AzureFunctions
[2021-07-22T00:12:50.711Z] System.IO.FileSystem: Access to the path '/dev/shm/AzureFunctions' is denied. Operation not permitted.
[2021-07-22T00:12:51.431Z] A host error has occurred during startup operation '4b33ffdd-e1a1-4a0e-b558-a0d07aa07d3f'.
[2021-07-22T00:12:51.431Z] Microsoft.Data.SqlClient: Microsoft.Data.SqlClient is not supported on this platform.
Value cannot be null. (Parameter 'provider')

Host.json

{
  "version": "2.0",
  "logging": {
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true,
        "excludedTypes": "Request"
      }
    },
    "logLevel": {
      "DurableTask.SqlServer": "Information",
      "DurableTask.Core": "Warning"
    }
  },
  "extensions": {
    "durableTask": {
      "storageProvider": {
        "type": "mssql",
        "connectionStringName": "SQLDB_Connection",
        "taskEventLockTimeout": "00:02:00"
      }
    }
  }
}

local.settings.json

{
  "IsEncrypted": false,
  "Values": {
    "FUNCTIONS_WORKER_RUNTIME": "python",
    "SQLDB_Connection": "<REDACTED>"
  }
}

extensions.csproj

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netcoreapp3.1</TargetFramework>
    <WarningsAsErrors></WarningsAsErrors>
    <DefaultItemExcludes>**</DefaultItemExcludes>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.Azure.WebJobs.Extensions.DurableTask" Version="2.5.0" />
    <PackageReference Include="Microsoft.Azure.WebJobs.Script.ExtensionsMetadataGenerator" Version="1.1.3" />
    <PackageReference Include="Microsoft.DurableTask.SqlServer.AzureFunctions" Version="0.9.1-beta" />
  </ItemGroup>
</Project>

.NET version 5.0.302 .NET Core 3.1 Functions Core Tools 3.0.3477 Running on MacOS Big Sur

This occurs with both version 0.9.1-beta and 0.10.0-beta of the Durable Functions SQL Server extension.

I'm really excited to use this new event source but I'm blocked here. Can someone please help?

marcd123 commented 3 years ago

I tested this deployed to AKS as well and got the same "not supported" exception

info: Host.Triggers.DurableTask[0]
      Using the mssql storage provider.
fail: Host.Startup[515]
      A host error has occurred during startup operation '69f1a875-a602-45b6-a839-2840f22066dc'.
System.PlatformNotSupportedException: Microsoft.Data.SqlClient is not supported on this platform.
   at Microsoft.Data.SqlClient.SqlConnectionStringBuilder..ctor(String connectionString)
   at DurableTask.SqlServer.AzureFunctions.SqlDurabilityOptions.GetOrchestrationServiceSettings(DurableTaskOptions extensionOptions, IConnectionStringResolver connectionStringResolver) in /_/src/DurableTask.SqlServer.AzureFunctions/SqlDurabilityOptions.cs:line 60
   at DurableTask.SqlServer.AzureFunctions.SqlDurabilityProviderFactory.GetOrchestrationServiceSettings() in /_/src/DurableTask.SqlServer.AzureFunctions/SqlDurabilityProviderFactory.cs:line 157
   at DurableTask.SqlServer.AzureFunctions.SqlDurabilityProviderFactory.GetOrchestrationService() in /_/src/DurableTask.SqlServer.AzureFunctions/SqlDurabilityProviderFactory.cs:line 106
   at DurableTask.SqlServer.AzureFunctions.SqlDurabilityProviderFactory.GetDurabilityProvider() in /_/src/DurableTask.SqlServer.AzureFunctions/SqlDurabilityProviderFactory.cs:line 56
   at Microsoft.Azure.WebJobs.Extensions.DurableTask.DurableTaskExtension..ctor(IOptions`1 options, ILoggerFactory loggerFactory, INameResolver nameResolver, IEnumerable`1 orchestrationServiceFactories, IApplicationLifetimeWrapper hostLifetimeService, IDurableHttpMessageHandlerFactory durableHttpMessageHandlerFactory, ILifeCycleNotificationHelper lifeCycleNotificationHelper, IMessageSerializerSettingsFactory messageSerializerSettingsFactory, IPlatformInformationService platformInformationService, IErrorSerializerSettingsFactory errorSerializerSettingsFactory, IWebHookProvider webhookProvider, ITelemetryActivator telemetryActivator) in D:\a\r1\a\azure-functions-durable-extension\src\WebJobs.Extensions.DurableTask\DurableTaskExtension.cs:line 154
   at lambda_method(Closure , IResolverContext )
   at DryIoc.Factory.<>c__DisplayClass26_0.<ApplyReuse>b__2() in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6605
   at DryIoc.Scope.TryGetOrAdd(ImMap`1 items, Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 7849
   at DryIoc.Scope.GetOrAdd(Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 7834
   at DryIoc.Factory.ApplyReuse(Expression serviceExpr, Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6604
   at DryIoc.Factory.GetExpressionOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6564
   at DryIoc.WrappersSupport.GetArrayExpression(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 2903
   at DryIoc.ExpressionFactory.CreateExpressionOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 7699
   at DryIoc.Factory.GetExpressionOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6554
   at DryIoc.ReflectionFactory.CreateExpressionOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 7083
   at DryIoc.Factory.GetExpressionOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6554
   at DryIoc.Factory.GetDelegateOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6634
   at DryIoc.Container.ResolveAndCacheDefaultFactoryDelegate(Type serviceType, IfUnresolved ifUnresolved) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 211
   at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType, IfUnresolved ifUnresolved) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 196
   at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetService(Type serviceType, IfUnresolved ifUnresolved) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/JobHostServiceProvider.cs:line 99
   at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetRequiredService(Type serviceType) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/JobHostServiceProvider.cs:line 82
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.Azure.WebJobs.WebJobsServiceCollectionExtensions.<>c.<AddWebJobs>b__1_0(IServiceProvider p) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Hosting\WebJobsServiceCollectionExtensions.cs:line 57
   at DryIoc.Microsoft.DependencyInjection.DryIocAdapter.<>c__DisplayClass3_0.<RegisterDescriptor>b__0(IResolverContext r) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/DryIocAdapter.cs:line 156
   at DryIoc.Registrator.<>c__DisplayClass27_0.<RegisterDelegate>b__0(IResolverContext r) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 4550
   at DryIoc.Factory.<>c__DisplayClass26_0.<ApplyReuse>b__2() in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6605
   at DryIoc.Scope.TryGetOrAdd(ImMap`1 items, Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 7849
   at DryIoc.Scope.GetOrAdd(Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 7834
   at DryIoc.Factory.ApplyReuse(Expression serviceExpr, Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6604
   at DryIoc.Factory.GetExpressionOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6564
   at DryIoc.Factory.GetDelegateOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6634
   at DryIoc.DelegateFactory.GetDelegateOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 7746
   at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType, Object serviceKey, IfUnresolved ifUnresolved, Type requiredServiceType, Request preResolveParent, Object[] args) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 291
   at lambda_method(Closure , IResolverContext )
   at DryIoc.Factory.<>c__DisplayClass26_0.<ApplyReuse>b__2() in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6605
   at DryIoc.Scope.TryGetOrAdd(ImMap`1 items, Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 7849
   at DryIoc.Scope.GetOrAdd(Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 7834
   at DryIoc.Factory.ApplyReuse(Expression serviceExpr, Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6604
   at DryIoc.Factory.GetExpressionOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6564
   at DryIoc.Factory.GetDelegateOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6634
   at DryIoc.Container.ResolveAndCacheDefaultFactoryDelegate(Type serviceType, IfUnresolved ifUnresolved) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 211
   at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType, IfUnresolved ifUnresolved) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 196
   at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetService(Type serviceType, IfUnresolved ifUnresolved) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/JobHostServiceProvider.cs:line 99
   at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetRequiredService(Type serviceType) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/JobHostServiceProvider.cs:line 82
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.Azure.WebJobs.WebJobsServiceCollectionExtensions.<>c.<AddWebJobs>b__1_4(IServiceProvider p) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Hosting\WebJobsServiceCollectionExtensions.cs:line 92
   at DryIoc.Microsoft.DependencyInjection.DryIocAdapter.<>c__DisplayClass3_0.<RegisterDescriptor>b__0(IResolverContext r) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/DryIocAdapter.cs:line 156
   at DryIoc.Registrator.<>c__DisplayClass27_0.<RegisterDelegate>b__0(IResolverContext r) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 4550
   at DryIoc.Factory.<>c__DisplayClass26_0.<ApplyReuse>b__2() in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6605
   at DryIoc.Scope.TryGetOrAdd(ImMap`1 items, Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 7849
   at DryIoc.Scope.GetOrAdd(Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 7834
   at DryIoc.Factory.ApplyReuse(Expression serviceExpr, Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6604
   at DryIoc.Factory.GetExpressionOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6564
   at DryIoc.Factory.GetDelegateOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6634
   at DryIoc.DelegateFactory.GetDelegateOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 7746
   at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType, Object serviceKey, IfUnresolved ifUnresolved, Type requiredServiceType, Request preResolveParent, Object[] args) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 291
   at lambda_method(Closure , IResolverContext )
   at DryIoc.Factory.<>c__DisplayClass26_0.<ApplyReuse>b__2() in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6605
   at DryIoc.Scope.TryGetOrAdd(ImMap`1 items, Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 7849
   at DryIoc.Scope.GetOrAdd(Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 7834
   at DryIoc.Factory.ApplyReuse(Expression serviceExpr, Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6604
   at DryIoc.Factory.GetExpressionOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6564
   at DryIoc.ReflectionFactory.CreateExpressionOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 7083
   at DryIoc.Factory.GetExpressionOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6554
   at DryIoc.ReflectionFactory.CreateExpressionOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 7083
   at DryIoc.Factory.GetExpressionOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6554
   at DryIoc.Factory.GetDelegateOrDefault(Request request) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 6634
   at DryIoc.Container.ResolveAndCacheDefaultFactoryDelegate(Type serviceType, IfUnresolved ifUnresolved) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 211
   at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType, IfUnresolved ifUnresolved) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs:line 196
   at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetService(Type serviceType, IfUnresolved ifUnresolved) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/JobHostServiceProvider.cs:line 99
   at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetService(Type serviceType) in /src/azure-functions-host/src/WebJobs.Script.WebHost/DependencyInjection/JobHostServiceProvider.cs:line 77
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
   at Microsoft.Azure.WebJobs.Script.WebHost.WebJobsScriptHostService.UnsynchronizedStartHostAsync(ScriptHostStartupOperation activeOperation, Int32 attemptCount, JobHostStartupMode startupMode) in /src/azure-functions-h
marcd123 commented 3 years ago

You can repro with this public repo

https://github.com/marcd123/durabletasktest

cgillum commented 3 years ago

Hey @marcd123, which version of the Microsoft.DurableTask.SqlServer.AzureFunctions package are you using? I didn't see it in your extensions.csproj and we released a v0.10.0-beta package yesterday with a major version update of Microsoft.Data.SqlClient. I'm wondering if that will make a difference.

marcd123 commented 3 years ago

Hey @cgillum!

<PackageReference Include="Microsoft.DurableTask.SqlServer.AzureFunctions" Version="0.9.1-beta" />

I've tried both v0.9.1-beta and v0.10.0-beta locally, and I receive the "Platform Not Supported" exception in both cases.

I've only deployed to AKS with v0.9.1-beta, and still receiving the exception in AKS. Given that both 0.9.1 and 0.10.0 give me the exception locally, I would expect 0.10.0 to give the exception in AKS as well, though I will give that a shot today.

Also worth mentioning that this is the image we use for our Dockerfile, it's the latest Python 3.9/Linux Azure Functions image:

FROM mcr.microsoft.com/azure-functions/python:3.0-python3.9

I've read Github issues that refer to this exception but in non Azure Functions context, and it often seems related to .NET or .NET Core versioning.

Full Dockerfile:

# To enable ssh & remote debugging on app service change the base image to the one below
# FROM mcr.microsoft.com/azure-functions/python:3.0-python3.9-appservice
FROM mcr.microsoft.com/azure-functions/python:3.0-python3.9

ENV AzureWebJobsScriptRoot=/home/site/wwwroot \
    AzureFunctionsJobHost__Logging__Console__IsEnabled=true

COPY requirements.txt /
RUN pip install -r /requirements.txt

COPY . /home/site/wwwroot
cgillum commented 3 years ago

Thanks - sorry - I don't know why I didn't see your package version in your original post. I'm working on creating a local repro now.

cgillum commented 3 years ago

Looks like I can easily reproduce this even on Windows with a .csx (C# scripting) project. This makes me think it's a problem with the Functions SDK build process (part of func extensions install) not bundling dependencies correctly. I'll play around with this a bit more, but I may need to have some of the Azure Functions core team help investigate if I get stuck.

marcd123 commented 3 years ago

Looks like I can easily reproduce this even on Windows with a .csx (C# scripting) project. This makes me think it's a problem with the Functions SDK build process (part of func extensions install) not bundling dependencies correctly. I'll play around with this a bit more, but I may need to have some of the Azure Functions core team help investigate if I get stuck.

Thanks @cgillum! Let me know if there is anything I can try on my end.

I greatly appreciate your rapid response!

cgillum commented 3 years ago

Through some research and experimentation, I found a workaround that seems to work for C# scripting. I had to add the following block to my extensions.csproj:

<Target Name="PostBuild" AfterTargets="PostBuildEvent">
  <Move SourceFiles="$(OutDir)/extensions.deps.json" DestinationFiles="$(OutDir)/function.deps.json" />
</Target>

I also had to update my referenced version of Microsoft.Azure.WebJobs.Script.ExtensionsMetadataGenerator to be the latest version.

@marcd123 here is a modified version of your extensions.csproj that includes my workarounds. Can you try it out for your Python app?

extensions.csproj

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netcoreapp3.1</TargetFramework>
    <WarningsAsErrors></WarningsAsErrors>
    <DefaultItemExcludes>**</DefaultItemExcludes>
  </PropertyGroup>

  <!-- Need to rename extensions.deps.json to function.deps.json for the native SQL dependencies to get loaded -->
  <Target Name="PostBuild" AfterTargets="PostBuildEvent">
    <Move SourceFiles="$(OutDir)/extensions.deps.json" DestinationFiles="$(OutDir)/function.deps.json" />
  </Target>

  <ItemGroup>
    <PackageReference Include="Microsoft.Azure.WebJobs.Extensions.DurableTask" Version="2.5.0" />
    <PackageReference Include="Microsoft.Azure.WebJobs.Script.ExtensionsMetadataGenerator" Version="1.2.3" />
    <PackageReference Include="Microsoft.DurableTask.SqlServer.AzureFunctions" Version="0.10.0-beta" />
  </ItemGroup>
</Project>
marcd123 commented 3 years ago

Hey @cgillum

I applied the workaround you mentioned to my extensions.csproj (AND IT WORKED)

I first encountered this error when running func start

[2021-07-22T19:53:45.939Z] Microsoft.Azure.WebJobs.Script.WebHost: Secret initialization from Blob storage failed due to missing both an Azure Storage connection string and a SAS connection uri. For Blob Storage, please provide at least one of these. If you intend to use files for secrets, add an App Setting key 'AzureWebJobsSecretStorageType' with value 'Files'.

I was not expecting to need a storage account, but I added an AzureWebJobsStorage connection string to my local.settings.json and then func start worked as expected. I also tested my client and orchestrator functions and everything appears to be working.

I'm quite curious about the need for the storage account, while we will be hosting this on Azure my understanding is that the Durable SQL Server extension removes the dependency on Azure Storage. Is it possible I have something else configured incorrectly?

Thanks again!

cgillum commented 3 years ago

@marcd123 Thanks for confirming! If you want to get rid of the storage account requirement, the simplest thing you can do is add a AzureWebJobsSecretStorageType and set it to Files. There might be an even more Kubernetes-native way to do secret storage. I don't know if it's explicitly documented anywhere, but if you generate your contains using the Functions Core tools and the func kubernetes commands, it is able to set things up in a more K8s-friendly way.

Anyways, glad you're unblocked and thanks again for confirming the workaround! I'm following up with the Functions team to try and figure out how we can get rid of the need for this workaround. Longer term, this backend will be part of extension bundles so you won't need an extensions.csproj file at all (we're hoping to have this ready later this calendar year).

marcd123 commented 3 years ago

Thanks again. I'll let you know if we encounter any other issues on our path with Durable Functions, AKS, and SQL Server task hub.

My issue is resolved and can be closed

marcd123 commented 3 years ago

@cgillum This is not exactly a SQL Service Durable Task Hub issue, but I would like to know what is the recommend approach for solving the issue below while using SQL Service Durable Task Hubs. You can refer to the public repo again if you'd like to repro:

https://github.com/marcd123/durabletasktest

When we deploy our Durable Function app to AKS, HTTP (HTTP-Starter) and Non-HTTP (Orchestrator/Activity) functions are separated into two deployments.

Because both deployments have the same host.json and local.settings.json, and are thus pointing to the same SQL task hub - I would think having separate deployments would be fine.

However, when we trigger our HTTP-Starter (OrchClient) which is supposed to trigger the Orchestrator (HelloOrchestrator) in a separate deployment, we get this exception from HTTP-Starter saying the orchestrator function could not be found....

Exception: Exception: The function 'HelloOrchestrator' doesn't exist, is disabled, or is not an orchestrator function. Additional info: No orchestrator functions are currently registered!

From what I've read elsewhere, this is because my HTTP function app is checking for the Orchestrator functions locally, which have actually been separated out to another deployment.

I have also read that it may be possible to bypass this local check for the function by using some ExternalClient binding ((https://github.com/Azure/azure-functions-core-tools/issues/2345#issuecomment-804362398)), but I've only seen C# project examples and am unsure how/if this ExternalClient binding can be used within host.json or individual function.json bindings.

Is it possible to bypass this local check with ExternalClient binding in a Python Function app, and if so could you please provide an example of the host.json/function.json configuration?

cgillum commented 3 years ago

@marcd123 would you mind opening a new GitHub issue for your new question? That will make things a little less confusing for me and others.

marcd123 commented 3 years ago

@cgillum Thanks, created a new issue here

https://github.com/microsoft/durabletask-mssql/issues/41