Azure / azure-functions-kafka-extension

Kafka extension for Azure Functions
MIT License
113 stars 77 forks source link

RegisterSchemaAsync Runtime error #423

Open dserdyuk opened 1 year ago

dserdyuk commented 1 year ago

Getting this on both local mode and Azure funapp deployment ( EP1 app plan , dotnet runtime)

System.Private.CoreLib: Method 'RegisterSchemaAsync' in type 'Microsoft.Azure.WebJobs.Extensions.Kafka.LocalSchemaRegistry' from assembly 'Microsoft.Azure.WebJobs.Extensions.Kafka, Version=3.8.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' does not have an implementation.

I've created my app via Visual Studio for Mac wizard.

why does it use local scheme registry at all in cloud deployment ?

func --version 4.0.4915

full stacktrace: 2023-01-25T23:05:10.049 [Error] A host error has occurred during startup operation '84561eeb-d54b-40e6-94fa-780c4c41fce0'.System.TypeLoadException : Method 'RegisterSchemaAsync' in type 'Microsoft.Azure.WebJobs.Extensions.Kafka.LocalSchemaRegistry' from assembly 'Microsoft.Azure.WebJobs.Extensions.Kafka, Version=3.3.2.0, Culture=neutral, PublicKeyToken=null' does not have an implementation.at System.Reflection.RuntimeAssembly.GetExportedTypes()at Microsoft.Azure.WebJobs.Host.JobHostMetadataProvider.<GetAttributesFromAssembly>d__11.MoveNext() at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Extensions\JobHostMetadataProvider.cs : 89at Microsoft.Azure.WebJobs.Host.JobHostMetadataProvider.AddAttributesFromAssembly(Assembly asm) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Extensions\JobHostMetadataProvider.cs : 80at Microsoft.Azure.WebJobs.Host.JobHostMetadataProvider.AddExtension(IExtensionConfigProvider extension) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Extensions\JobHostMetadataProvider.cs : 104at Microsoft.Azure.WebJobs.Host.JobHostMetadataProvider.Initialize() at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Extensions\JobHostMetadataProvider.cs : 46at Microsoft.Azure.WebJobs.Host.JobHostMetadataProviderFactory.Create() at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Extensions\JobHostMetadataProviderFactory.cs : 27at Microsoft.Azure.WebJobs.WebJobsServiceCollectionExtensions.<>c.<AddWebJobs>b__2_5(IServiceProvider p) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Hosting\WebJobsServiceCollectionExtensions.cs : 83at DryIoc.Microsoft.DependencyInjection.DryIocAdapter.<>c__DisplayClass3_0.<RegisterDescriptor>b__0(IResolverContext r) at /_/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/DryIocAdapter.cs : 162at DryIoc.Registrator.<>c__DisplayClass27_0.<RegisterDelegate>b__0(IResolverContext r) at /_/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 4550at lambda_method1213(Closure ,IResolverContext )at DryIoc.Factory.<>c__DisplayClass26_0.<ApplyReuse>b__2() at /_/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 6605at DryIoc.Scope.TryGetOrAdd(ImMap1 items,Int32 id,CreateScopedValue createValue,Int32 disposalOrder) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 7849at DryIoc.Scope.GetOrAdd(Int32 id,CreateScopedValue createValue,Int32 disposalOrder) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 7834at DryIoc.Factory.ApplyReuse(Expression serviceExpr,Request request) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 6604at DryIoc.Factory.GetExpressionOrDefault(Request request) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 6564at DryIoc.Factory.GetDelegateOrDefault(Request request) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 6634at DryIoc.DelegateFactory.GetDelegateOrDefault(Request request) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 7740at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType,Object serviceKey,IfUnresolved ifUnresolved,Type requiredServiceType,Request preResolveParent,Object[] args) at /_/src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 290at lambda_method1212(Closure ,IResolverContext )at DryIoc.Factory.<>c__DisplayClass26_0.b_2() at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 6605at DryIoc.Scope.TryGetOrAdd(ImMap`1 items,Int32 id,CreateScopedValue createValue,Int32 disposalOrder) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 7849at DryIoc.Scope.GetOrAdd(Int32 id,CreateScopedValue createValue,Int32 disposalOrder) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 7834at DryIoc.Factory.ApplyReuse(Expression serviceExpr,Request request) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 6604at DryIoc.Factory.GetExpressionOrDefault(Request request) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 6564at DryIoc.WrappersSupport.GetArrayExpression(Request request) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 2903at DryIoc.ExpressionFactory.CreateExpressionOrDefault(Request request) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 7699at DryIoc.Factory.GetExpressionOrDefault(Request request) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 6554at DryIoc.ReflectionFactory.CreateExpressionOrDefault(Request request) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 7083at DryIoc.Factory.GetExpressionOrDefault(Request request) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 6554at DryIoc.Factory.GetDelegateOrDefault(Request request) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 6634at DryIoc.Container.ResolveAndCacheDefaultFactoryDelegate(Type serviceType,IfUnresolved ifUnresolved) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 210at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType,IfUnresolved ifUnresolved) at //src/WebJobs.Script.WebHost/DependencyInjection/DryIoc/Container.cs : 195at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetService(Type serviceType,IfUnresolved ifUnresolved) at //src/WebJobs.Script.WebHost/DependencyInjection/JobHostServiceProvider.cs : 99at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetService(Type serviceType) at //src/WebJobs.Script.WebHost/DependencyInjection/JobHostServiceProvider.cs : 77at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)at async Microsoft.Azure.WebJobs.Script.WebHost.WebJobsScriptHostService.UnsynchronizedStartHostAsync(ScriptHostStartupOperation activeOperation,Int32 attemptCount,JobHostStartupMode startupMode) at /_/src/WebJobs.Script.WebHost/WebJobsScriptHostService.cs : 280

`

dserdyuk commented 1 year ago

Added project sample

TestKafkaTriggerBug.zip

krishna-kariya commented 1 year ago

Hi @dserdyuk, Thanks for sharing the code. In csproj file, I see that you have included the "Confluent.Kafka" package with version 1.9.3. I removed this reference, and the trigger function is working fine. Kafka Extension also uses this package and the version being used is 1.9.0. This might be causing the error.

dserdyuk commented 1 year ago

@krishna-kariya Yeah, I was trying to workaround another issue in Confluent.Kafka 1.9.0 I'm seeing on Apple M1 arch https://github.com/confluentinc/confluent-kafka-dotnet/issues/1865

I've tried several versions of Confluent.Kafka and apparently only 1.9.3 fix that librdkafka issue. This's why I had to override dependencies. And yes, I can confirm 3.8.0 extension works out of box when deployed to the cloud. Local deployment on M1 fails though either with that librdkafka or schema registry issue ( as it's stated in this bug) . SchemaRegistry issue is not specific to Apple M1.