search

Azure / azure-functions-dotnet-worker

Azure Functions out-of-process .NET language worker
MIT License
419 stars 184 forks source link

How to configure KafkaOptions at function level rather than using host.json file. #2231

Open NARESHARADHYULA opened 8 months ago

NARESHARADHYULA commented 8 months ago

We are migrating in process Kafka consumer trigger to isolated worker process. we faced an issue with migrating kafkaoptions

this is how we have configured kafkaoptions earlier.

services.Configure(opt => { opt.SubscriberIntervalInSeconds = 1; opt.AutoOffsetReset = Confluent.Kafka.AutoOffsetReset.Earliest; opt.MaxBatchSize = 1; opt.LibkafkaDebug = "all"; })

where KafkaOptions is an Ioptions implementation

https://github.com/Azure/azure-functions-kafka-extension/blob/135b227ff5b516abb3f8aa21a29065cff5e154bb/src/Microsoft.Azure.WebJobs.Extensions.Kafka/Config/KafkaOptions.cs

since we replaced Microsoft.Azure.WebJobs.Extensions.Kafka package with Microsoft.Azure.Functions.Worker.Extensions.Kafka i dont see any relevant Ioptions implementation. All the samples are suggesting to use host.json file like below

"extensions": { "kafka": { "maxBatchSize": 3 } }

but with this approach we are configuring options for all the functions in the function app service. we need a way to configure them at function level. is there a way to achieve this?
kshyju commented 8 months ago

@jainharsh98 / @raorugan Can you help with this?