Use `blobMetadata` as default checkpointing strategy for Azure Event Hubs scaler

[christle]

Proposal

I observed an unexpected behavior with Eventhub Checkpointing on Azure Functions v4.

Since we switched all our Functions to dotnet6 with Azure Functions 4, all Eventhub-scaler based Functions scales to the maximum number of replicas. The scaler cannot read checkpoint information from the storage Account.

So before v4, Azure functions use an older c# implementation. I mentioned it in the Eventhub scaler docs:

_"Legacy Checkpointing: C# applications, which use the Microsoft.Azure.EventHubs package ... stores checkpoint information as blob container content

C# Applications that use the current Azure.Messaging.EventHubs package ... stores checkpoint information as blob metadata."_

The new v4 uses the "Azure.Messaging.EventHubs package". This makes sense because many new implementations, even in other languages, use the same Metadata checkpointing format.

The solution is easy for now. We added an explicit configuration to our Functions:

- type: azure-eventhub
    metadata:
      checkpointStrategy: blobMetadata
      blobContainer: azure-webjobs-eventhub
      ....

It is important to name the blobContainer explicitly.

I would suggest changing the default behavior in the future. The current default refers to an old Azure Function version, and most of the new implementations use the metadata variant. I could prepare a PR for that.

Maybe we could mark that as an upcoming breaking change.

What do you think?

Use-Case

No response

Anything else?

No response

[tomkerkhove]

This is a breaking change so that will have to be part of KEDA v3.0, but it would be nice to open a PR to improve our documentation and mention Functions v4 though. Are you open to that?

[v-shenoy]

@tomkerkhove I was looking at the the checkpoint behavior, from the Functions team. I can raise a PR for the documentation.

[tomkerkhove]

Thank you!

[v-shenoy]

What are the future plans regarding this issue?

[tomkerkhove]

This is a breaking change so we'll need to wait with this one.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.