Known Bug: WebPubSub Extension causes serialization issues breaking OOProc DF apps

Description

There seems to be an issue in the WebPubSub extension (Microsoft.Azure.WebJobs.Extensions.WebPubSub) that overrides defaults for object serialization globally. In applications in which this extension is loaded by the host, this affects this line in the DF extension, serializing OOProc SDKs' payload in an unexpected format (enums are serialized as strings instead of numbers, and properties are camelcased when they shouldn't), causing errors for OOProc DF applications.

This error is particularly exacerbated for upcoming programming models for Python (V2) and Node (V4), which require the EnableWorkerIndexing feature flag to be set. Before https://github.com/Azure/azure-functions-host/issues/8870, setting this flag meant that all extensions in an extension bundle are loaded, even if they aren't used by the application. This means that any app using extension bundles (current or upcoming programming model) that sets the EnableWorkerIndexing flag, would experience this issue, unless they run a host version which includes the fix for https://github.com/Azure/azure-functions-host/issues/8870 (not yet rolled out in Azure).

Who's Affected

This issue affects app where both the WebPubSub extension and the DF extension are loaded. This can be:

PyStein (V2 Python programming model) or V4 Node programming model apps, as those require the EnableWorkerIndexing feature flag
Existing prog model apps that set the EnableWorkerIndexing flag for some reason
DF apps that also use the WebPubSub model in their code or specify it in their .csproj file

Reproduction

There are multiple ways to reproduce this error. Below are a few examples. The error should still exist in all OOProc languages, not just the one used in the example.

Using the upcoming programming models with extension bundles

Create a new Python V2 programming model (PyStein) application, with an HTTP starter, an orchestration, and an activity function.
Set AzureWebJobsFeatureFlags": "EnableWorkerIndexing" in the local.settings.json file
Make sure you are using extension bundles by specifying in the host.json file:

"extensionBundle": {
        "id": "Microsoft.Azure.Functions.ExtensionBundle",
        "version": "[3.15.0, 4.0.0)"
    }

Run func start and try calling your orchestrator
The orchestrator fails

This is true for the latest version of core tools (4.0.4915) running the latest version of the host (4.14.0.19631) since the fix for https://github.com/Azure/azure-functions-host/issues/8870 isn't released yet.

Setting the `EnableWorkerIndexing` flag on an existing programming model app using extension bundles

Go to your favorite existing DF OOProc app
Set AzureWebJobsFeatureFlags": "EnableWorkerIndexing" in the local.settings.json file
Make sure you are using extension bundles by specifying in the host.json file:

"extensionBundle": {
        "id": "Microsoft.Azure.Functions.ExtensionBundle",
        "version": "[3.15.0, 4.0.0)"
    }

Run func start and try calling your orchestrator
The orchestrator fails

Explicitly adding the WebPubSub extension to a `.csproj` file

Go to your favorite existing DF app
Make sure you remove references to extension bundles in the host.json file
Create an extension.csproj file that contains the DF extension and the WebPubSub extension, as below:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
  <WarningsAsErrors></WarningsAsErrors>
  <DefaultItemExcludes>**</DefaultItemExcludes>
  </PropertyGroup>
  <ItemGroup>
    <None Include="host.json" />
    <None Include="package.json" />
    <None Include="local.settings.json" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.Azure.WebJobs.Extensions.DurableTask" Version="2.9.0" />
    <PackageReference Include="Microsoft.Azure.WebJobs.Extensions.WebPubSub" Version="1.3.0" />
    <PackageReference Include="Microsoft.Azure.WebJobs.Script.ExtensionsMetadataGenerator" Version="4.0.1" />
  </ItemGroup>
</Project>

Run func extensions install to install the extensions
Run func start and try calling your orchestrator
The orchestrator fails

This repro still occurs even with the fix to https://github.com/Azure/azure-functions-host/issues/8870, as using the .csproj file coerces the host into loading the WebPubSub extension even if it's not actively used by the app.

Having a WebPubSub function in the same app as DF functions

Create a new DF TypeScript app with an orchestrator, activity function, and an HTTP starter
Make sure you are using extension bundles, by specifying in your host.json file:

"extensionBundle": {
        "id": "Microsoft.Azure.Functions.ExtensionBundle",
        "version": "[3.15.0, 4.0.0)"
    }

Create a new function that uses a WebPubSub binding. This example uses a webpubsub input binding, derived from this tutorial.

index.ts:

import { Context, HttpRequest } from "@azure/functions";

const pubSubFunc = async function (
  context: Context,
  req: HttpRequest,
  connection: any
) {
  context.res = { body: connection };
};

export default pubSubFunc;

function.json:

{
  "bindings": [
    {
      "authLevel": "anonymous",
      "type": "httpTrigger",
      "direction": "in",
      "name": "req"
    },
    {
      "type": "http",
      "direction": "out",
      "name": "res"
    },
    {
      "type": "webPubSubConnection",
      "name": "connection",
      "hub": "simplechat",
      "userId": "{headers.x-ms-client-principal-name}",
      "direction": "in"
    }
  ],
  "scriptFile": "../dist/negotiate/index.js"
}

Run func start and try to call your orchestrator
The orchestrator fails.

This repro still occurs even with the fix to https://github.com/Azure/azure-functions-host/issues/8870. The WebPubSub extension is always loaded regardless, since it is required by the app.

Known workarounds

In the existing prog model, so long as your app doesn't require the WebPubSub extension and you don't set the EnableWorkerIndexing feature flag, your app should not be affected.

In the upcoming prog models, if your app doesn't require the WebPubSub extension, you can:

a. Not use extension bundles, and specify the extensions you need in an extensions.csproj file; or b. Use a build of the host that includes the fix to https://github.com/Azure/azure-functions-host/issues/8870. This won't work in Azure as this fix hasn't started rolling out yet.

If your app requires both DF and WebPubSub, there are currently no known workarounds.

Azure / azure-functions-durable-extension