Activites cannot return None (the JSON object must be str, bytes or bytearray, not NoneType)

[evanlouie]

🐛 Describe the bug

Activities cannot return None.

If an activity returns None, the following exception is raised when the orchestrator attempts to parse the returned value.:

System.Private.CoreLib: Exception while executing function: Functions.hello_orchestrator. System.Private.CoreLib: Orchestrator function 'hello_orchestrator' failed: One or more errors occurred. (Result: Failure
Exception: TypeError: the JSON object must be str, bytes or bytearray, not NoneType
Stack:   File "/opt/homebrew/Cellar/azure-functions-core-tools@4/4.0.5198/workers/python/3.10/OSX/Arm64/azure_functions_worker/dispatcher.py", line 479, in _handle__invocation_request
    call_result = await self._loop.run_in_executor(
  File "/Users/***/.pyenv/versions/3.10.12/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/homebrew/Cellar/azure-functions-core-tools@4/4.0.5198/workers/python/3.10/OSX/Arm64/azure_functions_worker/dispatcher.py", line 752, in _run_sync_func
    return ExtensionManager.get_sync_invocation_wrapper(context,
  File "/opt/homebrew/Cellar/azure-functions-core-tools@4/4.0.5198/workers/python/3.10/OSX/Arm64/azure_functions_worker/extension.py", line 215, in _raw_invocation_wrapper
    result = function(**args)
  File "/Users/***/workspace/tmp/durable/.venv/lib/python3.10/site-packages/azure/durable_functions/orchestrator.py", line 69, in handle
    return Orchestrator(fn).handle(DurableOrchestrationContext.from_json(context_body))
  File "/Users/***/workspace/tmp/durable/.venv/lib/python3.10/site-packages/azure/durable_functions/orchestrator.py", line 47, in handle
    return self.task_orchestration_executor.execute(context, context.histories, self.fn)
  File "/Users/***/workspace/tmp/durable/.venv/lib/python3.10/site-packages/azure/durable_functions/models/TaskOrchestrationExecutor.py", line 93, in execute
    self.process_event(event)
  File "/Users/***/workspace/tmp/durable/.venv/lib/python3.10/site-packages/azure/durable_functions/models/TaskOrchestrationExecutor.py", line 144, in process_event
    self.set_task_value(event, is_success, id_key)
  File "/Users/***/workspace/tmp/durable/.venv/lib/python3.10/site-packages/azure/durable_functions/models/TaskOrchestrationExecutor.py", line 197, in set_task_value
    new_value = parse_history_event(event)
  File "/Users/***/workspace/tmp/durable/.venv/lib/python3.10/site-packages/azure/durable_functions/models/TaskOrchestrationExecutor.py", line 171, in parse_history_event
    return json.loads(directive_result.Result, object_hook=_deserialize_custom_object)
  File "/Users/***/.pyenv/versions/3.10.12/lib/python3.10/json/__init__.py", line 339, in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
).

🤔 Expected behavior

As per the docs, activiies do not need to return a value.

Activity function: It's called by the orchestrator function, performs work, and optionally returns a value.

☕ Steps to reproduce

import azure.functions as func
import azure.durable_functions as df

app = df.DFApp(http_auth_level=func.AuthLevel.ANONYMOUS)

# An HTTP-Triggered Function with a Durable Functions Client binding
@app.route(route="orchestrators/{functionName}")
@app.durable_client_input(client_name="client")
async def http_start(req: func.HttpRequest, client):
    function_name = req.route_params.get("functionName")
    instance_id = await client.start_new(function_name)
    response = client.create_check_status_response(req, instance_id)
    return response

# Orchestrator
@app.orchestration_trigger(context_name="context")
def hello_orchestrator(context):
    result1 = yield context.call_activity("hello", "Seattle")
    result2 = yield context.call_activity("hello", "Tokyo")
    result3 = yield context.call_activity("hello", "London")

    return [result1, result2, result3]

# Activity
@app.activity_trigger(input_name="city")
def hello(city: str):
    if city == "London":
        return None
    return "Hello " + city

Then call the orchestrator:

curl http://localhost:7071/api/orchestrators/hello_orchestrator

Other Notes

Their seems to be a misalignment between the durable functions runtime and the azure-functions-durable library:

• azure-functions-durable is expecting the Result of tasks to ALWAYS be a JSON serializeable string so it can json.loads(...) it (see TaskOrchestrationExecutor).
• The Result property for events in table storage is a string. But when an activty returns None, instead of storing "null" in the row, it saves nothing.

Either azure-functions-durable needs to check for None before doing json.loads(...) in TaskOrchestrationExecutor.set_task_value or the durable functions runtime needs to start emitting "null" for null result values.

[evanlouie]

probably related: https://github.com/Azure/azure-functions-durable-python/issues/260

[davidmrdavid]

Hi @evanlouie, thanks for reaching out and thanks for the detailed bug report.

As per the docs, activiies do not need to return a value.

You're correct, that's a documentation bug. I'm noting that down as a follow up item.

azure-functions-durable is expecting the Result of tasks to ALWAYS be a JSON serializeable string so it can json.loads(...) it (see TaskOrchestrationExecutor).

Yes, that's correct, there's an assumption that inputs and outputs can be JSON serialized.

The Result property for events in table storage is a string. But when an activty returns None, instead of storing "null" in the row, it saves nothing.

Just to confirm, are you actually triggering a bug where you receive a None result and the result isn't properly assigned to a task? To my understanding, since activities are failing at the point of returning None, then the error is occuring before we even attempt to set a task's output value, right?

[evanlouie]

Hi @davidmrdavid

Just to confirm, are you actually triggering a bug where you receive a None result and the result isn't properly assigned to a task? To my understanding, since activities are failing at the point of returning None, then the error is occuring before we even attempt to set a task's output value, right?

The activities which return None actually complete successfully. If I look the storage account in the History table (i.e TestHubNameHistory on local azurite), the activities which return None will have EventType == TaskCompleted with Result == null.

then the error is occuring before we even attempt to set a task's output value, right?

So from the looks of it, the durable functions runtime allow activities to return None (and is stored as null in table), but the TaskOrchestrationExecutor which the orchestrator calls does not deserialize the null as it expects a JSON string.

[cgillum]

@davidmrdavid @lilyjma I recently root caused another customer issue to this bug. It would be great if we could prioritize fixing it, as it seems like a trivially simple issue that is really hard to debug.

[nytian]

Close the issue as the PR is merged.

[lilyjma]

(Unpinning issue since it's fixed.)