Xcom view broken for non-JSON values

[jkramer-ginkgo]

Apache Airflow version

Other Airflow 2 version (please specify below)

If "Other Airflow 2 version" selected, which one?

2.10.1

What happened?

Non-JSON values are broken in the Xcom UI [screenshot attached]

Root cause is this line from https://github.com/apache/airflow/pull/40640. Setting stringify: false results in JSON encoding which results in TypeError: keys must be str, int, float, bool or None, not tuple exception from GET <Airflow URL>/api/v1/dags/<DAG>/dagRuns/<Run ID>/taskInstances/<Task ID>/xcomEntries/<Xcom name>?stringify=false endpoint.

The PR's intention is to make the whole view JSON, so not sure if the intention is to error on non-JSON Xcom values, or if it should be updated to have clean fallback logic.

[This is a dupe of #41981]

What you think should happen instead?

No response

How to reproduce

Create Xcom value that is non-JSON serializable (e.g. {('201009_NB502104_0421_AHJY23BGXG (SEQ_WF: 138898)', None): 82359}) and then try to view in UI Xcom tab

Operating System

Linux (Ubuntu 22.04)

Versions of Apache Airflow Providers

No response

Deployment

Virtualenv installation

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[jscheffl]

I tried to reproduce this if it is fixed with another fix I did in 2.10.2(rc1) but failed creating the XCom. When using in code:

ti.xcom_push("non_json", {('201009_NB502104_0421_AHJY23BGXG (SEQ_WF: 138898)', None): 82359})

I get the error TypeError: keys must be str, int, float, bool or None, not tuple already during task execution,

How did you generate the data or can you please paste a piece of the example how you generated it? Or even better, can you check if 2.10.2rc1 is fixing it already?

[jkramer-ginkgo]

Maybe there's some diff between our configs and/or DAG-style but this xcom is properly being set and used. The dag is set up using PythonOperators and chain()s for dependency management.

[example]

context["ti"].xcom_push(
    key="key", value={('201009_NB502104_0421_AHJY23BGXG (SEQ_WF: 138898)', None): 82359}
)

[jkramer-ginkgo]

Yes, we have enable_xcom_pickling = True set. As this is a supported config, it would imply non-JSON serializable Xcom values should be supported in the UI.

[jscheffl]

Oh yeah. With AIRFLOW__CORE__ENABLE_XCOM_PICKLING=True this is a problem. Probably I/we did not consider this when updating the UI. This is (now) clearly a bug.