search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
36.42k stars 14.11k forks source link

Xcom view broken for non-JSON values #41981

Closed jkramer-ginkgo closed 2 weeks ago

jkramer-ginkgo commented 2 weeks ago

Apache Airflow version

2.10.0

If "Other Airflow 2 version" selected, which one?

No response

What happened?

Non-JSON values are broken in the Xcom UI [screenshot attached]

Root cause is this line from https://github.com/apache/airflow/pull/40640. Setting stringify: false results in JSON encoding which results in TypeError: keys must be str, int, float, bool or None, not tuple exception from GET <Airflow URL>/api/v1/dags/<DAG>/dagRuns/<Run ID>/taskInstances/<Task ID>/xcomEntries/<Xcom name>?stringify=false endpoint.

The PR's intention is to make the whole view JSON, so not sure if the intention is to error on non-JSON Xcom values, or if it should be updated to have clean fallback logic.

Pasted Graphic 22

What you think should happen instead?

No response

How to reproduce

Create Xcom value that is non-JSON serializable (e.g. {('201009_NB502104_0421_AHJY23BGXG (SEQ_WF: 138898)', None): 82359}) and then try to view in UI Xcom tab

Operating System

Linux (Ubuntu 22.04)

Versions of Apache Airflow Providers

No response

Deployment

Virtualenv installation

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

Code of Conduct

potiuk commented 2 weeks ago

Duplicate of https://github.com/apache/airflow/pull/41605 fixed in https://github.com/apache/airflow/pull/41516 and available in 2.10.1rc1. Can you please double check if rc1 of 2.10.0 fixes the problem for you @jkramer-ginkgo - see https://github.com/apache/airflow/issues/41956 for links and details

potiuk commented 2 weeks ago

Ideally pleasee comment in https://github.com/apache/airflow/issues/41956 when you confirm the problem is fixed.

jkramer-ginkgo commented 2 weeks ago

@potiuk It is still broken in 2.10.1.. I should've mentioned I had tested against a manually patched version with https://github.com/apache/airflow/pull/41605

jkramer-ginkgo commented 2 weeks ago

@potiuk would you be able to reopen or should I dupe this issue?

potiuk commented 2 weeks ago

Idealy copy it - this is easier for our "release" process.

jkramer-ginkgo commented 1 week ago

@potiuk Cloned here: https://github.com/apache/airflow/issues/42117