github-actions[bot]
commented
1 year ago This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activity on this issue. Kindly asking to recheck the report against latest Airflow version and let us know if the issue is reproducible. The issue will be closed in next 30 days if no further activity occurs from the issue author.
Apache Airflow version: 1.10.12
Kubernetes version (if you are using kubernetes) (use
kubectl version):Environment: Google Cloud Composer 1.13.3
What happened: The SDF operator workflow does not work correctly. When you create a new SDF operation using GoogleDisplayVideo360CreateSDFDownloadTaskOperator it successfully creates the new task. The next step in the process is to use the GoogleDisplayVideo360SDFtoGCSOperator to download the SDF file that was created. This second task relies on the "name" of the SDF operation that was created. The second task tries to use xcom to pull the name. However, the first task does not push the name to the xcom. Thus, the GoogleDisplayVideo360SDFtoGCSOperator always fails because it cannot download an operation with name 'None'.
The error is:
ERROR - Parameter "name" value "None" does not match the pattern "^sdfdownloadtasks/operations/[^/]+$"What you expected to happen: The GoogleDisplayVideo360SDFtoGCSOperator should be able to pull the name from the xcom and successfully download the file.
How to reproduce it: Run any workflow trying to create and download an SDF, like this example dag. The GoogleDisplayVideo360SDFtoGCSOperator will fail every time unless you hardcode the name of a valid SDF operation.
Anything else we need to know: I have written a working patch for the issue and will submit it shortly.