search

aws / amazon-mwaa-docker-images

Apache License 2.0
24 stars 11 forks source link

dag_stats fails in v2.9, but not in v2.8 #86

Closed agupta01 closed 3 months ago

agupta01 commented 3 months ago

Describe the bug A clear and concise description of what the bug is.

The dag_stats API used to return task statuses with MWAA runner v2.8, but it fails with v2.9.

To Reproduce Steps to reproduce the behavior:

  1. Start v2.8 local runner.
  2. POST http://localhost:8080/dag_stats Response 
    {
"aws_credential_verification": [
    {
        "count": 0,
        "state": "queued"
    },
    {
        "count": 1,
        "state": "success"
    },
    {
        "count": 0,
        "state": "running"
    },
    {
        "count": 0,
        "state": "failed"
    }
],
"dag_with_taskflow_api": [
    {
        "count": 0,
        "state": "queued"
    },
    {
        "count": 0,
        "state": "success"
    },
    {
        "count": 0,
        "state": "running"
    },
    {
        "count": 0,
        "state": "failed"
    }
]
}
  3. Terminate v2.8 local runner and start v2.9 local runner
  4. POST http://localhost:8080/dag_stats Response 
    <!doctype html>
<html lang=en>
<title>400 Bad Request</title>
<h1>Bad Request</h1>
<p>The CSRF token is missing.</p>

System Information

Expected behavior A clear and concise description of what you expected to happen.

The dag_stats API in both the 2.8 and 2.9 images should return the same response. This doesn't appear to be a breaking change on Airflow as the dag_stats is used to populate the task statuses in the web UI, and network calls reveal they have the same response. The only difference is the requirement of the CSRF token in 2.9.

Screenshots If applicable, add screenshots to help explain your problem.

2.8

image

2.9

image

Additional context Add any other context about the problem here.

reganbaum commented 3 months ago

I was able to get this working by adding the following 2 headers, X-Csrftoken and Cookie:

curl --location --request POST 'http://localhost:8080/dag_stats' \
--header 'X-Csrftoken: Ijc4ZDBm...' \
--header 'Cookie: session=200904e2-...' \
--data ''

Is it possible to make this call without requiring those headers?

reganbaum commented 3 months ago

I noticed that in 2.8, the WTF_CSRF_ENABLED is set to False, but in 2.9, it's currently set to True.

Can we change it to False for 2.9? Or alternatively, we can add an env var like MWAA__CORE__WTF_CSRF_ENABLED and set it on our side if we want it to be configurable?

rafidka commented 3 months ago

The reason for the change is that the 2.8 local runner was purely for testing purposes in case our customers want to run a MWAA-like environment to test things like requirements, start-up script, etc. However, the new repo is going to be used for MWAA itself, hence security becomes a must.

Or alternatively, we can add an env var like MWAACOREWTF_CSRF_ENABLED

That's probably a good solution. However, the flag should be under the 'webserver' section, i.e. MWAA__WEBSERVER__WTF_CSRF_ENABLED, since this is a web-server related configuration. The default should be set to true, i.e. to force the user to be explicit about not enforcing this security option.

rafidka commented 3 months ago

Addressed in #87 . Resolving.