search

apache / superset

Apache Superset is a Data Visualization and Data Exploration Platform
https://superset.apache.org/
Apache License 2.0
63.06k stars 13.96k forks source link

fix: cache-warmup fails #31173

Open nsivarajan opened 4 days ago

nsivarajan commented 4 days ago

SUMMARY

This PR addresses #30900, which reports a "400 Bad Request: The CSRF session token is missing" error during cache warmup. The issue was due to missing header updates with the CSRF token and cookie (expecting session='session_cookie'). This PR ensures the token is correctly fetched and included in headers during the warmup process, resolving the error.

Fixes https://github.com/apache/superset/issues/30900

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Success:

...
...
[2024-11-26 19:00:00,054: INFO/ForkPoolWorker-62] cache-warmup[cad28d1b-df9c-4800-98bd-a09ec3af5d73]: Loading strategy
[2024-11-26 19:00:00,054: INFO/ForkPoolWorker-62] cache-warmup[cad28d1b-df9c-4800-98bd-a09ec3af5d73]: Loading DashboardTagsStrategy
[2024-11-26 19:00:00,055: INFO/ForkPoolWorker-62] cache-warmup[cad28d1b-df9c-4800-98bd-a09ec3af5d73]: Success!
[2024-11-26 19:00:00,131: INFO/ForkPoolWorker-62] fetch_url[90921f09-84a0-49ee-8b66-790864c72524]: Fetching https://sample.example.com/api/v1/security/csrf_token/
[2024-11-26 19:00:00,130: INFO/ForkPoolWorker-62] fetch_url[90921f09-84a0-49ee-8b66-790864c72524]: URL 'https://sample.example.com/api/v1/chart/warm_up_cache' is secure. Adding Referer header.
[2024-11-26 19:00:00,207: INFO/ForkPoolWorker-62] fetch_url[90921f09-84a0-49ee-8b66-790864c72524]: Fetching https://sample.example.com/api/v1/chart/warm_up_cache with payload {"chart_id": 1}
[2024-11-26 19:00:00,593: INFO/ForkPoolWorker-62] fetch_url[90921f09-84a0-49ee-8b66-790864c72524]: Fetched https://sample.example.com/api/v1/chart/warm_up_cache with payload {"chart_id": 1}, status code: 200
....
....
[2024-11-27 01:00:00,015: INFO/ForkPoolWorker-16] cache-warmup[35eba83f-4d8a-420c-9965-2a4a0411c891]: Loading strategy
[2024-11-27 01:00:00,016: INFO/ForkPoolWorker-16] cache-warmup[35eba83f-4d8a-420c-9965-2a4a0411c891]: Loading DashboardTagsStrategy
[2024-11-27 01:00:00,016: INFO/ForkPoolWorker-16] cache-warmup[35eba83f-4d8a-420c-9965-2a4a0411c891]: Success!
[2024-11-27 01:00:00,091: INFO/ForkPoolWorker-64] fetch_url[e9a5874d-0123-4d9b-8919-48b3bddd9ece]: Fetching https://sample.example.com/api/v1/security/csrf_token/
[2024-11-27 01:00:00,090: INFO/ForkPoolWorker-64] fetch_url[e9a5874d-0123-4d9b-8919-48b3bddd9ece]: URL 'https://sample.example.com/api/v1/chart/warm_up_cache' is secure. Adding Referer header.
[2024-11-27 01:00:00,163: INFO/ForkPoolWorker-64] fetch_url[e9a5874d-0123-4d9b-8919-48b3bddd9ece]: Fetching https://sample.example.com/api/v1/chart/warm_up_cache with payload {"chart_id": 1}
[2024-11-27 01:00:00,548: INFO/ForkPoolWorker-64] fetch_url[e9a5874d-0123-4d9b-8919-48b3bddd9ece]: Fetched https://sample.example.com/api/v1/chart/warm_up_cache with payload {"chart_id": 1}, status code: 200
...

TESTING INSTRUCTIONS

Since we are now explicitly fetching the CSRF token, we can enable WTF_CSRF_ENABLED = True or safely remove WTF_CSRF_EXEMPT_LIST for any cache warmup-related endpoints. This ensures that all requests, including those for cache warmup, are properly validated with CSRF protection, thereby enhancing application security.

If CSRF protection is disabled or the endpoint is exempted, no CSRF checks will be performed. However, the ChartRestApi.warm_up_cache endpoint still requires a session cookie (session=session_cookie) to determine the user's context and permissions. Without this, the request will fail with a 401 Unauthorized error.

This PR ensures the session cookie is correctly updated in the request headers under the Cookie field, enabling the user context to be accurately resolved for authorization.

ADDITIONAL INFORMATION

codecov[bot] commented 4 days ago

Codecov Report

Attention: Patch coverage is 53.84615% with 6 lines in your changes missing coverage. Please review.

Project coverage is 83.80%. Comparing base (76d897e) to head (e6a2aa5). Report is 1096 commits behind head on master.

Files with missing lines Patch % Lines
superset/tasks/cache.py 50.00% 6 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #31173 +/- ## =========================================== + Coverage 60.48% 83.80% +23.31% =========================================== Files 1931 536 -1395 Lines 76236 38943 -37293 Branches 8568 0 -8568 =========================================== - Hits 46114 32635 -13479 + Misses 28017 6308 -21709 + Partials 2105 0 -2105 ``` | [Flag](https://app.codecov.io/gh/apache/superset/pull/31173/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | |---|---|---| | [hive](https://app.codecov.io/gh/apache/superset/pull/31173/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.78% <15.38%> (-0.39%)` | :arrow_down: | | [javascript](https://app.codecov.io/gh/apache/superset/pull/31173/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | | | [mysql](https://app.codecov.io/gh/apache/superset/pull/31173/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `76.57% <53.84%> (?)` | | | [postgres](https://app.codecov.io/gh/apache/superset/pull/31173/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `76.63% <53.84%> (?)` | | | [presto](https://app.codecov.io/gh/apache/superset/pull/31173/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `53.28% <15.38%> (-0.53%)` | :arrow_down: | | [python](https://app.codecov.io/gh/apache/superset/pull/31173/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.80% <53.84%> (+20.31%)` | :arrow_up: | | [sqlite](https://app.codecov.io/gh/apache/superset/pull/31173/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `76.09% <53.84%> (?)` | | | [unit](https://app.codecov.io/gh/apache/superset/pull/31173/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `60.85% <0.00%> (+3.23%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.