apache / superset

Apache Superset is a Data Visualization and Data Exploration Platform
https://superset.apache.org/
Apache License 2.0
62.93k stars 13.92k forks source link

docker bug: werkzeug/utils.py" in send_from_directory #30183

Open vyouzhis opened 2 months ago

vyouzhis commented 2 months ago

Bug description

website img image

docker log -f

werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
2024-09-07 08:36:18,303:INFO:werkzeug:192.168.2.100 - - [07/Sep/2024 08:36:18] "GET /static/assets/images/superset-logo-horiz.png HTTP/1.1" 404 -
2024-09-07 08:36:18,364:INFO:werkzeug:192.168.2.100 - - [07/Sep/2024 08:36:18] "GET /static/appbuilder/css/webfonts/fa-solid-900.woff2 HTTP/1.1" 200 -
2024-09-07 08:36:18,429:WARNING:superset.views.base:HTTPException
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 713, in <lambda>
    view_func=lambda **kw: self_ref().send_static_file(**kw),  # type: ignore # noqa: B950
  File "/usr/local/lib/python3.10/site-packages/flask/scaffold.py", line 331, in send_static_file
    return send_from_directory(
  File "/usr/local/lib/python3.10/site-packages/flask/helpers.py", line 590, in send_from_directory
    return werkzeug.utils.send_from_directory(  # type: ignore[return-value]
  File "/usr/local/lib/python3.10/site-packages/werkzeug/utils.py", line 574, in send_from_directory
    raise NotFound()
werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
2024-09-07 08:36:18,430:INFO:werkzeug:192.168.2.100 - - [07/Sep/2024 08:36:18] "GET /static/assets/images/favicon.png HTTP/1.1" 404 -
2024-09-07 08:36:21,994:INFO:werkzeug:192.168.2.100 - - [07/Sep/2024 08:36:21] "POST /login/ HTTP/1.1" 302 -
2024-09-07 08:36:22,011:INFO:werkzeug:192.168.2.100 - - [07/Sep/2024 08:36:22] "GET / HTTP/1.1" 302 -
2024-09-07 08:36:22,065:DEBUG:superset.stats_logger:[stats_logger] (incr) welcome
2024-09-07 08:36:22,073:INFO:werkzeug:192.168.2.100 - - [07/Sep/2024 08:36:22] "GET /superset/welcome/ HTTP/1.1" 200 -
2024-09-07 08:36:22,157:WARNING:superset.views.base:HTTPException
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 713, in <lambda>
    view_func=lambda **kw: self_ref().send_static_file(**kw),  # type: ignore # noqa: B950
  File "/usr/local/lib/python3.10/site-packages/flask/scaffold.py", line 331, in send_static_file
    return send_from_directory(
  File "/usr/local/lib/python3.10/site-packages/flask/helpers.py", line 590, in send_from_directory
    return werkzeug.utils.send_from_directory(  # type: ignore[return-value]
  File "/usr/local/lib/python3.10/site-packages/werkzeug/utils.py", line 574, in send_from_directory
    raise NotFound()
werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
2024-09-07 08:36:22,158:INFO:werkzeug:192.168.2.100 - - [07/Sep/2024 08:36:22] "GET /static/assets/images/loading.gif HTTP/1.1" 404 -
2024-09-07 08:36:22,167:WARNING:superset.views.base:HTTPException
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 713, in <lambda>
    view_func=lambda **kw: self_ref().send_static_file(**kw),  # type: ignore # noqa: B950
  File "/usr/local/lib/python3.10/site-packages/flask/scaffold.py", line 331, in send_static_file
    return send_from_directory(
  File "/usr/local/lib/python3.10/site-packages/flask/helpers.py", line 590, in send_from_directory
    return werkzeug.utils.send_from_directory(  # type: ignore[return-value]
  File "/usr/local/lib/python3.10/site-packages/werkzeug/utils.py", line 574, in send_from_directory
    raise NotFound()
werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
2024-09-07 08:36:22,167:INFO:werkzeug:192.168.2.100 - - [07/Sep/2024 08:36:22] "GET /static/assets/images/favicon.png HTTP/1.1" 404 -
2024-09-07 08:36:43,056:DEBUG:superset.stats_logger:[stats_logger] (incr) health
2024-09-07 08:36:43,056:INFO:werkzeug:127.0.0.1 - - [07/Sep/2024 08:36:43] "GET /health HTTP/1.1" 200 -
2024-09-07 08:36:53,585:DEBUG:superset.stats_logger:[stats_logger] (incr) welcome
2024-09-07 08:36:53,593:INFO:werkzeug:192.168.2.100 - - [07/Sep/2024 08:36:53] "GET /superset/welcome/ HTTP/1.1" 200 -
2024-09-07 08:36:53,625:WARNING:superset.views.base:HTTPException
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 713, in <lambda>
    view_func=lambda **kw: self_ref().send_static_file(**kw),  # type: ignore # noqa: B950
  File "/usr/local/lib/python3.10/site-packages/flask/scaffold.py", line 331, in send_static_file
    return send_from_directory(
  File "/usr/local/lib/python3.10/site-packages/flask/helpers.py", line 590, in send_from_directory
    return werkzeug.utils.send_from_directory(  # type: ignore[return-value]
  File "/usr/local/lib/python3.10/site-packages/werkzeug/utils.py", line 574, in send_from_directory
    raise NotFound()
werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
2024-09-07 08:36:53,626:INFO:werkzeug:192.168.2.100 - - [07/Sep/2024 08:36:53] "GET /static/assets/images/loading.gif HTTP/1.1" 404 -
2024-09-07 08:37:13,160:DEBUG:superset.stats_logger:[stats_logger] (incr) health
2024-09-07 08:37:13,160:INFO:werkzeug:127.0.0.1 - - [07/Sep/2024 08:37:13] "GET /health HTTP/1.1" 200 -

How to reproduce the bug

  1. superset-4.0.2
  2. sudo docker compose up -d

Screenshots/recordings

No response

Superset version

master / latest-dev

Python version

3.9

Node version

16

Browser

Chrome

Additional context

No response

Checklist

dosubot[bot] commented 2 months ago

I found some similar issues and discussions that might help you resolve the 404 Not Found errors for static assets in Superset when using Docker:

  1. Closed Issue: GET /ws HTTP/1.1 werkzeug.exceptions.NotFound: 404 Not Found

    • This issue was resolved by ensuring that the static assets are correctly built and copied into the final Docker image [1].
  2. Closed Issue: NotFound errors on 2.0.1rc2

    • The suggested solution was to move to docker-compose and ensure proper setup and configuration [2].
  3. Closed Issue: [Errno 99] Cannot assign requested address in superset_init for docker-compose

    • A non-dev Docker compose file was added to help with frontend assets [3].

To resolve the 404 Not Found errors for static assets, follow these steps:

  1. Verify the build process: Ensure that the npm run build command completes without errors and generates the necessary static assets.

  2. Check the asset paths: Ensure that the paths to the static assets are correct and that they are being copied to the expected location in the final Docker image.

  3. Run the Docker build without skipping the build step: Make sure DEV_MODE is set to false to ensure the assets are built and copied:

    docker build --build-arg DEV_MODE=false -t your-superset-image .
  4. Configure Nginx correctly: Ensure that your Nginx configuration is set up to serve the static assets properly. Here is a sample configuration snippet from docker/nginx/nginx.conf:

    server {
       listen 80 default_server;
       server_name  _;
    
       location /ws {
           proxy_pass http://superset_websocket;
           proxy_http_version 1.1;
           proxy_set_header Upgrade $http_upgrade;
           proxy_set_header Connection "Upgrade";
           proxy_set_header Host $host;
       }
    
       location / {
           proxy_pass http://superset_app;
           proxy_set_header    Host                $host;
           proxy_set_header    X-Real-IP           $remote_addr;
           proxy_set_header    X-Forwarded-For     $remote_addr;
           proxy_set_header    X-Forwarded-Host    $host;
           proxy_set_header    X-Forwarded-Proto   $scheme;
           proxy_http_version 1.1;
           port_in_redirect off;
           proxy_connect_timeout 300;
       }
    }

Additionally, ensure that you follow the steps outlined in the Superset Docker documentation for proper setup and configuration:

  1. Prerequisites:

  2. Configuration:

    • The /app/pythonpath folder is mounted from ./docker/pythonpath_dev which contains a base configuration ./docker/pythonpath_dev/superset_config.py intended for use with local development.
    • For local overrides, copy ./docker/pythonpath_dev/superset_config_local.example to ./docker/pythonpath_dev/superset_config_docker.py and fill in your overrides.
  3. Initializing Database:

    • The database will initialize itself upon startup via the init container (superset-init). This may take a minute.
  4. Normal Operation:

    • To run the container, simply run: docker compose up
    • After waiting several minutes for Superset initialization to finish, you can open a browser and view http://localhost:8088.

By following these steps, you should be able to resolve the 404 Not Found errors for static assets in Superset when using Docker [4][5][6].

To continue talking to Dosu, mention @dosu.

pesekon2 commented 2 months ago

Getting exactly the same for the developer and non-dev images. The image one works for me but that one is not useful for me.

Darrenzzy commented 3 weeks ago

According to the configuration of Nginx in the Docker directory, there is no configuration directing static resources. It is recommended to make the following modifications:

Steps:

  1. Stop docker-compose first.

  2. Edit nginx.conf: vim docker/nginx/nginx.conf

Add and modify the following content:


server {
   listen 80 default_server;
   server_name  _;

   location /ws {
       proxy_pass http://superset_websocket;
       proxy_http_version 1.1;
       proxy_set_header Upgrade $http_upgrade;
       proxy_set_header Connection "Upgrade";
       proxy_set_header Host $host;
   }
   location /static {
            alias /app/superset-frontend/src;
            expires 30d;
            access_log off;
   }

   location / {
       proxy_pass http://superset_app;
       proxy_set_header    Host                $host;
       proxy_set_header    X-Real-IP           $remote_addr;
       proxy_set_header    X-Forwarded-For     $remote_addr;
       proxy_set_header    X-Forwarded-Host    $host;
       proxy_set_header    X-Forwarded-Proto   $scheme;
       proxy_http_version 1.1;
       port_in_redirect off;
       proxy_connect_timeout 300;
   }
}
  1. After making the changes, restart docker-compose:
docker compose up
mistercrunch commented 3 weeks ago

Related: https://github.com/apache/superset/pull/30752

fuomag9 commented 2 weeks ago

Same issue here, apache superset is unusable

mistercrunch commented 2 weeks ago

How did you build/start the docker?

fuomag9 commented 2 weeks ago

How did you build/start the docker?

By following the official documentation https://superset.apache.org/docs/installation/docker-compose/#option-1---for-an-interactive-development-environment

mistercrunch commented 2 weeks ago

Now that I think about it, after a pull/rebase you may have to run docker compose build, prior to docker compose up let me clarify in the docs

fuomag9 commented 2 weeks ago

Now that I think about it, after a pull/rebase you may have to run docker compose build, prior to docker compose up let me clarify in the docs

I don't think that's the problem, since when running docker compose up docker will try to build the image locally first, if defined inside the docker-compose file and alternatively it will try to pull the image defined for all containers from the specified or default registry

mistercrunch commented 2 weeks ago

I think it may matter in this case, from my experience, running docker compose up after say a rebase where requirements.txt has changed, won't run a build cycle while the app will now require the new lib and fail. Seems like the very existence of --build as an argument says that it won't build by default. Though note that if the image does not exist at all it will have to build. Note that this only affects docker-compose.yml and not the 2 other files as the former references a Dockerfile while others reference properly labeled image that are consistent with themselves.

Please try running it and report back.

Related: https://github.com/apache/superset/pull/30864

fuomag9 commented 2 weeks ago

I think it may matter in this case, from my experience, running docker compose up after say a rebase where requirements.txt has changed, won't run a build cycle while the app will now require the new lib and fail. Seems like the very existence of --build as an argument says that it won't build by default. Though note that if the image does not exist at all it will have to build. Note that this only affects docker-compose.yml and not the 2 other files as the former references a Dockerfile while others reference properly labeled image that are consistent with themselves.

Please try running it and report back.

Related: #30864

As expected, this does not fix the issue even after running. I can replicate this on another machine as well

superset_app          | 2024-11-06 23:11:00,682:WARNING:superset.views.error_handling:HTTPException
superset_app          | Traceback (most recent call last):
superset_app          |   File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1484, in full_dispatch_request
superset_app          |     rv = self.dispatch_request()
superset_app          |   File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1469, in dispatch_request
superset_app          |     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
superset_app          |   File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 502, in <lambda>
superset_app          |     view_func=lambda **kw: self_ref().send_static_file(**kw),  # type: ignore # noqa: B950
superset_app          |   File "/usr/local/lib/python3.10/site-packages/flask/scaffold.py", line 317, in send_static_file
superset_app          |     return send_from_directory(
superset_app          |   File "/usr/local/lib/python3.10/site-packages/flask/helpers.py", line 557, in send_from_directory
superset_app          |     return werkzeug.utils.send_from_directory(  # type: ignore[return-value]
superset_app          |   File "/usr/local/lib/python3.10/site-packages/werkzeug/utils.py", line 575, in send_from_directory
superset_app          |     raise NotFound()
superset_app          | werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
superset_app          | 2024-11-06 23:11:00,683:INFO:werkzeug:172.18.0.1 - - [06/Nov/2024 23:11:00] "GET /static/assets/images/loading.gif HTTP/1.1" 404 -
superset_app          | 2024-11-06 23:11:00,686:WARNING:superset.views.error_handling:HTTPException
superset_app          | Traceback (most recent call last):
superset_app          |   File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1484, in full_dispatch_request
superset_app          |     rv = self.dispatch_request()
superset_app          |   File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1469, in dispatch_request
superset_app          |     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
superset_app          |   File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 502, in <lambda>
superset_app          |     view_func=lambda **kw: self_ref().send_static_file(**kw),  # type: ignore # noqa: B950
superset_app          |   File "/usr/local/lib/python3.10/site-packages/flask/scaffold.py", line 317, in send_static_file
superset_app          |     return send_from_directory(
superset_app          |   File "/usr/local/lib/python3.10/site-packages/flask/helpers.py", line 557, in send_from_directory
superset_app          |     return werkzeug.utils.send_from_directory(  # type: ignore[return-value]
superset_app          |   File "/usr/local/lib/python3.10/site-packages/werkzeug/utils.py", line 575, in send_from_directory
superset_app          |     raise NotFound()
superset_app          | werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
superset_app          | 2024-11-06 23:11:00,687:INFO:werkzeug:172.18.0.1 - - [06/Nov/2024 23:11:00] "GET /static/assets/images/favicon.png HTTP/1.1" 404 -
mistercrunch commented 2 weeks ago

So what's different from your environment and everyone else's? Please share which git ref/sha you're using and local environment settings that may be relevant (platform, docker-version, docker-compose version). Also maybe nuking you docker environment might help (?) If that's reasonable to do in your env:

docker builder prune --all --force
docker system prune --all --volumes --force

@rusackas if we had this https://github.com/apache/superset/pull/29609 merged I could just say "please run this script and post the output", it would gather relevant info. Thinking about it I should add host-platform, cpu-type, as they would be relevant here. Probably makes sense to merge so that we can iterate on it.

fuomag9 commented 2 weeks ago

So what's different from your environment and everyone else's? Please share which git ref/sha you're using and local environment settings that may be relevant (platform, docker-version, docker-compose version). Also maybe nuking you docker environment might help (?) If that's reasonable to do in your env:

docker builder prune --all --force
docker system prune --all --volumes --force

@rusackas if we had this #29609 merged I could just say "please run this script and post the output", it would gather relevant info. Thinking about it I should add host-platform, cpu-type, as they would be relevant here. Probably makes sense to merge so that we can iterate on it.

Here you go

(base) ➜  superset git:(master)  git show-ref
de8282cea0d36c41ce2b780cc5f13ce9d5d3f0ee refs/heads/master
de8282cea0d36c41ce2b780cc5f13ce9d5d3f0ee refs/remotes/origin/HEAD
de8282cea0d36c41ce2b780cc5f13ce9d5d3f0ee refs/remotes/origin/master
(base) ➜  superset git:(master)  docker version
Client:
 Version:           27.3.1
 API version:       1.47
 Go version:        go1.22.7
 Git commit:        ce12230
 Built:             Fri Sep 20 11:38:18 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.35.1 (173168)
 Engine:
  Version:          27.3.1
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.22.7
  Git commit:       41ca978
  Built:            Fri Sep 20 11:41:19 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.21
  GitCommit:        472731909fa34bd7bc9c087e4c27943f9835f111
 runc:
  Version:          1.1.13
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
fuomag9 commented 2 weeks ago

@mistercrunch FYI https://github.com/apache/superset/pull/30752 fixes this correctly, I just tested it, so it's not related to the docker documentation but a lack of correct configuration in the repository files

mistercrunch commented 1 week ago

Still wondering why current docker-compose setup works in some environments and not others, but can review/merge https://github.com/apache/superset/pull/30752

mistercrunch commented 1 week ago

Not sure if/how it would matter, maybe local settings, but why not using http://localhost:8088 instead of http://{ip}:8088