tiangolo / meinheld-gunicorn-flask-docker

Docker image with Meinheld and Gunicorn for Flask applications in Python.
MIT License
486 stars 60 forks source link

Long Loading Time for Application #33

Closed raby1996 closed 3 years ago

raby1996 commented 4 years ago

Hello! First off let me thank you the great project, it's a wonderful starting point for those of use using a flask framework.

I'm somewhat new to building web applications using flask, and I am close to finishing one however I've been struggling to figure the cause of what seems like a thread lock, GIL or worker starvation of sorts. Basically these are my symptoms,

The first time I load my webpage it immediately starts loading the relevant content which is some static html with dynamic html that is rendered from ajax calls, so when you login you can visibly see the loading bars telling me that ajax has made the calls to my python routes to load data.

However I notice that if I have more than one session active, lets say I have 2 tabs open to the same page or if more that one user loads the page then the subsequent loads will be stalled, so when I login instead of immediately loading the page and starting the ajax calls it seems like it's frozen/stalled (DOM does not load) and I either have to wait or hard refresh the page several times for it to load correctly and start the ajax calls.

I've spent quite of bit of time trying to figure it out and these are my observations,

image

[10] [ERROR] ConnectionResetError: [Errno 104] Connection reset by peer

I attempted to debug it from a browser perspective but came up short and was hoping maybe you could provide some insight into the issue. If you need any additional information from me please let me know ! Thank you!

betoksoft commented 4 years ago

Hello,

I have this problem in a certain endpoint, this endpoint works when it is returning some response but when it returns an specific response it takes the same time 2 mins, first i tought that it was some kind of limit in the response size because i was returning a huge JSON document, but i got the same behaviour by returning just None.

I tweaked some of the gunicorn settings to test if any could solve this issue but none seems to fix it: timeout graceful-timeout workers threads worker_class (but i couldn't get it changed, i think is because it's meant to use egg:meinheld#gunicorn_worker by design)

The flask development server works perfect. This image works perfect: https://github.com/tiangolo/uwsgi-nginx-flask-docker

This issue is really weird, i'll try to replicate this problem in a minimal setup.

Regards

vumaasha commented 4 years ago

application loading takes more than 2 minutes. Below is my gunicorn conf. I am running a flask appbuilder application. Is there any chance to get this working? any pointers on how to debug will be very helpful.

import json
import multiprocessing
import os

workers_per_core_str = os.getenv("WORKERS_PER_CORE", "2")
max_workers_str = os.getenv("MAX_WORKERS")
use_max_workers = None
if max_workers_str:
    use_max_workers = int(max_workers_str)
web_concurrency_str = os.getenv("WEB_CONCURRENCY", None)

host = os.getenv("HOST", "0.0.0.0")
port = os.getenv("PORT", "80")
bind_env = os.getenv("BIND", None)
use_loglevel = os.getenv("LOG_LEVEL", "debug")
if bind_env:
    use_bind = bind_env
else:
    use_bind = f"{host}:{port}"

cores = multiprocessing.cpu_count()
workers_per_core = float(workers_per_core_str)
default_web_concurrency = workers_per_core * cores
if web_concurrency_str:
    web_concurrency = int(web_concurrency_str)
    assert web_concurrency > 0
else:
    web_concurrency = max(int(default_web_concurrency), 2)
    if use_max_workers:
        web_concurrency = min(web_concurrency, use_max_workers)
accesslog_var = os.getenv("ACCESS_LOG", "-")
use_accesslog = accesslog_var or None
errorlog_var = os.getenv("ERROR_LOG", "-")
use_errorlog = errorlog_var or None
graceful_timeout_str = os.getenv("GRACEFUL_TIMEOUT", "120")
timeout_str = os.getenv("TIMEOUT", "120")
keepalive_str = os.getenv("KEEP_ALIVE", "5")

# Gunicorn config variables
loglevel = use_loglevel
workers = web_concurrency
bind = use_bind
errorlog = use_errorlog
worker_tmp_dir = "/dev/shm"
accesslog = use_accesslog
graceful_timeout = int(graceful_timeout_str)
timeout = int(timeout_str)
keepalive = int(keepalive_str)

# For debugging and testing
log_data = {
    "loglevel": loglevel,
    "workers": workers,
    "bind": bind,
    "graceful_timeout": graceful_timeout,
    "timeout": timeout,
    "keepalive": keepalive,
    "errorlog": errorlog,
    "accesslog": accesslog,
    # Additional, non-gunicorn variables
    "workers_per_core": workers_per_core,
    "use_max_workers": use_max_workers,
    "host": host,
    "port": port,
}
print(json.dumps(log_data))
tiangolo commented 3 years ago

Hey everyone, loading time is probably related to different things for each case, depending on how you create your application, etc.

If you are still having problems, please create a minimal self-contained example, that can be used to replicate the issue.

github-actions[bot] commented 3 years ago

Assuming the original issue was solved, it will be automatically closed now. But feel free to add more comments or create new issues.

pramos84 commented 3 years ago

Hi, guys. To me, some static contents is taking about 2 min to load, too. The Chrome raises the error below:

image

My flask app is quiet simple. Even when loading a single static file in the browser, I can see the error:

image

My gunicorn configuration is the original from tiangolo/meinheld-gunicorn-flask image. It seems like the server is manipulating data without refreshing data length info.

Anyway, this container image do a great job, and bring impressive performance for my flask app (in dynamic content, at least).

Regards.

p4block commented 3 years ago

I've been having issues with this image for a while, it's basically impossible for flask to host any file >100KB without something in the chain completely stopping to send bytes.

I moved all pictures and resources to nginx and am now using flask only for the html/css/js, but even then, on heavier pages sometimes it just stops loading for no reason. F5 most of the time solves it instantly.