search

e-mission / op-admin-dashboard

An admin/deployer dashboard for the NREL OpenPATH platform
0 stars 10 forks source link

Finalize production docker build #32

Open shankari opened 1 year ago

shankari commented 1 year ago

I am not sure if the production docker build was ever finalized. I see a comment from @AlirezaRa94 https://github.com/e-mission/op-admin-dashboard/issues/21#issuecomment-1483731231 indicating that

I haven't yet tested the production docker-compose, and it is not ready for your testing. I will let you know when it's done.

I haven't seen a comment indicating that it is actually complete, and I see several issues in the startup logs below.

DB host = ...
Connecting to database URL ...
emission/tests/data/real_examples/shankari_2015-07-22
Loading file emission/tests/data/real_examples/shankari_2015-07-22
After registration, test_july_22 -> cd78a80c-45f1-49be-8985-7ad685902bb3
Finished loading 0 entries into the usercache and 1906 entries into the timeseries
WARNING:root:push service not configured, push notifications not supported
INFO:dash.dash:Dash is running on http://0.0.0.0:.../admin/
Connecting to database URL ...
Dash is running on http://0.0.0.0:.../admin/
* Serving Flask app 'app_sidebar_collapsible'
* Debug mode: off
INFO:werkzeug:WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
AlirezaRa94 commented 1 year ago

I tested the production docker-compose, and it worked without any problems. It is ready to use with an appropriate production server. However, some modifications to the Dockerfile or other files may be necessary before deploying it in a production environment.

shankari commented 1 year ago

@AlirezaRa94 it is not ready to work on production yet. As a concrete example, even the production docker-compose loads test data into the database on startup. This is not acceptable - it will pollute the real database and mess up the results

Please see the list of issues I've identified just by looking at the startup logs https://github.com/e-mission/op-admin-dashboard/issues/32#issue-1671473675

shankari commented 1 year ago

@AlirezaRa94 is there an ETA for fixing this? As I said via email, I would like to push the phone app to production this weekend, and I was hoping to push the admin dashboard to production at the same time as well as parallel features.

AlirezaRa94 commented 1 year ago

@AlirezaRa94 it is not ready to work on production yet. As a concrete example, even the production docker-compose loads test data into the database on startup. This is not acceptable - it will pollute the real database and mess up the results

Please see the list of issues I've identified just by looking at the startup logs #32 (comment)

  • loading test data on startup!!! ❗ ❗ 😱
  • push service is not configured
  • using a development server not suitable for production

As I mentioned here https://github.com/e-mission/op-admin-dashboard/issues/32#issuecomment-1519403713, it needs some modifications to other files.

AlirezaRa94 commented 1 year ago

@AlirezaRa94 is there an ETA for fixing this? As I said via email, I would like to push the phone app to production this weekend, and I was hoping to push the admin dashboard to production at the same time as well as parallel features.

The problem of loading test data has been solved.

shankari commented 1 year ago

@AlirezaRa94

The problem of loading test data has been solved.

Great! What about the "development server not suitable for production"?

AlirezaRa94 commented 1 year ago

@AlirezaRa94

The problem of loading test data has been solved.

Great! What about the "development server not suitable for production"?

As I know, you can use Gunicorn or uWSGI in your production environment.

shankari commented 1 year ago

@AlirezaRa94

As I know, you can use Gunicorn or uWSGI in your production environment.

Can you change the production docker-compose to support that? In general, it would be good to have the production docker file work "out of the box", only requiring API key/config file changes to work.

push service is not configured

I am fine with holding off on this one since it is a config, although we should probably document how to configure it.

shankari commented 1 year ago

@AlirezaRa94 Do you have an ETA for when the final production blocker will be done?

AlirezaRa94 commented 1 year ago

blocker

I will be on leave starting tomorrow for one month and won't be available during that time. I will be able to work on it once I return.

shankari commented 1 year ago

@AlirezaRa94 so you suggest that we wait for a month to deploy this to production?! I really don't see how that is acceptable I made a last minute, late night change in an hour to unblock the time use survey, and am disappointed that the admin dashboard will continue to be blocked for another whole month.

shankari commented 1 year ago

wrt

As I know, you can use Gunicorn or uWSGI in your production environment.

I tried using this with gunicorn

gunicorn app_sidebar_collapsible:server -b :8050

It fails with 

  File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
    worker.init_process()
  File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/gunicorn/workers/base.py", line 134, in init_process
    self.load_wsgi()
  File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
  File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/gunicorn/util.py", line 359, in import_app
    mod = importlib.import_module(module)
  File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/usr/src/app/app_sidebar_collapsible.py", line 40, in <module>
    app = Dash(
  File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/dash/dash.py", line 400, in __init__
    pages_folder=pages_folder_config(name, pages_folder, use_pages),
  File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/dash/_configs.py", line 136, in pages_folder_config
    raise exceptions.InvalidConfig(error_msg)
dash.exceptions.InvalidConfig: A folder called `pages` does not exist. If a folder for pages is not
required in your application, set `pages_folder=""`. For example:
`app = Dash(__name__,  pages_folder="")`

There is clearly a pages folder

app_sidebar_collapsible.py
pages

It's not clear what the working directory for gunicorn is

shankari commented 1 year ago

I am also not sure that we need to fix this ASAP. We already run this behind an ngnix proxy, and we assume that most other deployers will do so as well to get the microservices to work together. Will spend a little bit of time trying to get this to work but will punt on this for now otherwise

shankari commented 1 year ago

Looking at the initial commit, we had a file called Procfile https://github.com/e-mission/op-admin-dashboard/commit/67f6c5bf162dd500129aa848244da05567d9fa6e#diff-0a99231995da379e7aebabe76c9d849a23737a42c3b3a8994043e2aa80958424

with the contents

web: gunicorn app:server
shankari commented 1 year ago

Procfiles are used in heroku and similar web hosting platforms. So gunicorn app:server should work

But note that, while comparing app.py and app_sidebar_collapsible.py, the new app uses pages and the old one does not

Per https://stackoverflow.com/a/44664876/4040267 we should use chdir to specify the working directory for gunicorn

Let's see if that works...

shankari commented 1 year ago

Did not work. Same error

  File "/usr/src/app/app_sidebar_collapsible.py", line 40, in <module>
    app = Dash(
  File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/dash/dash.py", line 400, in __init__
    pages_folder=pages_folder_config(name, pages_folder, use_pages),
  File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/dash/_configs.py", line 136, in pages_folder_config
    raise exceptions.InvalidConfig(error_msg)
dash.exceptions.InvalidConfig: A folder called `pages` does not exist. If a folder for pages is not
required in your application, set `pages_folder=""`. For example:
`app = Dash(__name__,  pages_folder="")`

This has a clue: https://stackoverflow.com/questions/73906943/dash-app-cannot-find-pages-folder-when-deploying-on-gcp-using-gunicorn

But I'm not using DashProxy. This is taking too long and getting too complicated and I will punt on it for now given that it is running behind an ngnix reverse proxy anyway

For the record, the code to wrap the code in gunicorn should look something like this

diff --git a/docker/start.sh b/docker/start.sh
index 05b2f12..b961778 100755
--- a/docker/start.sh
+++ b/docker/start.sh
@@ -12,4 +12,13 @@ fi

 # run the app
 # python app.py
-python app_sidebar_collapsible.py
+if [[ $DASH_DEBUG_MODE == "True" ]]; then
+    echo "Debug mode set, running flask directly"
+    python app_sidebar_collapsible.py
+else
+    echo "Debug mode not set, running using gunicorn"
+    echo "We support only 3 admin users at a time, and they are unlikely to all be logged in at the same time"
+    echo "So let's stick to the default single worker thread to avoid issues"
+    gunicorn --chdir /usr/src/app app_sidebar_collapsible:server
+    # tail -f /dev/null
+fi