Support serving the dashboard from a non-root path (ngnix proxy)

[swastis10]

While serving the dashboard from a non-root path - e.g. using an ngnix reverse proxy, we need to special configuration. Otherwise, dash tries to load the assets from the root URL, which fails.

We use an ngnix reverse proxy in the NREL hosted environment, where the app is hosted at https://[deployment]-openpath.nrel.gov/admin

The application was working fine when we are testing on localhost because the app is served from the root URL but when we deploy, the app is served from the /admin URL.

The error we get is:

In order to fix it, we have used url_base_pathname

A local URL prefix to use app-wide. Default '/'. Both requests_pathname_prefix and routes_pathname_prefix default to url_base_pathname.

This launches the app at http://0.0.0.0:8050/admin/ instead of http://0.0.0.0:8050/

Dash is running on http://0.0.0.0:8050/admin/

However, this continued to fail on deployment with the error "The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again." aka 404

it seems like it is getting routed correctly to the dash app since https://openpath-stage.nrel.gov/admin1/ gives a different 404 and is redirected to https://www.nrel.gov/notfound

https://openpath-stage.nrel.gov/admin/ was still giving us a different 404, so presumably from within the dash app

On further investigation, this was because the app was published at http://0.0.0.0:8050/admin/, but the reverse proxy was still redirecting to proxy_pass http://[continername]:8050/;

So we ended up with

INFO:dash.dash:Dash is running on http://0.0.0.0:8050/admin/ INFO:werkzeug:192.168.1.73 - - [13/Apr/2023 02:30:46] "�[33mGET / HTTP/1.1�[0m" 404 -

Anyway, so I attempted to fix this by changing ngnix so that it would redirect to proxy_pass http://[containername]:8050/admin;

We are now getting an infinite redirect between http and https

[shankari]

Incidentally, the web app logs show a consistent 308 redirection

INFO:werkzeug:192.168.1.71 - - [13/Apr/2023 03:53:14] "GET /admin HTTP/1.1" 308 -
INFO:werkzeug:192.168.1.71 - - [13/Apr/2023 03:53:14] "GET /admin HTTP/1.1" 308 -
INFO:werkzeug:192.168.1.71 - - [13/Apr/2023 03:53:14] "GET /admin HTTP/1.1" 308 -
INFO:werkzeug:192.168.1.71 - - [13/Apr/2023 03:53:14] "GET /admin HTTP/1.1" 308 -
INFO:werkzeug:192.168.1.71 - - [13/Apr/2023 03:53:14] "GET /admin HTTP/1.1" 308 -

So from the logs above, the web app must be getting the https://openpath-stage.nrel.gov/admin requests and redirecting them to http. We need to figure out why.

For the record:

[shankari]

It could be that the redirect is from AWS cognito, but we do need to support cognito, so I decided to revert the previous changes, start investigating from scratch and fix properly.

[shankari]

Ok, so here's what's happening

As I expected, we are trying to retrieve the assets (e.g. dash_core_components.v2_9_0m1681396826.js) but because of the way that dash is set up, we are doing so from https://openpath-stage.nrel.gov/_dash-component-suites/dash/dcc/dash_core_components.v2_9_0m1681396826.js

instead of https://openpath-stage.nrel.gov/_dash-component-suites/admin/dash/dcc/dash_core_components.v2_9_0m1681396826.js

which is why none of the assets are found and the loading fails

we don't need to (and shouldn't need to) serve the whole app at admin we need to simply ensure that when the assets are loaded, they are prepended with admin

It seems like this should be handled by either requests_pathname_prefix or post_pathname_prefix and not url_base_pathname but the documentation is a bit unclear on which one.

[shankari]

Per https://github.com/plotly/dash/issues/489 we can do

app.config.update({
    # as the proxy server will remove the prefix
    "routes_pathname_prefix": "/", 
    # the front-end will prefix this string to the requests
    # that are made to the proxy server
    'requests_pathname_prefix': '/dev/'
})

and if we add the assets path assets_url_path='assets', then maybe everything will work now.

But this is an open source project; let's verify from the source code

[shankari]

Bingo (I think)! Note that the dash documentation says that

In some deployment environments, like Dash Enterprise, requests_pathname_prefix is set to the application name, e.g. my-dash-app.

However, when the app is deployed to a URL like /my-dash-app, then app.get_relative_path('/page-2') will return /my-dash-app/page-2. This can be used as an alternative to get_asset_url as well with app.get_relative_path('/assets/logo.png')

So this is a supported use case and should work as long as we get configure everything correctly.

Looking at the dash configuration

https://github.com/plotly/dash/blob/82e4fb3247146772f0996e8f1f37092eeba05509/dash/_configs.py#L110

    app_name = load_dash_env_vars().DASH_APP_NAME

    if not requests_pathname_prefix and app_name:
        requests_pathname_prefix = "/" + app_name + routes_pathname_prefix
    elif requests_pathname_prefix is None:
        requests_pathname_prefix = routes_pathname_prefix

the requests_pathname_prefix in the enterprise case is set by prepending the app_name to the routes_pathname_prefix

This indicates that the config here https://github.nrel.gov/nrel-cloud-computing/nrelopenpath-admin-dashboard/issues/9#issuecomment-49525 is correct. Since the routes_pathname_prefix defaults to /

    if url_base_pathname is not None and routes_pathname_prefix is None:
        routes_pathname_prefix = url_base_pathname
    elif routes_pathname_prefix is None:
        routes_pathname_prefix = "/"

we should just need

    # the front-end will prefix this string to the requests
    # that are made to the proxy server
    'requests_pathname_prefix': '/admin/'

[shankari]

Bingo!

I don't even think that we need to change any code to do this; we just need to set the DASH_REQUESTS_PATHNAME_PREFIX

Yup!

we initialize it from

    requests_pathname_prefix = get_combined_config(
        "requests_pathname_prefix", requests_pathname_prefix
    )

and get_combined_config reads the environment value if we don't provide one on input we can ask Jianli to set this up, or put it into config.py

def get_combined_config(name, val, default=None):
    """Consolidate the config with priority from high to low provided init
    value > OS environ > default."""

    if val is not None:
        return val

    env = load_dash_env_vars().get(f"DASH_{name.upper()}")
    if env is None:
        return default

    return env.lower() == "true" if env.lower() in {"true", "false"} else env

[shankari]

Moving on, I can log in with the AWS cognito authenticator, but the data is not displayed. There's a 500 error on the server.

[shankari]

One of the errors is

ERROR:app_sidebar_collapsible:Exception on /_dash-update-component [POST]
Traceback (most recent call last):
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/flask/app.py", line 2528, in wsgi_app
response = self.full_dispatch_request()
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/flask/app.py", line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/flask/app.py", line 1823, in full_dispatch_request
rv = self.dispatch_request()
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/flask/app.py", line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/dash/dash.py", line 1289, in dispatch
callback_context=g,
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/dash/_callback.py", line 447, in add_context
output_value = func(*func_args, **func_kwargs) # %% callback invoked %%
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/dash/dash.py", line 2064, in update
self.strip_relative_path(pathname)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/dash/dash.py", line 1519, in strip_relative_path
self.config.requests_pathname_prefix, path
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/dash/_get_paths.py", line 141, in app_strip_relative_path
"""
dash.exceptions.UnsupportedRelativePath: Paths that aren't prefixed with requests_pathname_prefix are not supported.

[shankari]

ok so the paths error is from https://github.com/plotly/dash/blob/4b03e5ac9e6406a4fd35f8b7b2fde33663959885/dash/_get_paths.py#L139

def app_strip_relative_path(requests_pathname, path):
    if path is None:
        return None
    if (
        requests_pathname != "/" and not path.startswith(requests_pathname.rstrip("/"))
    ) or (requests_pathname == "/" and not path.startswith("/")):
        raise exceptions.UnsupportedRelativePath(
            f"""
            Paths that aren't prefixed with requests_pathname_prefix are not supported.
            You supplied: {path} and requests_pathname_prefix was {requests_pathname}
            """
        )

In the backtrace, I don't see any of our code

everything is from /root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/dash/

[shankari]

The related stripping happens here

            def update(pathname, search):
                """
                Updates dash.page_container layout on page navigation.
                Updates the stored page title which will trigger the clientside callback to update the app title
                """

                query_parameters = _parse_query_string(search)
                page, path_variables = self._path_to_page(
                    self.strip_relative_path(pathname)
                )

Ok so that's pretty straightforward the paths are like so: https://openpath-stage.nrel.gov/data https://openpath-stage.nrel.gov/tokens

instead of

https://openpath-stage.nrel.gov/admin/data https://openpath-stage.nrel.gov/admin/tokens

as reported by @ssharma before

We just need to use the relative URLs https://dash.plotly.com/reference#dash.get_relative_path

[shankari]

Fortunately, there are not a lot of hrefs

$ grep -r href .
./utils/cognito_utils.py:                dbc.Button('Login with AWS Cognito', id='login-button', href=CognitoConfig.AUTH_URL, style={
./app_sidebar_collapsible.py:                    href=url_path_prefix,
./app_sidebar_collapsible.py:                    href=url_path_prefix + "data",
./app_sidebar_collapsible.py:                    href=url_path_prefix + "tokens",
./app_sidebar_collapsible.py:                    href=url_path_prefix + "map",
./app_sidebar_collapsible.py:                    href=url_path_prefix + "push_notification",
./app_sidebar_collapsible.py:                    href=url_path_prefix + "settings",

[shankari]

Bingo!

We can now navigate across all the pages and even see the tokens

Still seeing a few 500 errors and the tables are not showing up

[shankari]

The 500 error is at

File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/dash/_callback.py", line 447, in add_context
output_value = func(*func_args, **func_kwargs) # %% callback invoked %%
File "app_sidebar_collapsible.py", line 190, in update_store_trips
df = query_confirmed_trips(start_date_obj, end_date_obj)
File "/usr/src/app/utils/db_utils.py", line 66, in query_confirmed_trips
df = pd.json_normalize(list(query_result))
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/pymongo/cursor.py", line 1207, in next
if len(self.__data) or self._refresh():
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/pymongo/cursor.py", line 1124, in _refresh
self.__send_message(q)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/pymongo/cursor.py", line 1001, in __send_message
address=self.__address)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
exhaust=exhaust)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1471, in _retryable_read
return func(session, server, sock_info, slave_ok)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1366, in _cmd
unpack_res)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/pymongo/server.py", line 137, in run_operation_with_response
first, sock_info.max_wire_version)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/pymongo/helpers.py", line 168, in _check_command_response
max_wire_version)
pymongo.errors.OperationFailure: Internal server error, full error: {'ok': 0.0, 'operationTime': Timestamp(1681425547, 1), 'code': 42, 'errmsg': 'Internal server error'}

Related code is here https://github.nrel.gov/nrel-cloud-computing/nrelopenpath-admin-dashboard/blob/dev/utils/db_utils.py#L65

and is a fairly simple query/projection.

This is a documentDB issue so tracking it in a separate issue.

[shankari]

reverted @swastis10's earlier changes using force-push https://github.com/e-mission/op-admin-dashboard/pull/25

[shankari]

To fix this, we first need to be able to reproduce it, and then we need to show that we have fixed it. To reproduce, we need to be able to run the app with a reverse proxy so that we can verify that our fixes work