Closed swastis10 closed 1 year ago
Incidentally, the web app logs show a consistent 308 redirection
INFO:werkzeug:192.168.1.71 - - [13/Apr/2023 03:53:14] "[32mGET /admin HTTP/1.1[0m" 308 -
INFO:werkzeug:192.168.1.71 - - [13/Apr/2023 03:53:14] "[32mGET /admin HTTP/1.1[0m" 308 -
INFO:werkzeug:192.168.1.71 - - [13/Apr/2023 03:53:14] "[32mGET /admin HTTP/1.1[0m" 308 -
INFO:werkzeug:192.168.1.71 - - [13/Apr/2023 03:53:14] "[32mGET /admin HTTP/1.1[0m" 308 -
INFO:werkzeug:192.168.1.71 - - [13/Apr/2023 03:53:14] "[32mGET /admin HTTP/1.1[0m" 308 -
So from the logs above, the web app must be getting the https://openpath-stage.nrel.gov/admin requests and redirecting them to http. We need to figure out why.
For the record:
It could be that the redirect is from AWS cognito, but we do need to support cognito, so I decided to revert the previous changes, start investigating from scratch and fix properly.
Ok, so here's what's happening
As I expected, we are trying to retrieve the assets (e.g. dash_core_components.v2_9_0m1681396826.js) but because of the way that dash is set up, we are doing so from https://openpath-stage.nrel.gov/_dash-component-suites/dash/dcc/dash_core_components.v2_9_0m1681396826.js
which is why none of the assets are found and the loading fails
we don't need to (and shouldn't need to) serve the whole app at admin
we need to simply ensure that when the assets are loaded, they are prepended with admin
It seems like this should be handled by either requests_pathname_prefix
or post_pathname_prefix
and not url_base_pathname
but the documentation is a bit unclear on which one.
Per https://github.com/plotly/dash/issues/489 we can do
app.config.update({
# as the proxy server will remove the prefix
"routes_pathname_prefix": "/",
# the front-end will prefix this string to the requests
# that are made to the proxy server
'requests_pathname_prefix': '/dev/'
})
and if we add the assets path assets_url_path='assets',
then maybe everything will work now.
But this is an open source project; let's verify from the source code
Bingo (I think)! Note that the dash documentation says that
In some deployment environments, like Dash Enterprise, requests_pathname_prefix is set to the application name, e.g. my-dash-app.
However, when the app is deployed to a URL like /my-dash-app, then app.get_relative_path('/page-2') will return /my-dash-app/page-2. This can be used as an alternative to get_asset_url as well with app.get_relative_path('/assets/logo.png')
So this is a supported use case and should work as long as we get configure everything correctly.
Looking at the dash configuration
https://github.com/plotly/dash/blob/82e4fb3247146772f0996e8f1f37092eeba05509/dash/_configs.py#L110
app_name = load_dash_env_vars().DASH_APP_NAME
if not requests_pathname_prefix and app_name:
requests_pathname_prefix = "/" + app_name + routes_pathname_prefix
elif requests_pathname_prefix is None:
requests_pathname_prefix = routes_pathname_prefix
the requests_pathname_prefix
in the enterprise case is set by prepending the app_name
to the routes_pathname_prefix
This indicates that the config here
https://github.nrel.gov/nrel-cloud-computing/nrelopenpath-admin-dashboard/issues/9#issuecomment-49525
is correct. Since the routes_pathname_prefix
defaults to /
if url_base_pathname is not None and routes_pathname_prefix is None:
routes_pathname_prefix = url_base_pathname
elif routes_pathname_prefix is None:
routes_pathname_prefix = "/"
we should just need
# the front-end will prefix this string to the requests
# that are made to the proxy server
'requests_pathname_prefix': '/admin/'
Bingo!
I don't even think that we need to change any code to do this; we just need to set the DASH_REQUESTS_PATHNAME_PREFIX
Yup!
we initialize it from
requests_pathname_prefix = get_combined_config(
"requests_pathname_prefix", requests_pathname_prefix
)
and get_combined_config
reads the environment value if we don't provide one on input
we can ask Jianli to set this up, or put it into config.py
def get_combined_config(name, val, default=None):
"""Consolidate the config with priority from high to low provided init
value > OS environ > default."""
if val is not None:
return val
env = load_dash_env_vars().get(f"DASH_{name.upper()}")
if env is None:
return default
return env.lower() == "true" if env.lower() in {"true", "false"} else env
Moving on, I can log in with the AWS cognito authenticator, but the data is not displayed. There's a 500 error on the server.
One of the errors is
ERROR:app_sidebar_collapsible:Exception on /_dash-update-component [POST]
Traceback (most recent call last):
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/flask/app.py", line 2528, in wsgi_app
response = self.full_dispatch_request()
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/flask/app.py", line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/flask/app.py", line 1823, in full_dispatch_request
rv = self.dispatch_request()
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/flask/app.py", line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/dash/dash.py", line 1289, in dispatch
callback_context=g,
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/dash/_callback.py", line 447, in add_context
output_value = func(*func_args, **func_kwargs) # %% callback invoked %%
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/dash/dash.py", line 2064, in update
self.strip_relative_path(pathname)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/dash/dash.py", line 1519, in strip_relative_path
self.config.requests_pathname_prefix, path
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/dash/_get_paths.py", line 141, in app_strip_relative_path
"""
dash.exceptions.UnsupportedRelativePath: Paths that aren't prefixed with requests_pathname_prefix are not supported.
ok so the paths error is from https://github.com/plotly/dash/blob/4b03e5ac9e6406a4fd35f8b7b2fde33663959885/dash/_get_paths.py#L139
def app_strip_relative_path(requests_pathname, path):
if path is None:
return None
if (
requests_pathname != "/" and not path.startswith(requests_pathname.rstrip("/"))
) or (requests_pathname == "/" and not path.startswith("/")):
raise exceptions.UnsupportedRelativePath(
f"""
Paths that aren't prefixed with requests_pathname_prefix are not supported.
You supplied: {path} and requests_pathname_prefix was {requests_pathname}
"""
)
In the backtrace, I don't see any of our code
everything is from /root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/dash/
The related stripping happens here
def update(pathname, search):
"""
Updates dash.page_container layout on page navigation.
Updates the stored page title which will trigger the clientside callback to update the app title
"""
query_parameters = _parse_query_string(search)
page, path_variables = self._path_to_page(
self.strip_relative_path(pathname)
)
Ok so that's pretty straightforward the paths are like so: https://openpath-stage.nrel.gov/data https://openpath-stage.nrel.gov/tokens
instead of
https://openpath-stage.nrel.gov/admin/data https://openpath-stage.nrel.gov/admin/tokens
as reported by @ssharma before
We just need to use the relative URLs https://dash.plotly.com/reference#dash.get_relative_path
Fortunately, there are not a lot of hrefs
$ grep -r href .
./utils/cognito_utils.py: dbc.Button('Login with AWS Cognito', id='login-button', href=CognitoConfig.AUTH_URL, style={
./app_sidebar_collapsible.py: href=url_path_prefix,
./app_sidebar_collapsible.py: href=url_path_prefix + "data",
./app_sidebar_collapsible.py: href=url_path_prefix + "tokens",
./app_sidebar_collapsible.py: href=url_path_prefix + "map",
./app_sidebar_collapsible.py: href=url_path_prefix + "push_notification",
./app_sidebar_collapsible.py: href=url_path_prefix + "settings",
Bingo!
We can now navigate across all the pages and even see the tokens
Still seeing a few 500 errors and the tables are not showing up
The 500 error is at
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/dash/_callback.py", line 447, in add_context
output_value = func(*func_args, **func_kwargs) # %% callback invoked %%
File "app_sidebar_collapsible.py", line 190, in update_store_trips
df = query_confirmed_trips(start_date_obj, end_date_obj)
File "/usr/src/app/utils/db_utils.py", line 66, in query_confirmed_trips
df = pd.json_normalize(list(query_result))
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/pymongo/cursor.py", line 1207, in next
if len(self.__data) or self._refresh():
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/pymongo/cursor.py", line 1124, in _refresh
self.__send_message(q)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/pymongo/cursor.py", line 1001, in __send_message
address=self.__address)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
exhaust=exhaust)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1471, in _retryable_read
return func(session, server, sock_info, slave_ok)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1366, in _cmd
unpack_res)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/pymongo/server.py", line 137, in run_operation_with_response
first, sock_info.max_wire_version)
File "/root/miniconda-4.12.0/envs/emission/lib/python3.7/site-packages/pymongo/helpers.py", line 168, in _check_command_response
max_wire_version)
pymongo.errors.OperationFailure: Internal server error, full error: {'ok': 0.0, 'operationTime': Timestamp(1681425547, 1), 'code': 42, 'errmsg': 'Internal server error'}
Related code is here https://github.nrel.gov/nrel-cloud-computing/nrelopenpath-admin-dashboard/blob/dev/utils/db_utils.py#L65
and is a fairly simple query/projection.
This is a documentDB issue so tracking it in a separate issue.
reverted @swastis10's earlier changes using force-push https://github.com/e-mission/op-admin-dashboard/pull/25
To fix this, we first need to be able to reproduce it, and then we need to show that we have fixed it. To reproduce, we need to be able to run the app with a reverse proxy so that we can verify that our fixes work
While serving the dashboard from a non-root path - e.g. using an ngnix reverse proxy, we need to special configuration. Otherwise, dash tries to load the assets from the root URL, which fails.
We use an ngnix reverse proxy in the NREL hosted environment, where the app is hosted at https://[deployment]-openpath.nrel.gov/admin
The application was working fine when we are testing on localhost because the app is served from the root URL but when we deploy, the app is served from the /admin URL.
The error we get is:
In order to fix it, we have used url_base_pathname
This launches the app at
http://0.0.0.0:8050/admin/
instead ofhttp://0.0.0.0:8050/
However, this continued to fail on deployment with the error "The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again." aka 404
it seems like it is getting routed correctly to the dash app since https://openpath-stage.nrel.gov/admin1/ gives a different 404 and is redirected to https://www.nrel.gov/notfound
https://openpath-stage.nrel.gov/admin/ was still giving us a different 404, so presumably from within the dash app
On further investigation, this was because the app was published at http://0.0.0.0:8050/admin/, but the reverse proxy was still redirecting to proxy_pass http://[continername]:8050/;
So we ended up with
INFO:dash.dash:Dash is running on http://0.0.0.0:8050/admin/ INFO:werkzeug:192.168.1.73 - - [13/Apr/2023 02:30:46] "�[33mGET / HTTP/1.1�[0m" 404 -
Anyway, so I attempted to fix this by changing ngnix so that it would redirect to
proxy_pass http://[containername]:8050/admin;
We are now getting an infinite redirect between http and https