jupyter not in sirepo docker?

jeinstei commented 2 years ago

Container Version: Created 2022-05-17T17:54:56.571949063Z

Bug: jupyter sirepo app not in sirepo docker, but other codes seem to open and load fine. Instead receive an error message:

Not Found
The requested URL was not found on this server.

Expected: A jupyter interface when clicking on the jupyter app

Steps to reproduce:

pull sirepo docker: docker pull docker.io/radiasoft/sirepo
start docker using recommended command from README: docker run --rm -p 8000:8000 -v "$PWD:/sirepo" radiasoft/sirepo
Navigate to localhost:8000
Click on Supported Codes > Jupyter

URL in bar: http://localhost:8000/jupyterhublogin

172.17.0.1 - - [25/May/2022 15:14:50] "GET /static/img/favicon.png HTTP/1.1" 304 -
/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/sirepo/server.py:455:api_root unknown path=jupyterhublogin
/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/sirepo/uri_router.py:231:_dispatch exception=404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again. path=jupyterhublogin stack=Traceback (most recent call last):
  File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/sirepo/uri_router.py", line 229, in _dispatch
    return call_api(route.func, kwargs)
  File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/sirepo/uri_router.py", line 86, in call_api
    r = sirepo.http_reply.gen_exception(e)
  File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/sirepo/http_reply.py", line 60, in gen_exception
    return _gen_exception_werkzeug(exc)
  File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/sirepo/http_reply.py", line 386, in _gen_exception_werkzeug
    raise exc
  File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/sirepo/uri_router.py", line 77, in call_api
    r = flask.make_response(f(**kwargs) if kwargs else f())
  File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/sirepo/server.py", line 455, in api_root
    sirepo.util.raise_not_found(f'unknown path={path_info}')
  File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/sirepo/util.py", line 314, in raise_not_found
    _raise('NotFound', *args, **kwargs)
  File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/sirepo/util.py", line 409, in _raise
    raise getattr(werkzeug.exceptions, exc)()
werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.

Exception was printed at:

werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
  File "/home/vagrant/.pyenv/versions/3.7.2/lib/python3.7/threading.py", line 885, in _bootstrap
    self._bootstrap_inner()
  File "/home/vagrant/.pyenv/versions/3.7.2/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/home/vagrant/.pyenv/versions/3.7.2/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/home/vagrant/.pyenv/versions/3.7.2/lib/python3.7/socketserver.py", line 650, in process_request_thread
    self.finish_request(request, client_address)
  File "/home/vagrant/.pyenv/versions/3.7.2/lib/python3.7/socketserver.py", line 360, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/home/vagrant/.pyenv/versions/3.7.2/lib/python3.7/socketserver.py", line 720, in __init__
    self.handle()
  File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/werkzeug/serving.py", line 347, in handle
    BaseHTTPRequestHandler.handle(self)
  File "/home/vagrant/.pyenv/versions/3.7.2/lib/python3.7/http/server.py", line 426, in handle
    self.handle_one_request()
  File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/werkzeug/serving.py", line 379, in handle_one_request
    self.run_wsgi()
  File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/werkzeug/serving.py", line 324, in run_wsgi
    execute(self.server.app)
  File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/werkzeug/serving.py", line 313, in execute
    application_iter = app(environ, start_response)
  File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/flask/app.py", line 2091, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/flask/app.py", line 2073, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
172.17.0.1 - - [25/May/2022 15:14:52] "GET /jupyterhublogin HTTP/1.1" 404 -
172.17.0.1 - - [25/May/2022 15:14:52] "GET /favicon.ico HTTP/1.1" 200 -

e-carlin commented 2 years ago

Thanks for reporting. The problem is jupyter isn't configured when sirepo is run that way. That is why you get a 404. If you're curious this https://github.com/radiasoft/sirepo/blob/02accca312b115a60925e09a9090c614314798b6/container-conf/radia-run.sh is the script that get's run. Maybe when sirepo is started in this manner then we should remove jupyter from the list of supported codes?

jeinstei commented 2 years ago

Since all the other tools seem to run, it really does depend how you want to handle it. I'd personally rather see at least a basic jupyter server with notebooks be available, but I could easily see that increasing the docker size significantly depending on the python environments (pytorch == 1 GB). But if it doesn't start automatically, having it removed from the list would definitely be best from my perspective.

e-carlin commented 2 years ago

Thanks.

Here are my thoughts on the two options: Running jupyter:

In channel > dev we run jupyter using docker. You can't call docker from within docker so that won't work.
We could run using local servers (like we do in dev). Those require that jupyter is installed and the user logging in has a user account on the machine (the docker container). The only user is vagrant. So users would have to know to log in as vagrant@doesntmatter.foo. That's not the best user experience and would need to be communicated to them somehow. Maybe there is another option that could let the user run as whatever user the jupyterhub server is running as or something like that so they wouldn't have to log in

Removing from list of supported codes:

Easy enough, we configure the server when started this way to disable jupyterhublogin.

My preference is to disable jupyterhublogin. Running jupyterhub inside of docker is error prone and if users want to see jupyter they can go to sirepo.com/jupyter

robnagler commented 2 years ago

curl jupyter.run | bash

robnagler commented 2 years ago

@jeinstei did this fix your problem?

jeinstei commented 2 years ago

Not really -- this is about what exists in the individual docker that is hosted on docker.io, not about modifications needed to make it work. I have a solution that I can run a sirepo docker and a beamsim-jupyter docker separately for any sort of local or demo purposes -- otherwise I'm punting it back to @e-carlin's discussion above

robnagler commented 2 years ago

Jupyter is not in Sirepo's docker image by design. The failure you encountered will be fixed by https://github.com/radiasoft/sirepo/issues/4443.

Jupyter is in beamsim-jupyter, which really could be renamed radiasoft/jupyter. It contains Sirepo, but it is for single user jupyter, not JupyterHub.

radiasoft/jupyterhub does contain JupyterHub, but does not contain Sirepo or anything in beamsim.

IOW, starting JupyterHub is outside the scope of any particular image, because it involves starting many services with different images.

I have a solution that I can run a sirepo docker and a beamsim-jupyter docker separately for any sort of local or demo purposes

That's why we provide sirepo.run and jupyter.run. I don't think we want to offer other ways to do this to the general public, because it will become a maintenance headache. Starting the Jupyter Server is changing, and this should be fully encapsulated in jupyter.run.

jeinstei commented 2 years ago

As a third-party, jupyter.run is asking me to download a raw binary and run it locally, which I wouldn't be willing to do in terms of security in most cases. I'd much rather a docker for the limited security/containerization it provides.

Maybe I should just make a Dockerfile that just pulls and runs jupyter.run then.

Complete agreement here on the maintenance headache due to the different services. The only thing I've seen that could do similar would be a public k3 or Docker swarm file.

jeinstei commented 2 years ago

Actually, I'd probably be willing to do it if there was some sort of RSA check on the auto-download. But now we're still in to how to deploy on non-public systems.

robnagler commented 2 years ago

As a third-party, jupyter.run is asking me to download a raw binary and run it locally, which I wouldn't be willing to do in terms of security in most cases. I'd much rather a docker for the limited security/containerization it provides.

jupyter.run is a text file, not a raw binary. You can see that by typing curl jupyter.run. You can trace all the steps.

robnagler commented 2 years ago

Maybe I should just make a Dockerfile that just pulls and runs jupyter.run then.

jupyter.run wraps a docker pull and run so this actually wouldn't work.

Complete agreement here on the maintenance headache due to the different services. The only thing I've seen that could do similar would be a public k3 or Docker swarm file.

I do not understand the problem you are trying to solve. We have lots of documentation on how to do various things, including running a local Sirepo with JupyterHub.

Actually, I'd probably be willing to do it if there was some sort of RSA check on the auto-download. But now we're still in to how to deploy on non-public systems.

Which non-public systems are we talking about here? I don't know how an RSA check would help with this.

radiasoft / sirepo

jupyter not in sirepo docker? #4450