Trouble launching local image of RSP

[bmorris3]

Hi all,

I followed the README instructions for building and running a local image to test recent updates towards #35. I created a clean conda env with py3.11 and the requirements.txt deps for this repo. I'm seeing the following terminal dialog when I run-lab:

(rsp-1) dorado2022:science-platform-images bmmorris$ run-lab
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Entered start.sh with args: start-notebook.sh
Running hooks in: /usr/local/bin/start-notebook.d as uid: 1000 gid: 100
Done running hooks in: /usr/local/bin/start-notebook.d
Running hooks in: /usr/local/bin/before-notebook.d as uid: 1000 gid: 100
Sourcing shell script: /usr/local/bin/before-notebook.d/10activate-conda-env.sh
Done running hooks in: /usr/local/bin/before-notebook.d
Executing the command: start-notebook.sh
WARNING: Use start-notebook.py instead
[I 2024-06-10 16:53:04.526 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2024-06-10 16:53:04.551 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2024-06-10 16:53:04.578 ServerApp] jupyterlab | extension was successfully linked.
[I 2024-06-10 16:53:04.605 ServerApp] notebook | extension was successfully linked.
[I 2024-06-10 16:53:04.610 ServerApp] Writing Jupyter server cookie secret to /home/jovyan/.local/share/jupyter/runtime/jupyter_cookie_secret
[I 2024-06-10 16:53:05.688 ServerApp] notebook_shim | extension was successfully linked.
[I 2024-06-10 16:53:05.689 ServerApp] voila.server_extension | extension was successfully linked.
[I 2024-06-10 16:53:05.781 ServerApp] notebook_shim | extension was successfully loaded.
[I 2024-06-10 16:53:05.791 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2024-06-10 16:53:05.796 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2024-06-10 16:53:05.812 LabApp] JupyterLab extension loaded from /opt/conda/envs/roman-cal/lib/python3.11/site-packages/jupyterlab
[I 2024-06-10 16:53:05.813 LabApp] JupyterLab application directory is /opt/conda/envs/roman-cal/share/jupyter/lab
[I 2024-06-10 16:53:05.815 LabApp] Extension Manager is 'pypi'.
[I 2024-06-10 16:53:05.912 ServerApp] jupyterlab | extension was successfully loaded.
[I 2024-06-10 16:53:05.935 ServerApp] notebook | extension was successfully loaded.
[I 2024-06-10 16:53:05.946 ServerApp] voila.server_extension | extension was successfully loaded.
[I 2024-06-10 16:53:05.950 ServerApp] Serving notebooks from local directory: /home/jovyan
[I 2024-06-10 16:53:05.950 ServerApp] Jupyter Server 2.14.1 is running at:
[I 2024-06-10 16:53:05.951 ServerApp] http://08bed0b4603b:8888/lab?token=e861eda143cf5994a4025e64bb1449adc0b646302d427bdd
[I 2024-06-10 16:53:05.951 ServerApp]     http://127.0.0.1:8888/lab?token=e861eda143cf5994a4025e64bb1449adc0b646302d427bdd
[I 2024-06-10 16:53:05.951 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2024-06-10 16:53:05.970 ServerApp]

    To access the server, open this file in a browser:
        file:///home/jovyan/.local/share/jupyter/runtime/jpserver-8-open.html
    Or copy and paste one of these URLs:
        http://08bed0b4603b:8888/lab?token=e861eda143cf5994a4025e64bb1449adc0b646302d427bdd
        http://127.0.0.1:8888/lab?token=e861eda143cf5994a4025e64bb1449adc0b646302d427bdd
[I 2024-06-10 16:53:08.965 ServerApp] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server

When I follow the link to that address, the browser hangs and never loads a page. I checked that no other notebooks are being served to that port.

Any pointers?

[jaytmiller]

This looks like a configuration issue. In the file infrequent-env there is a variable IMAGE_RUN_PARS. Try adding "--platform ${PLATFORM}" to that and re-sourcing setup-env, then run-lab. In theory that should make your docker runtime match the image you built, which by default is built cross-platform for linux/amd64 where evidently your real platform is a aarch64 (Mac / M-x /ARM).

[bmorris3]

Hi @jaytmiller – I'm still seeing the same:

(rsp-1) dorado2022:science-platform-images bmmorris$ source setup-env
(rsp-1) dorado2022:science-platform-images bmmorris$ run-lab
Entered start.sh with args: start-notebook.sh
Running hooks in: /usr/local/bin/start-notebook.d as uid: 1000 gid: 100
Done running hooks in: /usr/local/bin/start-notebook.d
Running hooks in: /usr/local/bin/before-notebook.d as uid: 1000 gid: 100
Sourcing shell script: /usr/local/bin/before-notebook.d/10activate-conda-env.sh
Done running hooks in: /usr/local/bin/before-notebook.d
Executing the command: start-notebook.sh
WARNING: Use start-notebook.py instead
[I 2024-06-11 13:56:28.439 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2024-06-11 13:56:28.460 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2024-06-11 13:56:28.484 ServerApp] jupyterlab | extension was successfully linked.
[I 2024-06-11 13:56:28.508 ServerApp] notebook | extension was successfully linked.
[I 2024-06-11 13:56:28.512 ServerApp] Writing Jupyter server cookie secret to /home/jovyan/.local/share/jupyter/runtime/jupyter_cookie_secret
[I 2024-06-11 13:56:29.432 ServerApp] notebook_shim | extension was successfully linked.
[I 2024-06-11 13:56:29.433 ServerApp] voila.server_extension | extension was successfully linked.
[I 2024-06-11 13:56:29.515 ServerApp] notebook_shim | extension was successfully loaded.
[I 2024-06-11 13:56:29.524 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2024-06-11 13:56:29.528 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2024-06-11 13:56:29.548 LabApp] JupyterLab extension loaded from /opt/conda/envs/roman-cal/lib/python3.11/site-packages/jupyterlab
[I 2024-06-11 13:56:29.548 LabApp] JupyterLab application directory is /opt/conda/envs/roman-cal/share/jupyter/lab
[I 2024-06-11 13:56:29.550 LabApp] Extension Manager is 'pypi'.
[I 2024-06-11 13:56:29.634 ServerApp] jupyterlab | extension was successfully loaded.
[I 2024-06-11 13:56:29.653 ServerApp] notebook | extension was successfully loaded.
[I 2024-06-11 13:56:29.665 ServerApp] voila.server_extension | extension was successfully loaded.
[I 2024-06-11 13:56:29.668 ServerApp] Serving notebooks from local directory: /home/jovyan
[I 2024-06-11 13:56:29.668 ServerApp] Jupyter Server 2.14.1 is running at:
[I 2024-06-11 13:56:29.668 ServerApp] http://5050b3b00987:8888/lab?token=709566e15795f5ca1307347dd16d4f1efd44cf74ba90b177
[I 2024-06-11 13:56:29.668 ServerApp]     http://127.0.0.1:8888/lab?token=709566e15795f5ca1307347dd16d4f1efd44cf74ba90b177
[I 2024-06-11 13:56:29.668 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2024-06-11 13:56:29.683 ServerApp]

    To access the server, open this file in a browser:
        file:///home/jovyan/.local/share/jupyter/runtime/jpserver-8-open.html
    Or copy and paste one of these URLs:
        http://5050b3b00987:8888/lab?token=709566e15795f5ca1307347dd16d4f1efd44cf74ba90b177
        http://127.0.0.1:8888/lab?token=709566e15795f5ca1307347dd16d4f1efd44cf74ba90b177
[I 2024-06-11 13:56:32.527 ServerApp] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server

[jaytmiller]

I understand it's not working yet but I'm glad this is gone now:

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

That looks like a show stopper so we're definitely better off with out it. Usually it is not needed on Macs, but did you try cutting and pasting http://127.0.0.1:8888/lab?token=709566e15795f5ca1307347dd16d4f1efd44cf74ba90b177 into a browser window this time? On some platforms/situations automatically launching the browser doesn't work.

[bmorris3]

Yes, neither URL opens, the browser just hangs.

[jaytmiller]

Which "Docker solution" are you using? (I'm using Colima, seems to work OK for this.)

[bmorris3]

I'm not sure what that means?

[jaytmiller]

Are you using Colima? Docker Desktop? Something else? (following on...) Are you on a Mac? What kind of processor?

[bmorris3]

Docker Desktop, Mac 12.6, M1 Max chip.

[bmorris3]

I've documented the process I'm following here: https://innerspace.stsci.edu/display/~bmmorris/Building+a+local+RSP+image

[jaytmiller]

The link seems dead, you may need to share it.

I don't have Desktop myself, but look for container port forwarding configuration in the GUI and make sure 8888 is exposed as 8888 if you haven't already. bizarrely, it may override the docker CLI run-lab is based on, but I have not verified it.

[bmorris3]

The link seems dead, you may need to share it.

Here's the contents:

I don't have Desktop myself, but look for container port forwarding configuration in the GUI and make sure 8888 is exposed as 8888 if you haven't already.

That all looks good. How would I use colima instead of the linked workflow?

[bmorris3]

You should have permissions to view that page here now: https://innerspace.stsci.edu/x/8CPSHQ

[jaytmiller]

If it is an option, you might want to try just "not running" Docker Desktop and using whatever CLI comes with it.

Otherwise Colima is an OSS brew install with instructions on GitHub here: https://github.com/abiosoft/colima Note that since I do not have Docker Desktop I'm not sure if they can coexist. Colima does need to be configured and Mac Mx's have complicated that.

[jaytmiller]

Attached colima.spec.txt are my colima options

[bmorris3]

It sounds like that might require some docker CLI configuration, in addition to colima configuration.

Is there documentation anywhere that explains all of the steps missing from what I wrote in the innerspace page?

[haticekaratay]

Hi @bmorris3 and @jaytmiller, I successfully built the Docker images with the latest main and ran the jupyter lab without any issues. However, I wanted to mention that only the link starting with localhost(aka 127.0.0.1) is working. The other link prompted on the terminal isn't functional. Is this what you were referring to when you mentioned issues with running it, Brett?

[haticekaratay]

Hi @bmorris3 and @jaytmiller, I successfully built the Docker images with the latest main and ran the jupyter lab without any issues. However, I wanted to mention that only the link starting with localhost(aka 127.0.0.1) is working. The other link prompted on the terminal isn't functional. Is this what you were referring to when you mentioned issues with running it, Brett?

To add to this, I only ran the following commands without specifying a conda environment.

scripts/image-configure  roman
source setup-env
image-build  
run-lan

[bmorris3]

I have only been trying to use localhost, and haven't had success.

[jaytmiller]

Docker CLI config, I'm not sure, but nobody else on our scrum team is reporting that. Colima setup is really easy once you have options picked. The spec I posted would show you my Colima choices which worked for me yesterday/today. (The PLATFORM fix we discussed above is needed longer term to avoid unworkable cross-configurations, I PR'ed that but you're the first person to require that particular fix.

There is no documentation dedicated to Desktop users yet.

One suggestion did occur to me: get a "hello world" web server image from DockerHub then make sure you can set it up for port 8888 and access it there. If not, that may lead you to the steps needed to make run-lab work w/o the worry run-lab is causing some unknown blocking problem. Also note that there is no requirement to use run-lab, it just encapsulates boilerplate so we don't have to remember it.

[jaytmiller]

I finally got Docker Desktop and it worked A-Z with no changes. I turned off Colima, I didn't change any Desktop settings other than giving it 32G of memory. I am running M2 pro I just got so possibly there are issues with M1's or s/w for M1's. I have OS-X Ventura 13.6.3. To me it looks like Jupyter Lab is running correctly inside Docker and your browser cannot reach it for some reason. Other than debugging generic port 8888 connectivity I'm not sure what to do.

[haticekaratay]

@jaytmiller, I believe this issue was resolved for @bmorris3 by pruning the system and rebuilding the Docker images. There might have been a problem with caching or something similar. Brett can add more details and possibly close the issue.