gorenje / sysmlv2-jupyter-docker

Create docker image for running SysMLv2 with Jupyter
GNU Lesser General Public License v3.0
24 stars 18 forks source link

Update to latest SysML Release #10

Closed autoantwort closed 1 year ago

autoantwort commented 1 year ago

Thank you for this great project! This simplifies local testing a lot :)

gorenje commented 1 year ago

Thank you for your kind remarks 👍

I no longer maintain the repository but I've provided a description of the upgrade process if you like to create a PR.

PatrickF-mbse commented 1 year ago

@autoantwort @gorenje

Hi

Were you able to run successfully the make spin-up method? I tried on windows 10 and got various errors. ( A lot related to SSH...etc. )

Should this be run on a Linux? Or it should also be working with windows 10?

Thank you very much

PatrickF-mbse commented 1 year ago

@autoantwort @gorenje

Hi,

I was wondering it you had an idea about the root causes for these errors?

Error response from daemon: network with name thenetwork already exists ( this one probably due to my multiple try) Makefile:19: recipe for target 'create-periphery' failed mingw32-make: [create-periphery] Error 1 (ignored)

( About this one: do you think related to the make? I have installed mingw32-make on windows 10. ERROR: failed to solve: process "/bin/sh -c ./install.sh" did not complete successfully: exit code: 137 Makefile:15: recipe for target 'build-jupyter' failed mingw32-make: *** [build-jupyter] Error 1

long terminal screen below:

Thank you very much.

C:\y\dockerimages\sysmlv2-jupyter-docker-main> mingw32-make spin-up docker network create thenetwork Error response from daemon: network with name thenetwork already exists Makefile:19: recipe for target 'create-periphery' failed mingw32-make: [create-periphery] Error 1 (ignored) docker volume create postgresdbserver postgresdbserver docker build -t sysml.jupyter:2022-07 -f Dockerfile.jupyter --build-arg RELEASE=2022-07 . [+] Building 127.1s (19/28) => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load build definition from Dockerfile.jupyter 0.0s => => transferring dockerfile: 3.22kB 0.0s => [internal] load metadata for docker.io/library/openjdk:17-slim 1.3s => [internal] load build context 0.0s => => transferring context: 4.89kB 0.0s => [ 1/24] FROM docker.io/library/openjdk:17-slim@sha256:aaa3b3cb27e3e520b8f116863d0580c438ed55ecfa0bc126b41f68c3f62f9774 0.0s => CACHED [ 2/24] RUN apt-get --quiet --yes update && apt-get install -yqq gcc g++ wget inkscape texlive-fonts-recommended te 0.0s => CACHED [ 3/24] RUN adduser --disabled-password --gecos "Default user" --uid 1000 sysml 0.0s => CACHED [ 4/24] RUN chown -R 1000 /home/sysml 0.0s => CACHED [ 5/24] WORKDIR /home/sysml 0.0s => CACHED [ 6/24] RUN wget -q https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh 0.0s => CACHED [ 7/24] RUN wget -q https://github.com/Systems-Modeling/SysML-v2-Release/archive/2022-07.tar.gz?ts=20220819Z023100+00 -O 2022-07.tar.gz 0.0s => CACHED [ 8/24] RUN chmod 755 /home/sysml/Miniconda3-latest-Linux-x86_64.sh 0.0s => CACHED [ 9/24] RUN mkdir /home/sysml/conda 0.0s => CACHED [10/24] RUN /home/sysml/Miniconda3-latest-Linux-x86_64.sh -f -b -p /home/sysml/conda 0.0s => CACHED [11/24] RUN /home/sysml/conda/condabin/conda init 0.0s => CACHED [12/24] RUN tar xzf 2022-07.tar.gz 0.0s => CACHED [13/24] WORKDIR /home/sysml/SysML-v2-Release-2022-07/install/jupyter 0.0s => CACHED [14/24] RUN pip install euporie 0.0s => ERROR [15/24] RUN ./install.sh 125.7s

[15/24] RUN ./install.sh:

0 0.417 --- Step 1: Testing Conda installation ---

0 0.421 /home/sysml/conda/bin/conda

0 1.345 conda 23.3.1

0 1.434 --- Step 2: Testing Java installation ---

0 1.436 /usr/local/openjdk-17/bin/java

0 1.607 openjdk version "17.0.2" 2022-01-18

0 1.607 OpenJDK Runtime Environment (build 17.0.2+8-86)

0 1.607 OpenJDK 64-Bit Server VM (build 17.0.2+8-86, mixed mode, sharing)

0 1.616 --- Step 3: Installing Jupyter SysML kernel and dependencies ---

0 2.862 Collecting package metadata (current_repodata.json): ...working... done

0 31.48 Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.

0 31.48 Collecting package metadata (repodata.json): ...working... Killed


Dockerfile.jupyter:76

74 | 75 | ## Install SysMLv2 as Jupyter Kernel 76 | >>> RUN ./install.sh 77 | 78 | ## Point the publish command to the local API server.

ERROR: failed to solve: process "/bin/sh -c ./install.sh" did not complete successfully: exit code: 137 Makefile:15: recipe for target 'build-jupyter' failed mingw32-make: *** [build-jupyter] Error 1 PS C:\y\dockerimages\sysmlv2-jupyter-docker-main>

gorenje commented 1 year ago

I have no experience with Windows, so I can't say what is happening with make. The Makefile does everything via docker, so if we pull apart what is happening when make spin-up is called, it becomes:

#!/bin/bash

## change these as required
export release=2022-07
export sysml_release=2022-07

## create-periphery
docker network create thenetwork
docker volume create postgresdbserver
## build-jupyter
docker build -t sysml.jupyter:$(sysml_release) -f Dockerfile.jupyter --build-arg RELEASE=$(sysml_release) .
## build-api
docker build -t sysml.api:$(release) -f Dockerfile.api --build-arg RELEASE=$(release) .
## spin-up
SYSML_RELEASE=$(sysml_release) RELEASE=$(release) docker compose -f docker-compose.yml up

This is basically the spin-up rule in the Makefile.

So you can do everything that the makefile can by just executing each rule on its own, provided you have docker and bash (or some similar shell) installed.

If the docker commands fail, then there is something wrong with the compose files and/or the releases from SysMLv2

gorenje commented 1 year ago
#0 31.48 Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
#0 31.48 Collecting package metadata (repodata.json): ...working... Killed

Killed at the end looks to me like an operating-system/kernel thing where there are no more resources for the process and it was killed by the OS. That is a matter of giving docker more resources (whatever that be: memory or hard drive or ...)

gorenje commented 1 year ago
ERROR: failed to solve: process "/bin/sh -c ./install.sh" did not complete successfully: exit code: 137

That's SysMLv2 install script and it failed for a specific reason: error code 137! Whatever that may be ... or which ever step that might have been ... that's a specific something in the install.sh - i.e. something in there did a 137 error ...

PatrickF-mbse commented 1 year ago

@gorenje Hi,

Thank you very much for the follow up. Ok everything is happening inside the docker. I am newbie with Docker and tough I had the wrong windows "make". I will try to investigate the install.sh..maybe this is because the sysml v2 install.sh in the repository more recent than the one that was used when the makefile was created ( 2022-10)?.

Maybe I will get more chance trying to fix the SBT RUN/Java SDK error on windows for the manual installation process.

Anyway thank you very much for the follow up.

PatrickF-mbse commented 1 year ago

@gorenje

https://stackoverflow.com/questions/75516010/github-actions-workflow-throws-error-137-when-installing-dependencies

i found this about error 137 that seems to align with what you said about the memory ( someone wrote "Probably I have to resize the droplet? This is the current machine Regular Intel 1 vCPU, 512 MB, 10 Gb SSD, Transfer 0.5 TB") but this is above my systems engineer pay grade :) . I will try to go back to the manual install and try to fix the SBT RUN/Javascript SDK error.

Thank you anyway

gorenje commented 1 year ago

This is the 2022-07 version of install.sh or would seem to be it. You can always unpack the 2022-07 release tarball for complete assurity.

If you're using a droplet for this then 100% it's an out of memory issue (especially if you only have 512MB and running docker inside that). Java is notorious for taking up memory as if it's going out of fashion. I always built on my stand-around Linux PC and then as a GitHub action (to push to docker hub) - that always worked.

PatrickF-mbse commented 1 year ago

@gorenje

I am not sure what is dropplet. This was just an extract of the text I saw on the stackoverflow webpage that seems to refer to memory topics.

on my computer, I have 64BGB of RAM, windows 10 and docker installed then I run the the Make spin up. There maybe way to tell the install to use more memory as I noticed that it barely used more than 20% of what is available. A job for another day: if I find it, I will share here and on the group. Thank you very much anyway

gorenje commented 1 year ago

Then it might be case of giving docker more memory - docker also has memory limits and won't take all the 64GB available. My docker is installation is limited to 4 CPUs and 9GB memory - that seems to have worked.

gorenje commented 1 year ago

btw droplet the name given to computing instance at digitalocean.

gorenje commented 1 year ago

FYI more help at comment