search

pnnl / ExaGO

High-performance power grid optimization for stochastic, security-constrained, and multi-period ACOPF problems.
Other
68 stars 9 forks source link

Generating container fails #71

Closed pelesh closed 12 months ago

pelesh commented 1 year ago

Issue type

Relates to

Summary

Generating docker image following ExaGO instructions fails.

Exact commands to reproduce, if applicable

On MacOS arch follow instructions to the letter.

Relevant logs and/or screenshots, if applicable

Error output: ``` => => transferring context: 56B 0.0s => [builder 2/13] RUN apt-get -yqq update && apt-get -yqq upgrade && 14.2s => [builder 3/13] RUN mkdir /opt/spack-environment && (echo spack: && 0.5s => [builder 4/13] COPY coinhsl-archive-2019.05.21.tar.gz /opt/spack-env 0.0s => [builder 5/13] RUN cd /opt/spack-environment && spack env activate . 6.0s => [builder 6/13] RUN cd /opt/spack-environment && spack env activate 521.3s => [builder 7/13] RUN cd /opt/spack-environment && spack env activat 1112.6s => ERROR [builder 8/13] RUN cd /opt/spack-environment && spack env ac 140.1s ------ > [builder 8/13] RUN cd /opt/spack-environment && spack env activate . && spack install --fail-fast petsc: 3.979 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/libiconv-1.17-iga2iuwmbar5rlubtrf3ptsa2a7o3uzm 3.979 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/gmake-4.4.1-75r7yzw3krhytcs7xqptozjuxw6zdbiv 3.979 ==> Installing xz-5.4.1-ubcqxu5i44k6cv5qaec2h52nlllcusdq [3/47] 3.979 ==> No binary for xz-5.4.1-ubcqxu5i44k6cv5qaec2h52nlllcusdq found: installing from source 4.483 ==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/dd/dd172acb53867a68012f94c17389401b2f274a1aa5ae8f84cbfb8b7e383ea8d3.tar.bz2 4.483 ==> No patches needed for xz 4.489 ==> xz: Executing phase: 'autoreconf' 4.495 ==> xz: Executing phase: 'configure' 12.70 ==> xz: Executing phase: 'build' 21.38 ==> xz: Executing phase: 'install' 23.19 ==> xz: Successfully installed xz-5.4.1-ubcqxu5i44k6cv5qaec2h52nlllcusdq 23.19 Stage: 0.36s. Autoreconf: 0.00s. Configure: 8.20s. Build: 8.68s. Install: 1.69s. Post-install: 0.09s. Total: 19.05s 23.20 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/xz-5.4.1-ubcqxu5i44k6cv5qaec2h52nlllcusdq 24.14 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/nghttp2-1.57.0-6jehmemf3s5usdiqtkej3ydoyahy3s6r 24.14 ==> Installing libffi-3.4.4-gqqpti3kn5vqa5e7mc6ufqy52rcog26l [5/47] 24.14 ==> No binary for libffi-3.4.4-gqqpti3kn5vqa5e7mc6ufqy52rcog26l found: installing from source 24.35 ==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/d6/d66c56ad259a82cf2a9dfc408b32bf5da52371500b84745f7fb8b645712df676.tar.gz 24.35 ==> No patches needed for libffi 24.35 ==> libffi: Executing phase: 'autoreconf' 24.36 ==> libffi: Executing phase: 'configure' 30.39 ==> libffi: Executing phase: 'build' 31.74 ==> libffi: Executing phase: 'install' 32.08 ==> libffi: Successfully installed libffi-3.4.4-gqqpti3kn5vqa5e7mc6ufqy52rcog26l 32.08 Stage: 0.06s. Autoreconf: 0.00s. Configure: 6.02s. Build: 1.34s. Install: 0.27s. Post-install: 0.05s. Total: 7.78s 32.08 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/libffi-3.4.4-gqqpti3kn5vqa5e7mc6ufqy52rcog26l 33.16 ==> Installing findutils-4.9.0-tuub7pupj7k7hikttavztzi6ekhxmc3r [6/47] 33.16 ==> No binary for findutils-4.9.0-tuub7pupj7k7hikttavztzi6ekhxmc3r found: installing from source 33.52 ==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/a2/a2bfb8c09d436770edc59f50fa483e785b161a3b7b9d547573cb08065fd462fe.tar.xz 33.52 ==> Applied patch /opt/spack/var/spack/repos/builtin/packages/findutils/nonnull.patch 33.52 ==> findutils: Executing phase: 'autoreconf' 33.53 ==> findutils: Executing phase: 'configure' 71.89 ==> findutils: Executing phase: 'build' 76.50 ==> findutils: Executing phase: 'install' 77.90 ==> findutils: Successfully installed findutils-4.9.0-tuub7pupj7k7hikttavztzi6ekhxmc3r 77.90 Stage: 0.22s. Autoreconf: 0.00s. Configure: 38.35s. Build: 4.61s. Install: 1.26s. Post-install: 0.07s. Total: 44.55s 77.90 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/findutils-4.9.0-tuub7pupj7k7hikttavztzi6ekhxmc3r 78.95 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/zlib-ng-2.1.4-yigb7oxo5yoy6lfcreefhio5bprbrfod 78.95 ==> Installing libsigsegv-2.14-2gtgiwrfctti54245qjmhsoc2vkowdn3 [8/47] 78.95 ==> No binary for libsigsegv-2.14-2gtgiwrfctti54245qjmhsoc2vkowdn3 found: installing from source 79.12 ==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/cd/cdac3941803364cf81a908499beb79c200ead60b6b5b40cad124fd1e06caa295.tar.gz 79.12 ==> No patches needed for libsigsegv 79.13 ==> libsigsegv: Executing phase: 'autoreconf' 79.13 ==> libsigsegv: Executing phase: 'configure' 83.84 ==> libsigsegv: Executing phase: 'build' 84.71 ==> libsigsegv: Executing phase: 'install' 84.91 ==> libsigsegv: Successfully installed libsigsegv-2.14-2gtgiwrfctti54245qjmhsoc2vkowdn3 84.91 Stage: 0.03s. Autoreconf: 0.00s. Configure: 4.70s. Build: 0.86s. Install: 0.13s. Post-install: 0.05s. Total: 5.82s 84.91 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/libsigsegv-2.14-2gtgiwrfctti54245qjmhsoc2vkowdn3 85.89 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/berkeley-db-18.1.40-bw4gbyq362zopdk7r2yiiqp2wmkxjdfh 85.89 ==> Installing libmd-1.0.4-hvkah7zl2lbe3k5blalzhnnwtfbjs67c [10/47] 85.89 ==> No binary for libmd-1.0.4-hvkah7zl2lbe3k5blalzhnnwtfbjs67c found: installing from source 86.09 ==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/f5/f51c921042e34beddeded4b75557656559cf5b1f2448033b4c1eec11c07e530f.tar.xz 86.09 ==> No patches needed for libmd 86.09 ==> libmd: Executing phase: 'autoreconf' 86.14 ==> libmd: Executing phase: 'configure' 90.70 ==> libmd: Executing phase: 'build' 91.77 ==> libmd: Executing phase: 'install' 92.08 ==> libmd: Successfully installed libmd-1.0.4-hvkah7zl2lbe3k5blalzhnnwtfbjs67c 92.08 Stage: 0.04s. Autoreconf: 0.00s. Configure: 4.59s. Build: 1.06s. Install: 0.24s. Post-install: 0.05s. Total: 6.02s 92.08 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/libmd-1.0.4-hvkah7zl2lbe3k5blalzhnnwtfbjs67c 93.03 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/metis-5.1.0-7azwzl2xgjxrin2lqaxptj2d3jrzgxuv 93.03 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/bzip2-1.0.8-in57fqfaz6hyaqelvctbac2ytafuxwlk 93.03 ==> Installing zstd-1.5.5-e3tfgnyaqukts4k2jzv5mige6v4cnt5j [13/47] 93.03 ==> No binary for zstd-1.5.5-e3tfgnyaqukts4k2jzv5mige6v4cnt5j found: installing from source 93.29 ==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/98/98e9c3d949d1b924e28e01eccb7deed865eefebf25c2f21c702e5cd5b63b85e1.tar.gz 93.29 ==> No patches needed for zstd 93.30 ==> zstd: Executing phase: 'edit' 93.31 ==> zstd: Executing phase: 'build' 93.32 ==> zstd: Executing phase: 'install' 136.4 ==> zstd: Successfully installed zstd-1.5.5-e3tfgnyaqukts4k2jzv5mige6v4cnt5j 136.4 Stage: 0.09s. Edit: 0.00s. Build: 0.00s. Install: 42.97s. Post-install: 0.05s. Total: 43.16s 136.4 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/zstd-1.5.5-e3tfgnyaqukts4k2jzv5mige6v4cnt5j 137.5 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/openblas-0.3.25-yhvdc5frvmfruqnaivudtzq5marlygvp 137.5 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/diffutils-3.9-nggcs7jmd7jtwokiva2xkfbxqktb2a4b 137.5 ==> Installing util-macros-1.19.3-coq5xyesyc3ikwqh7ddx4x6svuldtda2 [16/47] 137.5 ==> No binary for util-macros-1.19.3-coq5xyesyc3ikwqh7ddx4x6svuldtda2 found: installing from source 137.7 ==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/0f/0f812e6e9d2786ba8f54b960ee563c0663ddbe2434bf24ff193f5feab1f31971.tar.bz2 137.7 ==> No patches needed for util-macros 137.7 ==> util-macros: Executing phase: 'autoreconf' 137.7 ==> util-macros: Executing phase: 'configure' 139.1 ==> util-macros: Executing phase: 'build' 139.1 ==> util-macros: Executing phase: 'install' 139.2 ==> util-macros: Successfully installed util-macros-1.19.3-coq5xyesyc3ikwqh7ddx4x6svuldtda2 139.2 Stage: 0.02s. Autoreconf: 0.00s. Configure: 1.34s. Build: 0.00s. Install: 0.02s. Post-install: 0.04s. Total: 1.48s 139.2 [+] /opt/software/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/util-macros-1.19.3-coq5xyesyc3ikwqh7ddx4x6svuldtda2 139.6 ==> Error: Couldn't find patch for package builtin.libffi with sha256: 070b1f3aa87f2b56f83aff38afc42157e1692bfaa580276ecdbad2048b818ed7. This usually means the patch was modified or removed. To fix this, either reconcretize or use the original package repository 139.6 ==> Updating view at /opt/views/view ------ Dockerfile-with-features:53 -------------------- 51 | 52 | # Install PETSc 53 | >>> RUN cd /opt/spack-environment && spack env activate . && spack install --fail-fast petsc 54 | 55 | # Install py-jupyterlab -------------------- ERROR: failed to solve: process "docker-shell cd /opt/spack-environment && spack env activate . && spack install --fail-fast petsc" did not complete successfully: exit code: 1 [1797783 ms] Error: Command failed: docker buildx build --load --build-arg BUILDKIT_INLINE_CACHE=1 -f /var/folders/b9/64xrxgrn12z891ttr3856zb1d4rlq3/T/devcontainercli/container-features/0.52.1-1700330517994/Dockerfile-with-features -t vsc-exago-c8cb40076f187c020704a3ecb671fc06d1aca656f975162185c3d1431ac55d34 --target dev_containers_target_stage --no-cache --pull --build-arg _DEV_CONTAINERS_BASE_IMAGE=dev_container_auto_added_stage_label /Users/55y/src/exago/exago/ExaGO [1797784 ms] at cAA (/Users/55y/.vscode/extensions/ms-vscode-remote.remote-containers-0.321.0/dist/spec-node/devContainersSpecCLI.js:463:1860) [1797784 ms] at process.processTicksAndRejections (node:internal/process/task_queues:95:5) [1797784 ms] at async zw (/Users/55y/.vscode/extensions/ms-vscode-remote.remote-containers-0.321.0/dist/spec-node/devContainersSpecCLI.js:462:1841) [1797784 ms] at async kK (/Users/55y/.vscode/extensions/ms-vscode-remote.remote-containers-0.321.0/dist/spec-node/devContainersSpecCLI.js:462:610) [1797784 ms] at async NAA (/Users/55y/.vscode/extensions/ms-vscode-remote.remote-containers-0.321.0/dist/spec-node/devContainersSpecCLI.js:479:3660) [1797784 ms] at async vC (/Users/55y/.vscode/extensions/ms-vscode-remote.remote-containers-0.321.0/dist/spec-node/devContainersSpecCLI.js:479:4775) [1797784 ms] at async etA (/Users/55y/.vscode/extensions/ms-vscode-remote.remote-containers-0.321.0/dist/spec-node/devContainersSpecCLI.js:613:10941) [1797784 ms] at async AtA (/Users/55y/.vscode/extensions/ms-vscode-remote.remote-containers-0.321.0/dist/spec-node/devContainersSpecCLI.js:613:10682) [1797806 ms] Exit code 1 [1797815 ms] Command failed: /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) --ms-enable-electron-run-as-node /Users/55y/.vscode/extensions/ms-vscode-remote.remote-containers-0.321.0/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /Users/55y/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-0b0ea4b4-bacc-4dc3-bd79-4ca08b0198481700330515815 --workspace-folder /Users/55y/src/exago/exago/ExaGO --workspace-mount-consistency cached --id-label devcontainer.local_folder=/Users/55y/src/exago/exago/ExaGO --id-label devcontainer.config_file=/Users/55y/src/exago/exago/ExaGO/.devcontainer/devcontainer.json --log-level debug --log-format json --config /Users/55y/src/exago/exago/ExaGO/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --build-no-cache --remove-existing-container --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root [1797815 ms] Exit code 1 ```

Possible fixes, if applicable

No obvious fixes.

cameronrutherford commented 1 year ago

Unforunately we will need more of the error output to really know what's going on here, and how to fix it.

Key is that we need to see what the concrete spec actually was, and then we can figure out a way to avoid this buggy dependency. We have also merged quite a few things into develop. I am rebuilding myself now, and am also going to ensure the viz frontend is built already into the container.

This will also greatly simplify example notebook, so I will clean that up as well.

cameronrutherford commented 1 year ago

@pelesh I was able to reproduce on my machine, and this issue seems to have originated from https://github.com/spack/spack/pull/41083

A temporary fix is I am now building with libffi@3.4.2, but I think this warrants a spack issue / comment on that PR.

I didn't look too hard to see if Spack PR already exists, but this change is 5 days old, so I think we happened to catch an edge case here...

cameronrutherford commented 1 year ago

This does motivate pinning of a spack version within the container that we are building, and would perhaps need some help from @rothpc and techniques discussed offline in order to maintain exago's package alongside the pinned version of Spack that won't be getting regular updates.

Then again it is nice to keep up to date with spack develop, and I think #52 and CI/CD to cache a build of this devconatiner will help a lot.

Unfortunately this build would not be with Ipopt until we can build ipopt+resolve.