search

maresb / micromamba-devcontainer-example

MIT License
16 stars 4 forks source link

instructions for building do not work #14

Open aaronjnewman opened 1 year ago

aaronjnewman commented 1 year ago

I tried following the instructions for Cruft. I did not make any modifications to the files, just to see if it would build. It didn't. Output is below. It appears to throw errors for the lines with COPY --chown=mamba.

Any sugestions?

[2023-07-28T20:55:53.039Z] Dev Containers 0.299.0 in VS Code 1.80.2 (2ccd690cbff1569e4a83d7c43d45101f817401dc).
[2023-07-28T20:55:53.039Z] Start: Resolving Remote
[2023-07-28T20:55:53.072Z] Setting up container for folder or workspace: /Users/aaron/dev/VS Code containers/micromamba-cruft
[2023-07-28T20:55:53.072Z] Context: desktop-linux
[2023-07-28T20:55:53.073Z] Start: Check Docker is running
[2023-07-28T20:55:53.073Z] Start: Run: docker version --format {{.Server.APIVersion}}
[2023-07-28T20:55:53.148Z] Stop (75 ms): Run: docker version --format {{.Server.APIVersion}}
[2023-07-28T20:55:53.149Z] Server API version: 1.41
[2023-07-28T20:55:53.149Z] Stop (76 ms): Check Docker is running
[2023-07-28T20:55:53.149Z] Start: Run: docker volume ls -q
[2023-07-28T20:55:53.205Z] Stop (56 ms): Run: docker volume ls -q
[2023-07-28T20:55:53.206Z] Start: Run: docker ps -q -a --filter label=vsch.local.folder=/Users/aaron/dev/VS Code containers/micromamba-cruft --filter label=vsch.quality=stable
[2023-07-28T20:55:53.265Z] Stop (59 ms): Run: docker ps -q -a --filter label=vsch.local.folder=/Users/aaron/dev/VS Code containers/micromamba-cruft --filter label=vsch.quality=stable
[2023-07-28T20:55:53.266Z] Start: Run: docker ps -q -a --filter label=devcontainer.local_folder=/Users/aaron/dev/VS Code containers/micromamba-cruft --filter label=devcontainer.config_file=/Users/aaron/dev/VS Code containers/micromamba-cruft/.devcontainer/devcontainer.json
[2023-07-28T20:55:53.328Z] Stop (62 ms): Run: docker ps -q -a --filter label=devcontainer.local_folder=/Users/aaron/dev/VS Code containers/micromamba-cruft --filter label=devcontainer.config_file=/Users/aaron/dev/VS Code containers/micromamba-cruft/.devcontainer/devcontainer.json
[2023-07-28T20:55:53.329Z] Start: Run: docker ps -q -a --filter label=devcontainer.local_folder=/Users/aaron/dev/VS Code containers/micromamba-cruft
[2023-07-28T20:55:53.389Z] Stop (60 ms): Run: docker ps -q -a --filter label=devcontainer.local_folder=/Users/aaron/dev/VS Code containers/micromamba-cruft
[2023-07-28T20:55:53.390Z] Start: Run: docker ps -q -a --filter label=devcontainer.local_folder=/Users/aaron/dev/VS Code containers/micromamba-cruft
[2023-07-28T20:55:53.450Z] Stop (60 ms): Run: docker ps -q -a --filter label=devcontainer.local_folder=/Users/aaron/dev/VS Code containers/micromamba-cruft
[2023-07-28T20:55:53.451Z] Start: Run: /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) --ms-enable-electron-run-as-node /Users/aaron/.vscode/extensions/ms-vscode-remote.remote-containers-0.299.0/dist/spec-node/devContainersSpecCLI.js read-configuration --workspace-folder /Users/aaron/dev/VS Code containers/micromamba-cruft --id-label devcontainer.local_folder=/Users/aaron/dev/VS Code containers/micromamba-cruft --id-label devcontainer.config_file=/Users/aaron/dev/VS Code containers/micromamba-cruft/.devcontainer/devcontainer.json --log-level debug --log-format json --config /Users/aaron/dev/VS Code containers/micromamba-cruft/.devcontainer/devcontainer.json --mount-workspace-git-root true
[2023-07-28T20:55:53.621Z] @devcontainers/cli 0.46.0. Node.js v16.17.1. darwin 22.6.0 arm64.
[2023-07-28T20:55:53.621Z] Start: Run: git rev-parse --show-cdup
[2023-07-28T20:55:53.634Z] Stop (13 ms): Run: git rev-parse --show-cdup
[2023-07-28T20:55:53.634Z] Start: Run: docker ps -q -a --filter label=devcontainer.local_folder=/Users/aaron/dev/VS Code containers/micromamba-cruft --filter label=devcontainer.config_file=/Users/aaron/dev/VS Code containers/micromamba-cruft/.devcontainer/devcontainer.json
[2023-07-28T20:55:53.696Z] Stop (62 ms): Run: docker ps -q -a --filter label=devcontainer.local_folder=/Users/aaron/dev/VS Code containers/micromamba-cruft --filter label=devcontainer.config_file=/Users/aaron/dev/VS Code containers/micromamba-cruft/.devcontainer/devcontainer.json
[2023-07-28T20:55:53.699Z] Stop (248 ms): Run: /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) --ms-enable-electron-run-as-node /Users/aaron/.vscode/extensions/ms-vscode-remote.remote-containers-0.299.0/dist/spec-node/devContainersSpecCLI.js read-configuration --workspace-folder /Users/aaron/dev/VS Code containers/micromamba-cruft --id-label devcontainer.local_folder=/Users/aaron/dev/VS Code containers/micromamba-cruft --id-label devcontainer.config_file=/Users/aaron/dev/VS Code containers/micromamba-cruft/.devcontainer/devcontainer.json --log-level debug --log-format json --config /Users/aaron/dev/VS Code containers/micromamba-cruft/.devcontainer/devcontainer.json --mount-workspace-git-root true
[2023-07-28T20:55:53.699Z] Start: Run: /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) --ms-enable-electron-run-as-node /Users/aaron/.vscode/extensions/ms-vscode-remote.remote-containers-0.299.0/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /Users/aaron/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-0c7405d5-f333-46c5-868f-7b933490bce01690577752651 --workspace-folder /Users/aaron/dev/VS Code containers/micromamba-cruft --workspace-mount-consistency cached --id-label devcontainer.local_folder=/Users/aaron/dev/VS Code containers/micromamba-cruft --id-label devcontainer.config_file=/Users/aaron/dev/VS Code containers/micromamba-cruft/.devcontainer/devcontainer.json --log-level debug --log-format json --config /Users/aaron/dev/VS Code containers/micromamba-cruft/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root true
[2023-07-28T20:55:53.871Z] @devcontainers/cli 0.46.0. Node.js v16.17.1. darwin 22.6.0 arm64.
[2023-07-28T20:55:53.871Z] Start: Run: docker buildx version
[2023-07-28T20:55:53.968Z] Stop (97 ms): Run: docker buildx version
[2023-07-28T20:55:53.969Z] github.com/docker/buildx v0.9.1 ed00243a0ce2a0aee75311b06e32d33b44729689
[2023-07-28T20:55:53.969Z] 
[2023-07-28T20:55:53.969Z] Start: Resolving Remote
[2023-07-28T20:55:53.971Z] Start: Run: git rev-parse --show-cdup
[2023-07-28T20:55:53.983Z] Stop (12 ms): Run: git rev-parse --show-cdup
[2023-07-28T20:55:53.984Z] Start: Run: docker ps -q -a --filter label=devcontainer.local_folder=/Users/aaron/dev/VS Code containers/micromamba-cruft --filter label=devcontainer.config_file=/Users/aaron/dev/VS Code containers/micromamba-cruft/.devcontainer/devcontainer.json
[2023-07-28T20:55:54.044Z] Stop (60 ms): Run: docker ps -q -a --filter label=devcontainer.local_folder=/Users/aaron/dev/VS Code containers/micromamba-cruft --filter label=devcontainer.config_file=/Users/aaron/dev/VS Code containers/micromamba-cruft/.devcontainer/devcontainer.json
[2023-07-28T20:55:54.046Z] Start: Run: docker inspect --type image ghcr.io/mamba-org/micromamba-devcontainer:git-d175103
[2023-07-28T20:55:54.106Z] Stop (60 ms): Run: docker inspect --type image ghcr.io/mamba-org/micromamba-devcontainer:git-d175103
[2023-07-28T20:55:54.107Z] local container features stored at: /Users/aaron/.vscode/extensions/ms-vscode-remote.remote-containers-0.299.0/dist/node_modules/vscode-dev-containers/container-features
[2023-07-28T20:55:54.108Z] Start: Run: tar --no-same-owner -x -f -
[2023-07-28T20:55:54.113Z] Stop (5 ms): Run: tar --no-same-owner -x -f -
[2023-07-28T20:55:54.114Z] Start: Run: docker buildx build --load --build-arg BUILDKIT_INLINE_CACHE=1 -f /var/folders/9s/6fvdmxlx00z9sz6ftkrnh8dw0000gn/T/devcontainercli/container-features/0.46.0-1690577754107/Dockerfile-with-features -t vsc-micromamba-cruft-ef59054f8288a67d215a74ab404fdbf5e0575dc83abfa6714adafb4392a8707a --target dev_containers_target_stage --build-arg CONTAINER_WORKSPACE_FOLDER=/workspaces/micromamba-cruft --build-arg _DEV_CONTAINERS_BASE_IMAGE=dev_container_auto_added_stage_label /Users/aaron/dev/VS Code containers/micromamba-cruft
[2023-07-28T20:55:54.369Z] [+] Building 0.0s (0/0)                                                         
[2023-07-28T20:55:54.506Z] [+] Building 0.0s (0/1)                                                         
[2023-07-28T20:55:54.598Z] [+] Building 0.1s (17/18)                                                       
 => [internal] load build definition from Dockerfile-with-features         0.0s
 => => transferring dockerfile: 4.43kB                                     0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
[2023-07-28T20:55:54.598Z]  => [internal] load metadata for ghcr.io/mamba-org/micromamba-devcontaine  0.0s
 => CANCELED https://github.com/hadolint/hadolint/releases/download/v2.10  0.0s
 => [internal] load build context                                          0.0s
 => => transferring context: 2B                                            0.0s
 => CACHED [dev_container_auto_added_stage_label  1/13] FROM ghcr.io/mamb  0.0s
 => CACHED [dev_container_auto_added_stage_label  2/13] ADD https://githu  0.0s
 => CACHED [dev_container_auto_added_stage_label  3/13] RUN sudo chmod a+  0.0s
 => ERROR [dev_container_auto_added_stage_label  4/13] COPY --chown=mamba  0.0s
 => CACHED [dev_container_auto_added_stage_label  5/13] RUN :     && micr  0.0s
 => CACHED [dev_container_auto_added_stage_label  6/13] RUN :   && pipx i  0.0s
 => CACHED [dev_container_auto_added_stage_label  7/13] RUN mkdir -p "/wo  0.0s
 => CACHED [dev_container_aut
[2023-07-28T20:55:54.599Z] o_added_stage_label  8/13] WORKDIR /workspac  0.0s
 => ERROR [dev_container_auto_added_stage_label  9/13] COPY --chown=mamba  0.0s
 => CACHED [dev_container_auto_added_stage_label 10/13] RUN :     && mkdi  0.0s
 => CACHED [dev_container_auto_added_stage_label 11/13] RUN pip install -  0.0s
 => ERROR [dev_container_auto_added_stage_label 12/13] COPY --chown=mamba  0.0s
------
 > [dev_container_auto_added_stage_label  4/13] COPY --chown=mambauser:mambauser conda-lock.yml /tmp/conda-lock.yml:
------
------
 > [dev_container_auto_added_stage_label  9/13] COPY --chown=mambauser:mambauser pyproject.toml ./:
------
------
 > [dev_container_auto_added_stage_label 12/13] COPY --chown=mambauser:mambauser     ./cruft-test/__init__.py     ./cruft-test/:
------
ERROR: failed to solve: failed to compute cache key: "/cruft-test/__init__.py" not found: not found
[2023-07-28T20:55:54.605Z] Stop (491 ms): Run: docker buildx build --load --build-arg BUILDKIT_INLINE_CACHE=1 -f /var/folders/9s/6fvdmxlx00z9sz6ftkrnh8dw0000gn/T/devcontainercli/container-features/0.46.0-1690577754107/Dockerfile-with-features -t vsc-micromamba-cruft-ef59054f8288a67d215a74ab404fdbf5e0575dc83abfa6714adafb4392a8707a --target dev_containers_target_stage --build-arg CONTAINER_WORKSPACE_FOLDER=/workspaces/micromamba-cruft --build-arg _DEV_CONTAINERS_BASE_IMAGE=dev_container_auto_added_stage_label /Users/aaron/dev/VS Code containers/micromamba-cruft
[2023-07-28T20:55:54.607Z] Error: Command failed: docker buildx build --load --build-arg BUILDKIT_INLINE_CACHE=1 -f /var/folders/9s/6fvdmxlx00z9sz6ftkrnh8dw0000gn/T/devcontainercli/container-features/0.46.0-1690577754107/Dockerfile-with-features -t vsc-micromamba-cruft-ef59054f8288a67d215a74ab404fdbf5e0575dc83abfa6714adafb4392a8707a --target dev_containers_target_stage --build-arg CONTAINER_WORKSPACE_FOLDER=/workspaces/micromamba-cruft --build-arg _DEV_CONTAINERS_BASE_IMAGE=dev_container_auto_added_stage_label /Users/aaron/dev/VS Code containers/micromamba-cruft
[2023-07-28T20:55:54.607Z]     at Ute (/Users/aaron/.vscode/extensions/ms-vscode-remote.remote-containers-0.299.0/dist/spec-node/devContainersSpecCLI.js:2008:1698)
[2023-07-28T20:55:54.607Z]     at process.processTicksAndRejections (node:internal/process/task_queues:96:5)
[2023-07-28T20:55:54.607Z]     at async fC (/Users/aaron/.vscode/extensions/ms-vscode-remote.remote-containers-0.299.0/dist/spec-node/devContainersSpecCLI.js:2007:1691)
[2023-07-28T20:55:54.607Z]     at async n3 (/Users/aaron/.vscode/extensions/ms-vscode-remote.remote-containers-0.299.0/dist/spec-node/devContainersSpecCLI.js:2007:610)
[2023-07-28T20:55:54.607Z]     at async rre (/Users/aaron/.vscode/extensions/ms-vscode-remote.remote-containers-0.299.0/dist/spec-node/devContainersSpecCLI.js:2024:3660)
[2023-07-28T20:55:54.607Z]     at async Yf (/Users/aaron/.vscode/extensions/ms-vscode-remote.remote-containers-0.299.0/dist/spec-node/devContainersSpecCLI.js:2024:4775)
[2023-07-28T20:55:54.608Z]     at async Dne (/Users/aaron/.vscode/extensions/ms-vscode-remote.remote-containers-0.299.0/dist/spec-node/devContainersSpecCLI.js:2156:12193)
[2023-07-28T20:55:54.608Z]     at async Sne (/Users/aaron/.vscode/extensions/ms-vscode-remote.remote-containers-0.299.0/dist/spec-node/devContainersSpecCLI.js:2156:11934)
[2023-07-28T20:55:54.609Z] Stop (910 ms): Run: /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) --ms-enable-electron-run-as-node /Users/aaron/.vscode/extensions/ms-vscode-remote.remote-containers-0.299.0/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /Users/aaron/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-0c7405d5-f333-46c5-868f-7b933490bce01690577752651 --workspace-folder /Users/aaron/dev/VS Code containers/micromamba-cruft --workspace-mount-consistency cached --id-label devcontainer.local_folder=/Users/aaron/dev/VS Code containers/micromamba-cruft --id-label devcontainer.config_file=/Users/aaron/dev/VS Code containers/micromamba-cruft/.devcontainer/devcontainer.json --log-level debug --log-format json --config /Users/aaron/dev/VS Code containers/micromamba-cruft/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root true
[2023-07-28T20:55:54.610Z] Exit code 1
[2023-07-28T20:55:54.612Z] Command failed: /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) --ms-enable-electron-run-as-node /Users/aaron/.vscode/extensions/ms-vscode-remote.remote-containers-0.299.0/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /Users/aaron/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-0c7405d5-f333-46c5-868f-7b933490bce01690577752651 --workspace-folder /Users/aaron/dev/VS Code containers/micromamba-cruft --workspace-mount-consistency cached --id-label devcontainer.local_folder=/Users/aaron/dev/VS Code containers/micromamba-cruft --id-label devcontainer.config_file=/Users/aaron/dev/VS Code containers/micromamba-cruft/.devcontainer/devcontainer.json --log-level debug --log-format json --config /Users/aaron/dev/VS Code containers/micromamba-cruft/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root true
[2023-07-28T20:55:54.612Z] Exit code 1
maresb commented 1 year ago

Hey, thanks for your interest, and sorry this isn't better documented.

It's expected that your package directory contains an __init__.py file. (This traditionally defines at least the package version number as __version__.)

Could you please try creating this and see if it works for you then?

I should document this requirement explicitly.

aaronjnewman commented 1 year ago

thanks - I needed to create a subfolder named the same as I named the project when running cruft, then put __init__.py in there. But it also required me to create pyproject.toml and conda-lock.yml files. I tried creating empy files for each of these but it sill fails at:

ERROR: failed to solve: process "/usr/local/bin/_dockerfile_shell.sh : && micromamba config append channels conda-forge && micromamba install --yes --name base --category dev --category main --file /tmp/conda-lock.yml && micromamba clean --all --yes ;" did not complete successfully: exit code: 1

I was actually trying to co-opt this for a different purpose (installing Python packages to run Jupyter notebooks) and it seems like not the best choice as a starting template, so this may simply reflect my naïveté/user error. But if it helps inform the documentation I hope it wasn't too much trouble!

dephora commented 2 months ago

The steps below are the minimum changes required to build after creating a new project with Hatch and the Cruft template. This was tested using VSCode and DevPod. It's possible this is not "proper", but it does work.

I started with https://gitlab.com/bmares/cookiecutter-micromamba-devcontainer#setup.

Basic Setup

  1. hatch new "sandbox"
  2. cd sandbox
  3. cruft create https://github.com/maresb/cookiecutter-micromamba-devcontainer [1/3] package_name (the_name_of_your_python_package): sandbox [2/3] timezone (Europe/Berlin): [3/3] packages_dir (.): python

Post Setup Changes

Below are required changes, otherwise the build fails.

  1. mv - anything related to src is not required if src is used as the packages_dir as this is what hatch new defaults to. I prefer python. (the example uses this as well)
  2. cp - the version is in the __about__.py, copy that to __init__.py
  3. sed - change the about to init and comment out the readme line.
  4. sed - same as 1, only required if not using src
mv src python && \
cp python/sandbox/__about__.py python/sandbox/__init__.py && \
sed -i.bak 's/__about__.py/__init__.py/; s/^readme/# &/' pyproject.toml && \
sed -i.bak 's|src/|python/|g' pyproject.toml

Conda

  1. Copy micromamba-devcontainer-example-main/conda-environment.yaml to project root
  2. conda-lock --file conda-environment.yaml --file .devcontainer/dev-conda-environment.yaml
    • Can be done with the relock-deps but in its current state it expects a conda-lock.yml file. Must change paths too.

Everything Else

Ref the docs for any other setup / copy remaining files from the example.

maresb commented 2 months ago

Thanks so much @dephora! What you write looks correct, thanks so much for codifying everything!

My latest advice would be to use pixi as the frontend (replacement for Hatch) but to keep hatchling as the Python build-backend. It has integrated lockfile management, is super-fast, and is under very active development. For Dockerization there's pixi-docker.

I'm fairly new to using pixi, and the only difficulty I've found is needing to separately declare Conda packages to cover the pip packages I'm using. (Otherwise things default to pip installation.) I'm not sure yet if there's an easy solution for that, but in worst case I could probably spin off a few helper tools from conda-lock.

I haven't yet gotten around to templating a dev environment. I've been meaning to ask the pixi devs some questions first to make sure I'm using everything correctly.

dephora commented 2 months ago

@maresb - No problem! Thank you for creating this to begin with. Funny you mention pixi, I was looking at that recently before finding your devcontainer. I will keep an eye out for your new dev env.