Update dev docs on how to run via docker

nkaretnikov commented 8 months ago

Context

Because RELEASE_VERSION was added in https://github.com/conda-incubator/conda-store/pull/621, the following command from docs/contributing.md no longer works:

docker-compose up --build

Note: there are multiple places in the file where this command is mentioned - update all of them.

Instead, this is what seems to work locally:

docker-compose build --build-arg RELEASE_VERSION=2023.10.1 && docker-compose up

I don't think RELEASE_VERSION matters much because later in Dockerfile it just installs from the current dir:

COPY ./ /opt/conda-store-server/

...

ARG RELEASE_VERSION
RUN cd /opt/conda-store-server && \
    /opt/conda/envs/conda-store-server/bin/pip install conda-store-server==${RELEASE_VERSION}

...

RUN cd /opt/conda-store-server && \
    /opt/conda/envs/conda-store-server/bin/pip install -e .

Note that this works fine on GitHub as is because there's an additional step that sets RELEASE_VERSION:

            RELEASE_VERSION=${{github.ref_name}}

Value and/or benefit

Instructions are valid and tell how to start the server via docker for local dev.

Anything else?

No response

trallard commented 8 months ago

@dcmcand I believe this was recently addressed - though it might have been on the conda-stor-ui side of things only. Could you have a quick look at whether the docs (user/developer) need updating for this case too?

nkaretnikov commented 8 months ago

Note: I did check the doc file when I reported this, so it's just necessary to make the suggested change.

UPD: The docs were recently moved, please check all files under docusaurus-docs for use of docker-compose

dcmcand commented 8 months ago

@nkaretnikov I can't reproduce this. From the main branch running docker compose up --build works fine. The RELEASE_VERSION arg is only on the prod target, so should only affect you if you are targeting prod, which the docker-compose.yaml file doesn't.

nkaretnikov commented 8 months ago

@dcmcand I cannot double-check right now, but before I forget: could it be due to a different version of docker compose being used? Note how the docs say "docker-compose", with a dash.

On my machine, these are two different tools:

$ docker-compose --version
docker-compose version 1.29.2, build unknown
$ docker compose --version
Docker version 24.0.5, build 24.0.5-0ubuntu1~22.04.1

The one with the dash is installed via conda.

dcmcand commented 8 months ago

I actually tried both to be sure and they both worked for me.

On Fri, Dec 22, 2023, 5:02 PM Nikita Karetnikov @.***> wrote:

@dcmcand https://github.com/dcmcand I cannot double-check right now, but before I forget: could it be due to a different version of docker compose being used? Note how the docs say "docker-compose", with a dash.

On my machine, these are two different tools:

$ docker-compose --version docker-compose version 1.29.2, build unknown $ docker compose --version Docker version 24.0.5, build 24.0.5-0ubuntu1~22.04.1

— Reply to this email directly, view it on GitHub https://github.com/conda-incubator/conda-store/issues/692#issuecomment-1868096307, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPVTRZP4MMCRW3DVFYQQF3YKX7O5AVCNFSM6AAAAABAO32D6OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRYGA4TMMZQG4 . You are receiving this because you were mentioned.Message ID: @.***>

nkaretnikov commented 8 months ago

@dcmcand Could you post the version numbers of both tools on your machine? I'll compare to what I have locally.

dcmcand commented 8 months ago

@nkaretnikov

docker compose version v2.21.0 docker-compose version 1.29.2

nkaretnikov commented 8 months ago

@dcmcand I've just tried it again in a new env, still fails. Full log attached. Any idea what might be wrong here?

(test-env) $ docker-compose up --build
Building conda-store-server
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
            Install the buildx component to build images with BuildKit:
            https://docs.docker.com/go/buildx/

Sending build context to Docker daemon  14.81MB
Step 1/20 : FROM condaforge/mambaforge:23.3.1-1 as base
 ---> 8b412704c49b
Step 2/20 : LABEL org.opencontainers.image.authors="conda-store development team"
 ---> Using cache
 ---> 4b7597ffb949
Step 3/20 : ENV PATH=/opt/conda/condabin:/opt/conda/envs/conda-store-server/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:${PATH}
 ---> Using cache
 ---> d7fb43b8d7b2
Step 4/20 : ENV TZ=Etc/UTC
 ---> Using cache
 ---> 0b5813454146
Step 5/20 : RUN apt-get update &&     apt-get install -yq --no-install-recommends     libgl1-mesa-glx     libegl1-mesa     libxrandr2     libxss1     libxcursor1     libxcomposite1     libasound2     libxi6     libxtst6     curl &&     apt-get clean &&     rm -rf /var/cache/apt/* &&    rm -rf /var/lib/apt/lists/* &&    rm -rf /tmp/*
 ---> Using cache
 ---> e1ae870f0951
Step 6/20 : RUN printf 'channels: []\n' > /opt/conda/.condarc &&     chown -R 1000:1000 /opt/conda &&     mkdir -p /opt/conda-store/conda-store &&     chown 1000:1000 /opt/conda-store/conda-store &&     mkdir -p /var/lib/conda-store &&     chown 1000:1000 /var/lib/conda-store &&     mkdir -p /opt/conda-store/envs &&     chown 1000:1000 /opt/conda-store/envs &&     mkdir /.cache &&     chown 1000:1000 /.cache
 ---> Using cache
 ---> 6b970c5a2751
Step 7/20 : USER 1000:1000
 ---> Using cache
 ---> 821015319605
Step 8/20 : COPY environment.yaml /opt/conda-store-server/environment.yaml
 ---> Using cache
 ---> 51ecbd2d210a
Step 9/20 : RUN mamba env create -f /opt/conda-store-server/environment.yaml &&         conda clean  --force-pkgs-dirs
 ---> Using cache
 ---> bd8eacfbe516
Step 10/20 : COPY ./ /opt/conda-store-server/
 ---> d6438369168f
Step 11/20 : USER 0:0
 ---> Running in a304628f01c2
Removing intermediate container a304628f01c2
 ---> 5144386905c6
Step 12/20 : RUN chown -R 1000:1000 /opt/conda-store-server/
 ---> Running in 26327355ffc9
Removing intermediate container 26327355ffc9
 ---> 19b8beef3e44
Step 13/20 : USER 1000:1000
 ---> Running in 296ccd96b80c
Removing intermediate container 296ccd96b80c
 ---> 6a6a4176982f
Step 14/20 : FROM base as prod
 ---> 6a6a4176982f
Step 15/20 : ARG RELEASE_VERSION
 ---> Running in 84940857405f
Removing intermediate container 84940857405f
 ---> bba59fa0fc35
Step 16/20 : RUN cd /opt/conda-store-server &&     /opt/conda/envs/conda-store-server/bin/pip install conda-store-server==${RELEASE_VERSION} &&     rm -rf /opt/conda-store-server/tests
 ---> Running in e4bb205c3457
ERROR: Could not find a version that satisfies the requirement conda-store-server== (from versions: 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.2.5, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5, 0.3.6, 0.3.7, 0.3.8, 0.3.9, 0.3.10, 0.3.11, 0.3.12, 0.3.13, 0.3.14, 0.3.15, 0.4.2, 0.4.3, 0.4.4, 0.4.5, 0.4.6, 0.4.7, 0.4.8, 0.4.9, 0.4.10, 0.4.11, 0.4.12, 0.4.13, 0.4.14, 0.4.15, 2023.9.1, 2023.9.2, 2023.10.1)
ERROR: No matching distribution found for conda-store-server==
The command '/bin/sh -c cd /opt/conda-store-server &&     /opt/conda/envs/conda-store-server/bin/pip install conda-store-server==${RELEASE_VERSION} &&     rm -rf /opt/conda-store-server/tests' returned a non-zero code: 1
ERROR: Service 'conda-store-server' failed to build : Build failed

Versions:

(test-env) $ which docker-compose
/home/test/miniconda3/envs/test-env/bin/docker-compose
(test-env) $ docker-compose --version
docker-compose version 1.29.2, build unknown

nkaretnikov commented 8 months ago

Nevermind, after I posted the above, I saw:

DEPRECATED: The legacy builder is deprecated and will be removed in a future release. Install the buildx component to build images with BuildKit: https://docs.docker.com/go/buildx/

So I went ahead and re-installed Docker Desktop and Docker Engine as specified in: https://docs.docker.com/desktop/install/ubuntu/ https://docs.docker.com/engine/install/ubuntu/

Now docker-compose up --build works.

conda-incubator / conda-store