Open DeNeutoy opened 10 months ago
We don't support this, as cog requires a lot of assumptions about running on ubuntu. However, as of 0.9.0-beta5, you can write the entire dockerfile yourself and use --dockerfile
to pass it in. cog debug
can be helpful for this. This is not officially supported; you're on your own for any errors. You should have cog.yaml in the workdir, have python3 -m cog.server.http
and python3 -m cog.command.openapi_schema
working, and have pip install cog==<version>
be a valid command.
@technillogue hello, i'm on cog version 0.9.0-beta1
. and I can't get --dockerfile
working.
Validating model schema...
ⅹ Model schema is invalid: value of openapi must be a non-empty string
ⅹ
ⅹ {}
running python3 -m cog.command.openapi_schema
does output the schema.. did I miss anything? Thanks!
It seems like this will work:
cog push r8.im/ynie/alteraspect --dockerfile Dockerfile.replicate --openapi-schema openapi_schema
but it will stuck at starting state.
can you send me your dockerfile? it needs to match the output of cog debug
fairly closely. pip install -U cog
and python3 -m cog.server.http
both have to work for your model to start. it's also worth checking if your container will boot at all on your own machine, or if you have some error
also, currently please use cog 0.9.0-beta10
@technillogue Ok, I was on cog version 0.9.0-beta11 (built 2023-11-28T17:38:26Z). Maybe that's the issue?
#syntax=docker/dockerfile:1.4
FROM nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04
ENV DEBIAN_FRONTEND=noninteractive
ENV PYTHONUNBUFFERED=1
ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/x86_64-linux-gnu:/usr/local/nvidia/lib64:/usr/local/nvidia/bin
RUN --mount=type=cache,target=/var/cache/apt set -eux; \
apt-get update -qq; \
apt-get install -qqy --no-install-recommends curl; \
rm -rf /var/lib/apt/lists/*; \
TINI_VERSION=v0.19.0; \
TINI_ARCH="$(dpkg --print-architecture)"; \
curl -sSL -o /sbin/tini "https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini-${TINI_ARCH}"; \
chmod +x /sbin/tini
ENTRYPOINT ["/sbin/tini", "--"]
ENV PATH="/root/.pyenv/shims:/root/.pyenv/bin:$PATH"
RUN --mount=type=cache,target=/var/cache/apt apt-get update -qq && apt-get install -qqy --no-install-recommends \
make \
build-essential \
libssl-dev \
zlib1g-dev \
libbz2-dev \
libreadline-dev \
libsqlite3-dev \
wget \
curl \
llvm \
libncurses5-dev \
libncursesw5-dev \
xz-utils \
tk-dev \
libffi-dev \
liblzma-dev \
git \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*
RUN curl -s -S -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash && \
git clone https://github.com/momo-lab/pyenv-install-latest.git "$(pyenv root)"/plugins/pyenv-install-latest && \
pyenv install-latest "3.11" && \
pyenv global $(pyenv install-latest --print "3.11") && \
pip install "wheel<1"
COPY replicate_server/cog-0.0.1.dev-py3-none-any.whl /tmp/cog-0.0.1.dev-py3-none-any.whl
RUN --mount=type=cache,target=/root/.cache/pip pip install /tmp/cog-0.0.1.dev-py3-none-any.whl
RUN --mount=type=cache,target=/var/cache/apt apt-get update -qq && apt-get install -qqy libgl1-mesa-glx && rm -rf /var/lib/apt/lists/*
COPY requirements.replicate.txt /tmp/requirements.txt
RUN --mount=type=cache,target=/root/.cache/pip pip install -r /tmp/requirements.txt
WORKDIR /src
COPY checkpoints /src/checkpoints
EXPOSE 5000
CMD ["python", "-m", "cog.server.http"]
COPY [^checkpoints]. /src
Hello!
I know that cog is aimed at research projects/researchers not super familiar with docker. However, I am investigating deploying models on replicate for the company I work for (i.e private models), and we already have a fully containerized workflow, which works with GPUs. It would be great if I could specify a parent image in the build section of
cog.yaml
.Perhaps this already works, and it is only the api specification of having a
Predictor
and associated server entrypoint in a given docker image that makes it "replicate" compatible, although I somehow doubt this is the case.My ideal workflow would be:
Perhaps this is hard! Let me know if you have any suggestions for an ideal approach here. Thanks!