Docker support

[hazre]

Title says it all, but basically docker is pretty much the standard of deploying things on servers, so it would be nice if Crystite had docker support.

[Nihlus]

I've been looking into it, and I'll probably start publishing a dockerfile in the repo soon. I don't use Docker myself, so I could use some input on what users would expect from Docker support.

[Merith-TK]

I am looking into it for an docker file, this is what I have so far however, its getting caught up as crystite is not a valid package

FROM debian:11

RUN apt update && apt install -y wget

RUN echo 'deb [signed-by=/usr/share/keyrings/algiz.gpg] https://repo.algiz.nu/crystite bullseye main' | tee /etc/apt/sources.list.d/crystite.list \
    && mkdir -p /usr/share/keyrings \
    && wget -O /usr/share/keyrings/algiz.gpg https://repo.algiz.nu/algiz.gpg \
    && apt install -y crystite

VOLUME /etc/crystite
VOLUME /var/lib/Resonite

ENV USERNAME ""
ENV PASSWORD ""
ENV BETA_KEY ""

## WIP, need base packages installed first
# ENTRYPOINT 

[uyjulian]

This is what I have right now

docker run --name crystite-test -dit --network=host \
    -v /mnt/data/crystite/var/lib/crystite:/var/lib/crystite:rw \
    -v /mnt/data/crystite/etc/crystite:/etc/crystite:rw \
    -v /mnt/data/crystite/data:/mnt/data:rw \
    public.ecr.aws/docker/library/debian:11 /bin/bash -c 'trap : TERM INT; coproc { exec >&-; read; }; eval exec "${COPROC[0]}<&-"; wait'
docker exec -it crystite-test /bin/bash
apt-get update
apt-get install -y sudo wget screen
wget https://packages.microsoft.com/config/debian/11/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
dpkg -i packages-microsoft-prod.deb
rm packages-microsoft-prod.deb
echo 'deb [signed-by=/usr/share/keyrings/algiz.gpg] https://repo.algiz.nu/crystite bullseye main' | tee /etc/apt/sources.list.d/crystite.list
mkdir -p /usr/share/keyrings
wget -O /usr/share/keyrings/algiz.gpg https://repo.algiz.nu/algiz.gpg
apt-get update
apt-get -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install -y crystite
exit
docker kill crystite-test
docker rm crystite-test

[hazre]

We shouldn't use Debian packages for docker. It should build from source. I will try to make a docker file myself

[Merith-TK]

it should NOT build from source, as an LONG TIME (4+ years) Docker user, building from source for a docker file is a pain in the ass, as you then have to factor in individuals physical hardware performance, network limitations (download speed of build chains) and storage.

An maybe 600mb-1gb container, should NOT require 3gb+ of dependencies that are present at all times just to build the container/software for each update.

[Merith-TK]

This is what I have right now

docker run --name crystite-test -dit --network=host \
  -v /mnt/data/crystite/var/lib/crystite:/var/lib/crystite:rw \
  -v /mnt/data/crystite/etc/crystite:/etc/crystite:rw \
  -v /mnt/data/crystite/data:/mnt/data:rw \
  public.ecr.aws/docker/library/debian:11 /bin/bash -c 'trap : TERM INT; coproc { exec >&-; read; }; eval exec "${COPROC[0]}<&-"; wait'
docker exec -it crystite-test /bin/bash
apt-get update
apt-get install -y sudo wget screen
wget https://packages.microsoft.com/config/debian/11/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
dpkg -i packages-microsoft-prod.deb
rm packages-microsoft-prod.deb
echo 'deb [signed-by=/usr/share/keyrings/algiz.gpg] https://repo.algiz.nu/crystite bullseye main' | tee /etc/apt/sources.list.d/crystite.list
mkdir -p /usr/share/keyrings
wget -O /usr/share/keyrings/algiz.gpg https://repo.algiz.nu/algiz.gpg
apt-get update
apt-get -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install -y crystite
exit
docker kill crystite-test
docker rm crystite-test

I just converted all of that into an dockerfile and... it builds, I just need to know how to launch the software to make it all work.

# no clue why using this, but it works!
FROM public.ecr.aws/docker/library/debian:11

# Create "persistant" storage containers for container lifetime
VOLUME /var/lib/crystite
VOLUME /etc/crystite
VOLUME /mnt/data

# The entire "wait" line is not nessecary

# Install dependencies manually rather than rely on package manager
RUN apt-get update && \
    apt-get install -y sudo wget screen && \
    wget https://packages.microsoft.com/config/debian/11/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && \
    dpkg -i packages-microsoft-prod.deb && \
    rm packages-microsoft-prod.deb

## Install Crystite
RUN echo 'deb [signed-by=/usr/share/keyrings/algiz.gpg] https://repo.algiz.nu/crystite bullseye main' | tee /etc/apt/sources.list.d/crystite.list && \
    mkdir -p /usr/share/keyrings && \
    wget -O /usr/share/keyrings/algiz.gpg https://repo.algiz.nu/algiz.gpg
RUN apt-get update && \
    apt-get -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install -y crystite

ENV STEAM_USER ""
ENV STEAM_PASS ""
ENV BETA_CODE  ""

# COPY entrypoint.sh /entrypoint.sh
## Entrypoint.sh is just an script that tells the container how to start up, 
# bash script to allow for easy pre-setup
# ENTRYPOINT ["/bin/bash", "/entrypoint.sh"]

[Nihlus]

I should probably chip in here too, now :P This is the dockerfile I've been using when experimenting. I would also strongly argue against building from source in the container, since only tagged and packaged releases have any semblance of workability guarantees.

FROM debian:11-slim
LABEL org.opencontainers.image.authors=jarl.gullberg@gmail.com
LABEL org.opencontainers.image.url=https://github.com/Nihlus/Crystite
LABEL org.opencontainers.image.vendor="Jarl Gullberg"
LABEL org.opencontainers.image.title=Crystite
LABEL org.opencontainers.image.description="Headless server for the VR sandbox Resonite"

RUN apt-get update

# Configure .NET 
ADD https://packages.microsoft.com/config/debian/11/packages-microsoft-prod.deb /tmp/packages-microsoft-prod.deb
RUN apt-get install -y /tmp/packages-microsoft-prod.deb
RUN rm /tmp/packages-microsoft-prod.deb

# Configure package sources
RUN echo 'deb [signed-by=/usr/share/keyrings/algiz.gpg] https://repo.algiz.nu/crystite bullseye main' > /etc/apt/sources.list.d/crystite.list
RUN mkdir -p /usr/share/keyrings
ADD https://repo.algiz.nu/algiz.gpg /usr/share/keyrings/algiz.gpg
RUN chmod +r /usr/share/keyrings/algiz.gpg

# Install Crystite
RUN apt-get update
RUN apt-get install -y crystite

RUN addgroup --system --gid 500 crystite-secrets
RUN usermod -aG crystite-secrets crystite

RUN --mount=type=secret,id=steam-credentials,target=/etc/crystite/conf.d/steam-credentials.json,required=true,mode=0440,gid=500
RUN --mount=type=secret,id=resonite-credentials,target=/etc/crystite/conf.d/resonite-credentials.json,mode=0440,gid=500
RUN --mount=type=secret,id=worlds,target=/etc/crystite/conf.d/worlds.json,mode=0440,gid=500

# Launch
USER crystite
WORKDIR /var/lib/crystite
ENTRYPOINT [ "/usr/bin/crystite" ]

This file uses docker secrets to configure the instance, but it's probably better to lift that out to something like docker-compose as for the stock headless (https://github.com/shadowpanther/resonite-headless).

[hazre]

it should NOT build from source, as an LONG TIME (4+ years) Docker user, building from source for a docker file is a pain in the ass, as you then have to factor in individuals physical hardware performance, network limitations (download speed of build chains) and storage.

An maybe 600mb-1gb container, should NOT require 3gb+ of dependencies that are present at all times just to build the container/software for each update.

That's why you do a two stage build. And the reason for building from source is to for example build on Alpine to ensure the smallest image size.

[Nihlus]

Some quick googling shows that the Alpine base image is ~5 MB and the slim Debian 11 image is ~30MB. A difference in roughly 25 MB is entirely negligible when the size of .NET, Crystite, and its dependencies easily surpasses hundreds of megabytes. On top of that, the software is only tested by me on Debian, and I don't want to take on more maintenance load at this time.

You are, of course, completely free to make an Alpine image yourself if you prefer that! However, any official docker image would be Debian-based and would use the already-built packages to lighten my own personal load.

[Merith-TK]

That's why you do a two stage build.

I would like to point out that my point still exists here, you are left with several gigs of toolchain data sitting around on your hard drive, and still have to worry about the hardware's actual capability to compile the fucking thing, The server I was running Remora on could never compile it due to the limited hard drive space, (it only had 50gb TOTAL at the time and had about 30 left after I had been using the thing for the longest time, and when we factor in how hard drive hungry the headless server was back in Neos at the time... yeah my thing kept litterally running out of storage)

So what makes you think that adding on the toolchains and compilation artifacts ontop of that would be a good idea?

The Headless server is hungry for disk space, because (I need to test this with resonite), it cached all assets just like the client did! (even though it was for the most part, entirely unessecary)

[hazre]

I should probably chip in here too, now :P This is the dockerfile I've been using when experimenting. I would also strongly argue against building from source in the container, since only tagged and packaged releases have any semblance of workability guarantees.

FROM debian:11-slim
LABEL org.opencontainers.image.authors=jarl.gullberg@gmail.com
LABEL org.opencontainers.image.url=https://github.com/Nihlus/Crystite
LABEL org.opencontainers.image.vendor="Jarl Gullberg"
LABEL org.opencontainers.image.title=Crystite
LABEL org.opencontainers.image.description="Headless server for the VR sandbox Resonite"

RUN apt-get update

# Configure .NET 
ADD https://packages.microsoft.com/config/debian/11/packages-microsoft-prod.deb /tmp/packages-microsoft-prod.deb
RUN apt-get install -y /tmp/packages-microsoft-prod.deb
RUN rm /tmp/packages-microsoft-prod.deb

# Configure package sources
RUN echo 'deb [signed-by=/usr/share/keyrings/algiz.gpg] https://repo.algiz.nu/crystite bullseye main' > /etc/apt/sources.list.d/crystite.list
RUN mkdir -p /usr/share/keyrings
ADD https://repo.algiz.nu/algiz.gpg /usr/share/keyrings/algiz.gpg
RUN chmod +r /usr/share/keyrings/algiz.gpg

# Install Crystite
RUN apt-get update
RUN apt-get install -y crystite

RUN addgroup --system --gid 500 crystite-secrets
RUN usermod -aG crystite-secrets crystite

RUN --mount=type=secret,id=steam-credentials,target=/etc/crystite/conf.d/steam-credentials.json,required=true,mode=0440,gid=500
RUN --mount=type=secret,id=resonite-credentials,target=/etc/crystite/conf.d/resonite-credentials.json,mode=0440,gid=500
RUN --mount=type=secret,id=worlds,target=/etc/crystite/conf.d/worlds.json,mode=0440,gid=500

# Launch
USER crystite
WORKDIR /var/lib/crystite
ENTRYPOINT [ "/usr/bin/crystite" ]

This file uses docker secrets to configure the instance, but it's probably better to lift that out to something like docker-compose as for the stock headless (https://github.com/shadowpanther/resonite-headless).

That's why you setup a github action that creates a image and publishes it only on new release.

[uyjulian]

Here is the dockerfile to build from source

FROM public.ecr.aws/docker/library/debian:11

ENV \
    LANG="en_US.UTF-8" \
    LC_ALL="en_US.UTF-8"
USER root
WORKDIR /root

RUN apt-get update
RUN apt-get install -y sudo wget git
RUN wget https://packages.microsoft.com/config/debian/11/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && \
    dpkg -i packages-microsoft-prod.deb && \
    rm packages-microsoft-prod.deb
RUN apt-get update
RUN apt-get install -y dotnet-sdk-7.0
RUN git clone https://github.com/Nihlus/Crystite.git
WORKDIR /root/Crystite
RUN dotnet publish -f net7.0 -c Release -r linux-x64 --self-contained false -o bin Crystite
RUN mkdir -p /usr/lib/crystite
RUN cp -Rp ./bin/* /usr/lib/crystite

# Second stage of Dockerfile
FROM public.ecr.aws/docker/library/debian:11

ENV \
    LANG="en_US.UTF-8" \
    LC_ALL="en_US.UTF-8"
USER root
WORKDIR /root

RUN mkdir -p /usr/lib/crystite
COPY --from=0 /root/Crystite/bin /usr/lib/crystite

RUN apt-get update
RUN apt-get install -y wget
RUN wget https://packages.microsoft.com/config/debian/11/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && \
    dpkg -i packages-microsoft-prod.deb && \
    rm packages-microsoft-prod.deb
RUN apt-get update
RUN apt-get install -y dotnet-runtime-7.0 aspnetcore-runtime-7.0 libassimp5 libfreeimage3 libfreetype6 libopus0 libbrotli1 zlib1g youtube-dl

WORKDIR /var/lib/crystite
CMD ["/usr/lib/crystite/crystite"]

Tested working

It is also possible to auto build and upload to GitHub container registry; see example here https://github.com/uyjulian/rhl_6_2_container/blob/ecbb148385384622f2b471152d39c17a63e506cc/.github/workflows/ci.yml

[Merith-TK]

It is also possible to auto build and upload to GitHub container registry; see example here https://github.com/uyjulian/rhl_6_2_container/blob/ecbb148385384622f2b471152d39c17a63e506cc/.github/workflows/ci.yml

doesnt seem to be wokring anymore

[user@rubbery-focus resonite]$ docker compose build > build.log
failed to solve: process "/bin/sh -c dotnet publish -f net7.0 -c Release -r linux-x64 --self-contained false -o bin Crystite" did not complete successfully: exit code: 1

build.log contains all build output that was not sent to the terminal

[Nihlus]

You need to explicitly specify the project to build, since crystitectl uses .NET 8, not .NET 7. Take a look here for proper build commands: https://github.com/Nihlus/Crystite/blob/main/debian/rules

[Nihlus]

Oh, I should also mention - Crystite now targets Debian 12, and I won't be releasing any more packages for Debian 11.

[uyjulian]

Instructions for Debian 12

docker run --name crystite-test -dit --network=host \
    -v /mnt/data/crystite/var/lib/crystite:/var/lib/crystite:rw \
    -v /mnt/data/crystite/etc/crystite:/etc/crystite:rw \
    -v /mnt/data/crystite/data:/mnt/data:rw \
    public.ecr.aws/docker/library/debian:12 /bin/bash -c 'trap : TERM INT; coproc { exec >&-; read; }; eval exec "${COPROC[0]}<&-"; wait'
docker exec -it crystite-test /bin/bash
apt-get update
apt-get install -y sudo wget screen
wget https://packages.microsoft.com/config/debian/12/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
dpkg -i packages-microsoft-prod.deb
rm packages-microsoft-prod.deb
echo 'deb [signed-by=/usr/share/keyrings/algiz.gpg] https://repo.algiz.nu/crystite bookworm main' | tee /etc/apt/sources.list.d/crystite.list
mkdir -p /usr/share/keyrings
wget -O /usr/share/keyrings/algiz.gpg https://repo.algiz.nu/algiz.gpg
apt-get update
apt-get -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install -y crystite
exit
docker kill crystite-test
docker rm crystite-test

[Nihlus]

You're still referring to the Debian 12 repo - update "bullseye" to "bookworm".

On Fri, 19 Jan 2024, 14:19 Julian Uy, @.***> wrote:

Instructions for Debian 12

docker run --name crystite-test -dit --network=host \ -v /mnt/data/crystite/var/lib/crystite:/var/lib/crystite:rw \ -v /mnt/data/crystite/etc/crystite:/etc/crystite:rw \ -v /mnt/data/crystite/data:/mnt/data:rw \ public.ecr.aws/docker/library/debian:12 /bin/bash -c 'trap : TERM INT; coproc { exec >&-; read; }; eval exec "${COPROC[0]}<&-"; wait' docker exec -it crystite-test /bin/bash apt-get update apt-get install -y sudo wget screen wget https://packages.microsoft.com/config/debian/12/packages-microsoft-prod.deb -O packages-microsoft-prod.deb dpkg -i packages-microsoft-prod.deb rm packages-microsoft-prod.debecho 'deb [signed-by=/usr/share/keyrings/algiz.gpg] https://repo.algiz.nu/crystite bullseye main' | tee /etc/apt/sources.list.d/crystite.list mkdir -p /usr/share/keyrings wget -O /usr/share/keyrings/algiz.gpg https://repo.algiz.nu/algiz.gpg apt-get https://repo.algiz.nu/algiz.gpgapt-get update apt-get -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install -y crystiteexit docker kill crystite-test docker rm crystite-test

— Reply to this email directly, view it on GitHub https://github.com/Nihlus/Crystite/issues/2#issuecomment-1900414885, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAR3MTGNSIQS2RIV2OJRP2TYPJXGLAVCNFSM6AAAAAA6K3BLW6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBQGQYTIOBYGU . You are receiving this because you commented.Message ID: @.***>

[Merith-TK]

Applied those fixes but...

[user@rubbery-focus resonite]$ docker logs crystite-headless
Unhandled exception. System.InvalidOperationException: Operation is not valid due to the current state of the object.
   at Program.<>c__DisplayClass0_1.<<Main>$>b__9()
   at Remora.Extensions.Options.Immutable.CreateOptions`1.Create()
   at Remora.Extensions.Options.Immutable.ReadOnlyOptionsFactory`1.Create(String name)
   at Microsoft.Extensions.Options.UnnamedOptionsManager`1.get_Value()
   at Crystite.ResoniteInstallation.ResoniteSteamClient..ctor(SteamClient steamClient, ILogger`1 log, IOptions`1 config)
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.ConstructorInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
   at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(ServiceIdentifier serviceIdentifier)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Program.<Main>$(String[] args)
   at Program.<Main>$(String[] args)
   at Program.<Main>(String[] args)

FROM public.ecr.aws/docker/library/debian:12

ENV \
    LANG="en_US.UTF-8" \
    LC_ALL="en_US.UTF-8" \
    USERID="1001"
USER root
WORKDIR /root

RUN apt-get update
RUN apt-get install -y wget screen
RUN wget https://packages.microsoft.com/config/debian/11/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && \
    dpkg -i packages-microsoft-prod.deb && \
    rm packages-microsoft-prod.deb
RUN echo 'deb [signed-by=/usr/share/keyrings/algiz.gpg] https://repo.algiz.nu/crystite bullseye main' | tee /etc/apt/sources.list.d/crystite.list
RUN mkdir -p /usr/share/keyrings && \
    wget -O /usr/share/keyrings/algiz.gpg https://repo.algiz.nu/algiz.gpg
RUN apt-get update
# RUN apt-get install -y dotnet-runtime-7.0 aspnetcore-runtime-7.0 libassimp5 libfreeimage3 libfreetype6 libopus0 libbrotli1 zlib1g youtube-dl
RUN apt-get -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install -y crystite
## Create user
RUN useradd -u ${USERID} resonite

# RUN chown ${USERID}:${USERID} /usr/lib/crystite -R
# RUN chown ${USERID}:${USERID} /etc/crystite -R

VOLUME [ "/etc/crystite" ]

WORKDIR /var/lib/crystite
COPY ./entrypoint.sh /entrypoint
# USER resonite
CMD ["/usr/lib/crystite/crystite"]

At first I thought it was because of running as an unprivileged user (like all services should be capable of doing in my opinion to minimize damages caused by potential RCE exploits), hence the lins referencing setting the user were commented out

[Merith-TK]

I am working to put together a template from the two configs I currently have for docker compose. Im just merging them together in a way that will support cyrstite and resonite headless

[Merith-TK]

https://github.com/Merith-TK/resonite-docker-compose need to flesh out documentation BUUUTTT

it works-ish!

if crystite has any launch arguments that lets me force feed it data and cache directories without a custom config file that would be great

[djsime1]

I've whipped up a container image and compose file solely for running Crystite if anyone else would like to give it a try. I've tested it with Docker and Podman, and everything appears to be working.