dockerize it

[nicholasamorim]

Hi, thank you so much for this project.

I'm trying to dockerize this application for our use-case (multiple servers in a swarm) and for publishing on Dockerhub but I'm running into a couple problems.

It's funny because I dockerized civetweb as a play project and it worked fine.

There are 2 issues:

1) Log output is erratic apparently due to buffering issues. In Docker we don't daemon the process but rather let it run on the foreground and output to stdout. However only in Docker the log output is very erratic. If I make a request, the log is cut halfway and I have to make many more requests for the rest of the logs to come up and is buffered halfway again. Is there a variable or a setting to reduce or tune buffering?

2) This is most likely related to Docker and not to uts-server (although it doesn't happen with civetweb in docker): once docker is up and the service is running I cannot ctrl-c it, I can only stop it by killing the container. If I log into the container using bash and run the server on my own, it works just fine - both log output and killing it with ctrl-c.

Could you shed some light if possible?

[kakwa]

for 1) it should be fixable, but it will require some modifications. Right now, logging to stdout is meant for debugging only, not really for a production setup, but I could easily add a parameter in the configuration file to set the type of logging (syslog or stdout).

For the erratic log output, it's actually more of a bug independent of the dockerization, I should disable stdout buffering.

The function needing modification is declared here: https://github.com/kakwa/uts-server/blob/master/src/lib/utils.c#L146

and it requires also some modification on the configuration file parsing and in the context struct:

https://github.com/kakwa/uts-server/blob/master/inc/context.h#L30 https://github.com/kakwa/uts-server/blob/master/src/lib/utils.c#L280

For the second one, I'm not sure what is happening, I will have to test.

Do you have a DockerFile you can share?

[nicholasamorim]

Yes! Was going to share on the original post but forgot. I'm bundling civetweb for a quick test.

FROM debian:stretch-slim

RUN groupadd -r uts && useradd -r -g uts uts
RUN mkdir /etc/uts-server && chown uts:uts /etc/uts-server

RUN apt-get update; \
    apt-get install -y --no-install-recommends ca-certificates curl git build-essential cmake libssl-dev unzip openssl

ENV CIVETWEB_GITTAG=v1.11
ENV UTS_SERVER_DOWNLOAD_URL https://github.com/kakwa/uts-server/archive/master.zip

RUN set -ex; \
    curl -s -LO "$UTS_SERVER_DOWNLOAD_URL"; \
    unzip master.zip; \
    mv uts-server-master uts-server; \
    mv uts-server /opt; \
    rm master.zip; \
    cd /opt/uts-server; \
    cmake . -DBUNDLE_CIVETWEB=ON; \
    make;

WORKDIR /opt/uts-server
RUN ./tests/cfg/pki/create_tsa_certs; \
    sed -i 's/127.0.0.1/0.0.0.0/g' /opt/uts-server/tests/cfg/uts-server.cnf; \
    sed -i 's/127.0.0.1/0.0.0.0/g' /opt/uts-server/tests/cfg/uts-server-ssl.cnf;

VOLUME /etc/uts-server

# not relevant for this test, at the moment
# COPY docker-entrypoint.sh /usr/local/bin/
# ENTRYPOINT ["docker-entrypoint.sh"]

EXPOSE 2020
CMD ["/opt/uts-server/uts-server", "-c", "/opt/uts-server/tests/cfg/uts-server.cnf", "-D"]

[kakwa]

Hello,

I've done a few modifications that should help you dockerize uts-server:

• add a few configuration parameters to disable syslog and enable stdout logging (stdout logging was previously only when running in debug mode, which is not ideal in production).
• I've removed stdout buffering when logging in stdout
• I've also fixed the ctrl-c issue, when running in foreground, I was not configuring a signal handler, as a consequence, uts-server was not responding to SIGTERM in the container (you could see the exact same behavior with '/bin/sleep 60' as CMD btw)

For info, here is the slightly tweaked Dockerfile I used:

FROM debian:stretch-slim

RUN groupadd -r uts && useradd -r -g uts uts
RUN mkdir /etc/uts-server && chown uts:uts /etc/uts-server

RUN apt-get update; \
    apt-get install -y --no-install-recommends ca-certificates curl git build-essential cmake libssl-dev unzip openssl

ENV CIVETWEB_GITTAG=v1.11
ENV UTS_SERVER_DOWNLOAD_URL https://github.com/kakwa/uts-server/archive/master.zip

RUN set -ex; \
    curl -s -LO "$UTS_SERVER_DOWNLOAD_URL"; \
    unzip master.zip; \
    mv uts-server-master uts-server; \
    mv uts-server /opt; \
    rm master.zip; \
    cd /opt/uts-server; \
    cmake . -DBUNDLE_CIVETWEB=ON; \
    make;

WORKDIR /opt/uts-server
RUN ./tests/cfg/pki/create_tsa_certs; \
    sed -i 's/127.0.0.1/0.0.0.0/g' /opt/uts-server/tests/cfg/uts-server.cnf; \
    sed -i 's/127.0.0.1/0.0.0.0/g' /opt/uts-server/tests/cfg/uts-server-ssl.cnf; \
    sed -i 's/.*log_to_syslog.*/log_to_syslog = no/g' /opt/uts-server/tests/cfg/uts-server.cnf; \
    sed -i 's/.*log_to_syslog.*/log_to_syslog = no/g' /opt/uts-server/tests/cfg/uts-server-ssl.cnf; \
    sed -i 's/.*log_to_stdout.*/log_to_stdout = yes/g' /opt/uts-server/tests/cfg/uts-server.cnf; \
    sed -i 's/.*log_to_stdout.*/log_to_stdout = yes/g' /opt/uts-server/tests/cfg/uts-server-ssl.cnf;

VOLUME /etc/uts-server

# not relevant for this test, at the moment
# COPY docker-entrypoint.sh /usr/local/bin/
# ENTRYPOINT ["docker-entrypoint.sh"]

EXPOSE 2020
CMD ["/opt/uts-server/uts-server", "-c", "/opt/uts-server/tests/cfg/uts-server.cnf"]

(just to be sure, I'm guessing this Dockerfile is only a first batch, it's not suited right now to build a container running in production, specially the certificate/CA management, but also the fact the container is unnecessary big with all the build dependencies).

[nicholasamorim]

Thank you so much for your time on making those changes!

That was just a first iteration that stopped short when hitting the issues.

The idea is to:

• Remove the the deps after compiling uts-server
• Be able to use the VOLUME to store/pass configuration files along with certificates in /etc/uts-server but also allow an user to get the container running for test/development by having test certificates in place.
• Execute the server as uts-server
• Allow any arguments to be be passed on startup using docker-entrypoint.sh
• Freeze uts-server version by checking out the project from a URL with a commit hash as there are no "releases".

Any suggestions?

[kakwa]

For the first point, I'm far from a docker expert, but the cleaner way to do it, IMHO, is to use multi-stage builds: https://docs.docker.com/develop/develop-images/multistage-build/ Basically, you create a build container, compile, create a "run" container, copy the binary from the build container to the "run" container).

For the latter point, I will create a tag with these changes soon, making it a proper release. But in any case, you can use the following URL: https://github.com/kakwa/uts-server/archive/${VERSION}.tar.gz.

The nice thing is it works with both tags and commit hash:

# tag example:
export VERSION=0.1.10
wget https://github.com/kakwa/uts-server/archive/${VERSION}.tar.gz

# Commit hash:
export VERSION=225fbd29128854e75ca9664d8b6f262c0d1aaf88
wget https://github.com/kakwa/uts-server/archive/${VERSION}.tar.gz

[nicholasamorim]

The multistage build does make sense, although I'm yet to see one. I'll give it a go, though.

Thanks again!

[nicholasamorim]

I've used the multistage build and it reduced the size of the image about 30% if not more. Thanks for that and for all the changes, much appreciated.

Here it is: https://hub.docker.com/r/nicholasamorim/uts-server

[kakwa]

I've published a new version with these changes: 0.2.0

It also includes a few other changes (nicer landing page, download link for the CA and the certificate).

I think I can close this issue now.