search

graphite-project / docker-graphite-statsd

Official Docker image for Graphite
MIT License
478 stars 174 forks source link

Reduce Docker Image size from 864.8 MB to 465.3 MB #51

Closed davecoutts closed 6 years ago

davecoutts commented 6 years ago

I have done a little experimentation and it appears it is possible to reduce the size of the Docker image by 50%.

The size reduction is mostly achieved by,

The key thing to note with Docker is that once a RUN is complete nothing can be deleted from the resulting layer. Clean-up/deletion needs to happen in the RUN(layer) to be effective.

The Dockerfile below is less user friendly to read than the current published one. With the majority of the install work being done in one RUN. The size reduction might warrant that loss of readability.

I have only tested the resulting container briefly. I did nothing more than run the example events generator bash snippet and observe stats on a chart in graphite-web.

Might Docker Image size reduction following something similar to Dockerfile changes below be of interest to the project?

FROM phusion/baseimage:0.10.2
MAINTAINER XXXX YYYY <xxxx.yyyy@gmail.com>

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update --fix-missing && \
    apt-get install --yes --no-install-recommends \
    nginx \
    python-flup \
    python-pip \
    python-ldap \
    expect \
    memcached \
    sqlite3 \
    libcairo2 \
    python-cairo \
    python-rrdtool \
    nodejs && \
    apt-get clean && \
    apt-get autoremove --yes  && \
    rm -rf /var/lib/apt/lists/*

# choose a timezone at build-time
# use `--build-arg CONTAINER_TIMEZONE=Europe/Brussels` in `docker build`
ARG CONTAINER_TIMEZONE

RUN if [ ! -z "${CONTAINER_TIMEZONE}" ]; \
    then ln -sf /usr/share/zoneinfo/$CONTAINER_TIMEZONE /etc/localtime && \
    dpkg-reconfigure -f noninteractive tzdata; \
    fi

ARG version=1.1.4
ARG whisper_version=${version}
ARG carbon_version=${version}
ARG graphite_version=${version}

ARG whisper_repo=https://github.com/graphite-project/whisper.git
ARG carbon_repo=https://github.com/graphite-project/carbon.git
ARG graphite_repo=https://github.com/graphite-project/graphite-web.git

ARG statsd_version=v0.8.0
ARG statsd_repo=https://github.com/etsy/statsd.git

# Install compiler and dev libraries.
# Clone sources from statsd, whisper, carbon and graphite github repositories.
# Build and install whisper, carbon and graphite.
# Remove compiler and dev libraries.
# Delete source code directories.
# Clear apt cache.
RUN apt-get update --fix-missing && \
    apt-get install --yes --no-install-recommends gcc python-dev libffi-dev libcairo2-dev pkg-config git && \

    python -m pip install --no-cache-dir --upgrade pip setuptools && \
    pip install --no-cache-dir django==1.11.15 fadvise msgpack-python && \

    git clone -b ${statsd_version} ${statsd_repo} /opt/statsd && \
    git clone -b ${whisper_version} --depth 1 ${whisper_repo} /usr/local/src/whisper && \
    git clone -b ${carbon_version} --depth 1 ${carbon_repo} /usr/local/src/carbon && \
    git clone -b ${graphite_version} --depth 1 ${graphite_repo} /usr/local/src/graphite-web && \

    cd /usr/local/src/whisper && \
    python ./setup.py install && \

    cd /usr/local/src/carbon && \
    pip install --no-cache-dir --requirement requirements.txt && \
    python ./setup.py install && \

    cd /usr/local/src/graphite-web && \
    pip install --no-cache-dir --requirement requirements.txt && \
    python ./setup.py install && \

    apt-get purge --yes --auto-remove cpp gcc python-dev libffi-dev libcairo2-dev pkg-config git && \
    apt-get install --yes --no-install-recommends libpython2.7 && \
    apt-get clean && \
    apt-get autoremove --yes  && \
    rm -rf /var/lib/apt/lists/* && \

    cd ${HOME} && \
    rm -rf /usr/local/src/whisper /usr/local/src/carbon /usr/local/src/graphite-web /tmp/* /var/tmp/* /usr/share/python-wheels/*

# config graphite
ADD conf/opt/graphite/conf/*.conf /opt/graphite/conf/
ADD conf/opt/graphite/webapp/graphite/local_settings.py /opt/graphite/webapp/graphite/local_settings.py
# ADD conf/opt/graphite/webapp/graphite/app_settings.py /opt/graphite/webapp/graphite/app_settings.py
WORKDIR /opt/graphite/webapp
RUN mkdir -p /var/log/graphite && \
    PYTHONPATH=/opt/graphite/webapp django-admin.py collectstatic --noinput --settings=graphite.settings

# config statsd
ADD conf/opt/statsd/config_*.js /opt/statsd/

# config nginx
RUN rm /etc/nginx/sites-enabled/default
ADD conf/etc/nginx/nginx.conf /etc/nginx/nginx.conf
ADD conf/etc/nginx/sites-enabled/graphite-statsd.conf /etc/nginx/sites-enabled/graphite-statsd.conf

# init django admin
ADD conf/usr/local/bin/django_admin_init.exp /usr/local/bin/django_admin_init.exp
ADD conf/usr/local/bin/manage.sh /usr/local/bin/manage.sh
RUN chmod +x /usr/local/bin/manage.sh && /usr/local/bin/django_admin_init.exp

# logging support
RUN mkdir -p /var/log/carbon /var/log/graphite /var/log/nginx
ADD conf/etc/logrotate.d/graphite-statsd /etc/logrotate.d/graphite-statsd

# daemons
ADD conf/etc/service/carbon/run /etc/service/carbon/run
ADD conf/etc/service/carbon-aggregator/run /etc/service/carbon-aggregator/run
ADD conf/etc/service/graphite/run /etc/service/graphite/run
ADD conf/etc/service/statsd/run /etc/service/statsd/run
ADD conf/etc/service/nginx/run /etc/service/nginx/run

# default conf setup
ADD conf /etc/graphite-statsd/conf
ADD conf/etc/my_init.d/01_conf_init.sh /etc/my_init.d/01_conf_init.sh

# defaults
EXPOSE 80 2003-2004 2023-2024 8080 8125 8125/udp 8126
VOLUME ["/opt/graphite/conf", "/opt/graphite/storage", "/opt/graphite/webapp/graphite/functions/custom", "/etc/nginx", "/opt/statsd", "/etc/logrotate.d", "/var/log"]
WORKDIR /
ENV HOME /root
ENV STATSD_INTERFACE udp

CMD ["/sbin/my_init"]
deniszh commented 6 years ago

Hello @davecoutts, Thanks a lot for your contribution to the project, much appreciated! Unfortunately, we can't easily remove gcc, dev libraries and python sources from the target image, because currently it can be needed to make offline Graphite installation if /opt/graphite directory is mounted by docker and empty - https://github.com/graphite-project/docker-graphite-statsd/blob/master/conf/etc/my_init.d/01_conf_init.sh#L20-L22. Probably you can copy whole /opt/graphite somewhere and copy it back instead of installation in runtime, which is possible, but require some work - and it will probably consume space of final image too. I have no big interest in the reducing Docker images' size per se, but, of course, I will gladly accept PR if it will not break too much of existing functionality of the image.

davecoutts commented 6 years ago

Hi @deniszh Oh I see. I hadn't appreciated the image had that functionality. The size optimisations I have applied will certainly break the setup where local volumes are mounts. I'm going to close this issue as what I have suggested is not appropriate in this case. Thanks.

deniszh commented 6 years ago

Hi @davecoutts , I made similar PR using your idea, but using multistage build instead - https://github.com/graphite-project/docker-graphite-statsd/pull/56 Thanks!

davecoutts commented 6 years ago

Hi Denis

Good job. I like the multistage approach.

I came across dive the other day. It seems a very good tool for gauging image bloat. https://github.com/wagoodman/dive https://github.com/wagoodman/dive/releases

Your new image scores very well at 88% when tested with dive. sudo dive docker-graphite-statsd:latest

Regards

Dave

On Sat, 27 Oct 2018 at 23:13, Denis Zhdanov notifications@github.com wrote:

Hi @davecoutts https://github.com/davecoutts , I made similar PR using your idea, but using multistage build instead -

56 https://github.com/graphite-project/docker-graphite-statsd/pull/56

Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/graphite-project/docker-graphite-statsd/issues/51#issuecomment-433659291, or mute the thread https://github.com/notifications/unsubscribe-auth/AAdj0hdMVAFSc_VPlnT7sR6Y-CYuFhrZks5upNqPgaJpZM4W1vIU .

deniszh commented 6 years ago

Nice tool, thanks, @davecoutts !