Open jiasli opened 2 years ago
so we dont think this will work reliably.
so we dont think this will work reliably.
It will. If you see any errors or problems, please reply to this GitHub issue (https://github.com/Azure/azure-cli/issues/19591) with the error message you see.
These instructions are great and they 100% work, I'm just curious, why not just use the official MS azure-cli image?
Seems to already be alpine-based.
❯ docker run -it --rm mcr.microsoft.com/azure-cli:latest sh
/ # cat /etc/alpine-release
3.14.2
/ # apk --version
apk-tools 2.12.7, compiled for x86_64.
/ # az --version 2>&1 | grep -i azure-cli
azure-cli 2.29.0
@OmegaVVeapon With Alpine being an OS, there's a plethora of cloud-provider-agnostic images based on alpine - like postgres:11-alpine, golang:17-alpine etc. Image creators can't (and shouldn't) make decisions about the cloud provider.
I don't get your point about cloud provider. The Azure CLI docker image is simply using the public alpine
image:
https://github.com/Azure/azure-cli/blob/14d5784d041c2aad5e0a7e5610366fba736fb096/Dockerfile#L8
The Azure CLI image is not related to Azure as a cloud provider at all.
I may have misunderstood your comment @OmegaVVeapon
What I was trying to say is that these install instructions are helpful/necessary whether or not the azure-cli
image is based on Alpine. It doesn't make sense to change the base to azure-cli
for most scenarios.
For example, if I want Postgres/Alpine/Azure CLI, it makes more sense to do this:
FROM postgres:11-alpine
RUN apk add gcc musl-dev python3-dev libffi-dev openssl-dev cargo make
RUN pip install azure-cli
Rather than to base the image on azure-cli
and complement with contents of the Postgres Dockerfile
FROM azure-cli
// [...]
@sebnyberg, this is exactly why I posted this GitHub issue! 😉
Found this, and it works, but I'm wondering why APKs of the CLI haven't simply(?) been made available for download, as there are for other major distros? I don't know Alpine very well; is this a difficult thing to do?
@dhduvall, this is because Alpine Linux is not as widely used as DEB and RPM. Let me mark this issue as a feature candidate and maybe plan it in the future.
Thanks for that dokumentation, it helps alot.
How ever I still have an issue as our image build server is offline and has only access to external packages like Docker Images, Linux Packages, Python packages to a Nexus OOS proxying the external ressources.
Every step works, but pip install azure-cli ...
fails.
Error:
Updating crates.io index
warning: spurious network error (2 tries remaining): failed to connect to github.com: Connection refused; class=Os (2)
warning: spurious network error (1 tries remaining): failed to connect to github.com: Connection refused; class=Os (2)
error: failed to get `asn1` as a dependency of package `cryptography-rust v0.1.0 (/tmp/pip-install-aomiw9lh/cryptography_f95470c855944b4e823e40877402bacf/src/rust)`
Caused by:
failed to fetch `https://github.com/rust-lang/crates.io-index`
Caused by:
failed to connect to github.com: Connection refused; class=Os (2)
Debug Info
Python: 3.9.7
platform: Linux-4.18.0-240.el8.x86_64-x86_64-with
pip: n/a
setuptools: 60.5.0
setuptools_rust: 1.1.2
How ever I see on the former step that package installation works fine
/ # apk add gcc musl-dev python3-dev libffi-dev openssl-dev cargo make
(1/23) Upgrading libcrypto1.1 (1.1.1l-r7 -> 1.1.1l-r8)
(2/23) Upgrading libssl1.1 (1.1.1l-r7 -> 1.1.1l-r8)
**(3/23) Installing rust-stdlib (1.56.1-r0)** <-- Rust Stuff
(4/23) Installing binutils (2.37-r3)
(5/23) Installing libgomp (10.3.1_git20211027-r0)
(6/23) Installing libatomic (10.3.1_git20211027-r0)
(7/23) Installing libgphobos (10.3.1_git20211027-r0)
(8/23) Installing gmp (6.2.1-r1)
(9/23) Installing isl22 (0.22-r0)
(10/23) Installing mpfr4 (4.1.0-r0)
(11/23) Installing mpc1 (1.2.1-r0)
(12/23) Installing gcc (10.3.1_git20211027-r0)
(13/23) Installing musl-dev (1.2.2-r7)
(14/23) Installing libxml2 (2.9.12-r2)
(15/23) Installing llvm12-libs (12.0.1-r0)
**(16/23) Installing rust (1.56.1-r0)** <-- Rust Stuff
(17/23) Installing cargo (1.56.1-r0)
(18/23) Installing linux-headers (5.10.41-r0)
(19/23) Installing pkgconf (1.8.0-r0)
(20/23) Installing libffi-dev (3.4.2-r1)
(21/23) Installing make (4.3-r0)
(22/23) Installing openssl-dev (1.1.1l-r8)
(23/23) Installing python3-dev (3.9.7-r4)
Works almost fine. Any idea on that how to avoid this error?
@Joerg-L, your issue doesn't seem to be caused by Azure CLI, but by cryptography
.
Does pip install cryptography
report the same error?
Does
pip install cryptography
report the same error?
yes, the same issue comes up
I have run now pip install -U pip
and after that agein pip install cryptography
which leeds now to
Collecting cryptography
Downloading http://lwfra1artifap01.gfklctx.local:8184/repository/python-pypi-proxy/packages/cryptography/36.0.1/cryptography-36.0.1-cp36-abi3-musllinux_1_1_x86_64.whl (3.8 MB)
|████████████████████████████████| 3.8 MB 19.3 MB/s
Collecting cffi>=1.12
Downloading http://lwfra1artifap01.gfklctx.local:8184/repository/python-pypi-proxy/packages/cffi/1.15.0/cffi-1.15.0.tar.gz (484 kB)
|████████████████████████████████| 484 kB 31.5 MB/s
Preparing metadata (setup.py) ... done
Collecting pycparser
Downloading http://lwfra1artifap01.gfklctx.local:8184/repository/python-pypi-proxy/packages/pycparser/2.21/pycparser-2.21-py2.py3-none-any.whl (118 kB)
|████████████████████████████████| 118 kB 28.0 MB/s
Using legacy 'setup.py install' for cffi, since package 'wheel' is not installed.
Installing collected packages: pycparser, cffi, cryptography
Running setup.py install for cffi ... done
Successfully installed cffi-1.15.0 cryptography-36.0.1 pycparser-2.21
which looks better
Thanks @Joerg-L for the confirmation. I have updated my original description to include pip install --upgrade pip
.
Both building from alpine and using it from mcr.microsoft.com/azure-cli:latest make it output image around 1.2GB for something that should be used in cli is awful :)
Compare it with awscli (amazon/aws-cli) which also is quite heavy 339MB, but why is it getting 3 times larger is my question?
Maybe you think it's a dumb question but I feel like every image in docker is exploding in storage space!
Here I saved some storage but its only around 200MB, which is made it to the size of mcr.microsoft.com/azure-cli
RUN apk add --no-cache -q --virtual=build gcc musl-dev python3-dev libffi-dev openssl-dev cargo make \
&& pip install --no-cache-dir azure-cli -q \
&& apk del --purge build
@Looooopy, the size-too-big issue of Azure CLI is tracked at https://github.com/Azure/azure-cli/issues/7387. It is mainly due to the size of Azure Python SDKs (https://github.com/Azure/azure-sdk-for-python/issues/11149). We are tightly working with SDK team to reduce the size. Stay tuned!
Question. Is there not an alpine az-cli that you can just apk add to the image?
@hholst80, I have created https://github.com/Azure/azure-cli/issues/24872 to track this feature request. We will evaluate it if the vote number is high. 🙂
One purpose to use Alpine as docker base image is to save the image size, and less packages, less security concerns
But when I saw this line in Dockerfile, install with gcc, musl-dev, python3-dev and make, etc, the size can't be small.
RUN apk add gcc musl-dev python3-dev libffi-dev openssl-dev cargo make
and check its size of the image mcr.microsoft.com/azure-cli:latest
, I think there are a lot of tunings waiting for us to make it better
$ docker images
mcr.microsoft.com/azure-cli latest 3a5555b53114 2 weeks ago 1.32GB
...
python 3-alpine 407b57cd39cc 3 days ago 52.4MB
only a command with 1.3GB, that surprises me.
Any way to improve by Multi-stage builds and this document as well
https://rodneyosodo.medium.com/minimizing-python-docker-images-cf99f4468d39
# Stage 1 - Install build dependencies
FROM python:3.7-alpine AS builder
WORKDIR /app
RUN python -m venv .venv && .venv/bin/pip install --no-cache-dir -U pip setuptools
COPY requirements.txt .
RUN .venv/bin/pip install --no-cache-dir -r requirements.txt && find /app/.venv ( -type d -a -name test -o -name tests \) -o \( -type f -a -name '*.pyc' -o -name '*.pyo' \) -exec rm -rf '{}' \+
# Stage 2 - Copy only necessary files to the runner stage
FROM python:3.7-alpine
WORKDIR /app
COPY --from=builder /app /app
COPY app.py .
ENV PATH="/app/.venv/bin:$PATH"
CMD ["python", "app.py"]
Got it. It's quite big.
I've managed to construct the image using a multi-stage build, yet the size remains considerable, exceeding 1.2GB
# I put here only for reference, not useful to size reduce
Dockerfile
# Stage 1 - Install build dependencies
FROM python:3-alpine AS builder
WORKDIR /app
RUN apk add gcc musl-dev python3-dev libffi-dev openssl-dev cargo make
RUN python -m venv .venv && .venv/bin/pip install --no-cache-dir -U pip setuptools
#RUN .venv/bin/pip install --no-cache-dir azure-cli && find /app/.venv ( -type d -a -name test -o -name tests \) -o \( -type f -a -name '*.pyc' -o -name '*.pyo' \) -exec rm -rf '{}' \+
RUN .venv/bin/pip install --no-cache-dir azure-cli
ENV PATH="/app/.venv/bin:$PATH"
# Stage 2 - Copy only necessary files to the runner stage
FROM python:3-alpine
WORKDIR /app
COPY --from=builder /app /app
#COPY app.py .
ENV PATH="/app/.venv/bin:$PATH"
#CMD ["python", "app.py"]
Upon investigation, I discovered that the Azure CLI package itself occupies the majority of the space. This is due to the fact that all versions of the mgmt package are saved and installed together to adapt various versions of the Azure API.
$ pwd
/app/.venv/lib/python3.11/site-packages/azure/mgmt/network
$ ls -l
...
9708 v2018_11_01
9964 v2018_12_01
10180 v2019_02_01
10768 v2019_04_01
11280 v2019_06_01
11640 v2019_07_01
11860 v2019_08_01
11944 v2019_09_01
12172 v2019_11_01
12344 v2019_12_01
12736 v2020_03_01
12924 v2020_04_01
13592 v2020_05_01
14180 v2020_06_01
14420 v2020_07_01
14596 v2020_08_01
14700 v2020_11_01
14880 v2021_02_01
16984 v2022_01_01
$ ls -ld v*|wc -l
33
In this example, when install an az network extension with mgmt, the related packages are installed 33 times. This same design applies to other extensions as well, that's the main reason why this Azure-CLI python package is so huge.
To reduce the size, we need explore installing only the latest version of the packages, which would significantly decrease the overall size.
I have tried to address that fact. If you look at the az-cli PRs you will find a packaging PR regarding the Docker build.
Note that docker images will show the uncompressed size of the image. There is a lot of python code.
And... also a lot of __pycache__
. I deleted that as well. The output image size is now roundabout 50% smaller.
Install azure cli by python, it works, but so slow to build. plz support installing on apk packages.. :( :(
This is broken since apk currently returns python 3.12 and support isn't there: https://github.com/Azure/azure-cli/issues/27673
you can force APK to use an older repo:
# enforce old repo for python version: https://github.com/Azure/azure-cli/issues/27673
RUN echo "http://dl-cdn.alpinelinux.org/alpine/v3.19/main" > /etc/apk/repositories \
&& echo "http://dl-cdn.alpinelinux.org/alpine/v3.19/community" >> /etc/apk/repositories
https://pkgs.alpinelinux.org/packages?name=python3-dev*&branch=v3.19&repo=&arch=&maintainer=#
This is broken since apk currently returns python 3.12 and support isn't there: #27673
You can install setuptools
via pip and it works.
@jiasli Could we have some reference to e.g. https://github.com/Azure/azure-cli/blob/dev/alpine.dockerfile in the official docs? That there is at least a hint how to do the installation in alpine?
Full working Dockerfile as 2024-07-27:
FROM alpine
RUN apk add py3-pip
RUN apk add gcc musl-dev python3-dev libffi-dev openssl-dev cargo make
RUN pip install --break-system-packages --upgrade pip setuptools
RUN pip install --break-system-packages azure-cli
Full working Dockerfile as 2024-07-27:
FROM alpine RUN apk add py3-pip RUN apk add gcc musl-dev python3-dev libffi-dev openssl-dev cargo make RUN pip install --break-system-packages --upgrade pip setuptools RUN pip install --break-system-packages azure-cli
The big issue here is, a simple command line, the image size is over several GBs.
could you confirm it with your way?
could you confirm it with your way?
Image build from the Dockerfile that I posted is a bit bellow 3 GB in size.
could you confirm it with your way?
Image build from the Dockerfile that I posted is a bit bellow 3 GB in size.
Take a look at this image I built (https://hub.docker.com/repository/docker/alpine/azure_cli/general). It's useful for CI/CD pipelines or personal use locally, with a size of less than 1GB.
@ozbillwang do you have the sources available somewhere where one does not have to create an account first?
Users frequently face issues while installing Azure CLI on Alpine Linux docker containers (#7437, #8863, #9167, #4352) , this issue serves as an official installation guide. We will consider moving it to the How to install the Azure CLI official document if it is proven to be useful.
Azure CLI official docker image
The official Azure CLI docker image is built upon Alpine Linux. We recommend using it whenever possible. You may also take the Dockerfile as a reference.
Install Azure CLI step-by-step
Select base image
python:alpine
The official Python docker image is preferred as the base image.
alpine
If you would like to use the original official Alpine docker image, you need to install
pip
andpython
:Install dependencies
As Alpine Linux is not
manylinux2010
(or greater) compatible, usingpip
to install libraries likecryptography
will fail without underlying dependencies. Install dependencies with:See https://cryptography.io/en/latest/installation/#building-cryptography-on-linux
Install Azure CLI
Simply upgrade
pip
and usepip
to install Azure CLI:Use
Dockerfile
Here are the
Dockerfile
s you may use as a starting point:Create
Dockerfile
python:alpine
as base imagealpine
as base imageBuild the image
Run
az
commands