Azure / azure-storage-azcopy

The new Azure Storage data transfer utility - AzCopy v10
MIT License
602 stars 217 forks source link

azcopy: command not found #621

Closed worldspawn closed 4 years ago

worldspawn commented 4 years ago

Which version of the AzCopy was used?

https://azcopyvnext.azureedge.net/release20190703/azcopy_linux_amd64_10.2.1.tar.gz

Which platform are you using? (ex: Windows, Mac, Linux)

"alpine" container

What command did you run?

So I downloaded this with: curl -L -JO https://azcopyvnext.azureedge.net/release20190703/azcopy_linux_amd64_10.2.1.tar.gz

Then extracted

tar -xf azcopy_linux_amd64_10.2.1.tar.gz

I get a file called "azcopy"...

-rwxr-xr-x 1 501 staff 18914367 Jul  4 00:10 azcopy

that I cannot run bash: azcopy: command not found

I'm not really a linux user but it seems to already be set to allow execution. Something else needed?

JohnRusk commented 4 years ago

Try running it by calling it

./azcopy

Instead of just azcopy.

And make sure your working directory, at the time, is the place where the azcopy executable is.

If that doesn't work, please let us know and we'll see what else we can do to help.

zezha-msft commented 4 years ago

Hi @worldspawn, the OS can only find azcopy if it was part of the path. Invoking it with a relative path ./azcopy or absolute path /usr/name/bla/azcopy would be the alternatively way.

worldspawn commented 4 years ago

From the top...

kubectl.exe run -it alpine --image=alpine

now i'm in the container...

apk update
apk add curl
curl -L -JO https://azcopyvnext.azureedge.net/release20190703/azcopy_linux_amd64_10.2.1.tar.gz
tar -xf azcopy_linux_amd64_10.2.1.tar.gz
cd azcopy_linux_amd64_10.2.1/

I am now in /azcopy_linux_amd64_10.2.1. ls -l output is:

/azcopy_linux_amd64_10.2.1 # ls -l
total 18488
-rw-r--r--    1 501      dialout      16068 Jul  4 00:10 ThirdPartyNotice.txt
-rwxr-xr-x    1 501      dialout   18914367 Jul  4 00:10 azcopy

Now I...

/azcopy_linux_amd64_10.2.1 # ./azcopy
/bin/sh: ./azcopy: not found

@zezha-msft @JohnRusk azcopy is definately in my cwd.

worldspawn commented 4 years ago

Repro!

Dockerfile

# => Run container
FROM alpine

RUN apk update
RUN apk add curl
RUN curl -L -JO https://azcopyvnext.azureedge.net/release20190703/azcopy_linux_amd64_10.2.1.tar.gz
RUN tar -xf azcopy_linux_amd64_10.2.1.tar.gz

WORKDIR /azcopy_linux_amd64_10.2.1

docker build -t azcopy . docker run -it azcopy /bin/ash

zezha-msft commented 4 years ago

Sorry @worldspawn, we cannot debug your environment unfortunately.

derekbekoe commented 4 years ago

@worldspawn I ran into the same issue. In short, to resolve it I ran apk add libc6-compat. Let me know if that works for you as well.

--

The linux_amd64 build of azcopy requires some shared libraries that aren't pre-installed on alpine.

ldd azcopy will list these shared libraries:

# ldd azcopy
        /lib64/ld-linux-x86-64.so.2 (0x7f94150c9000)
        libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7f94150c9000)
        libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f94150c9000)

A search of ld-linux-x86-64.so.2 on https://pkgs.alpinelinux.org/ reveals it can be found in libc6-compat.

Then, running azcopy works as expected on alpine.

derekbekoe commented 4 years ago

@zezha-msft If the above works for others, it would be nice to see it added to the doc. I've filed https://github.com/MicrosoftDocs/azure-docs/issues/40155 for this.

worldspawn commented 4 years ago

Ah thanks @derekbekoe. i ended up just changing my base to ubuntu. I'll try it on Monday.

worldspawn commented 4 years ago

@derekbekoe confirmed this made it work. 👍 🥇

Working (improved) Dockerfile

# => Run container
FROM alpine

# Add bash
RUN apk update
RUN apk add libc6-compat
RUN apk add curl
RUN mkdir /azcopy 
WORKDIR /azcopy
RUN curl -L -J https://aka.ms/downloadazcopy-v10-linux -o ./azcopyarchive.tar.gz
RUN tar -xf ./azcopyarchive.tar.gz --strip-components=1

Would love to see an image maintained by the azcopy team.

zezha-msft commented 4 years ago

Thanks a lot @derekbekoe for posting the solution here! And thanks @worldspawn for confirming it solved your problem.

Hi @normesta, could you please add this information into our docs? Thanks!

alliagesas commented 2 years ago

apk add libc6-compa

Works for me!

Thanks a lot !!!

milanio commented 1 year ago

Thank you @derekbekoe!

Snehil03 commented 1 year ago

Hello,

I have upgraded alpine image to 3:17 now it stopped working again. I can see azcopy binaries are in place but it's complain about the /bin/sh: ./azcopy: not found

my code looks like below :

FROM alpine:3.17 as azcopy

# Install AzCopy
RUN apk add --no-cache curl  && \
    apk add -U python3 bash && \
    apk add --no-cache py-pip && \
    apk add -U --no-cache -q --virtual=build gcc musl-dev python3-dev libffi-dev openssl-dev cargo make libc6-compat && \
    curl -sL https://aka.ms/downloadazcopy-v10-linux | tar zx && \
    mv azcopy_linux_amd64_*/azcopy /usr/local/bin/ && \
    rm -rf azcopy_linux_amd64_* && \
    chown -R root:root /usr/local/bin/azcopy && \
    chmod a+x /usr/local/bin/azcopy

# Add the directory containing the AzCopy binary to the PATH
ENV PATH="/usr/local/bin:${PATH}"

# -----------------------------------------------------------------------------
FROM azcopy as azure-agent
# -----------------------------------------------------------------------------
# Push helm packages to OCR images in Azure
ENV HELM_EXPERIMENTAL_OCI=1
ENV AZURE_CLI_VERSION 2.45.0
ENV AZCOPY_VERSION 10.17.0
RUN apk add -U python3 bash \
    && apk add -U --no-cache -q --virtual=build gcc musl-dev python3-dev libffi-dev openssl-dev cargo make libc6-compat \
    && pip install --no-cache-dir azure-cli==${AZURE_CLI_VERSION} -q \
    && apk del --purge build \
    && chown -R root:root /usr/local/bin/ 

COPY --from=azcopy /usr/local/bin/azcopy /usr/local/bin

ENV PATH="/usr/local/bin/:${PATH}"

command used :

docker build -t azcp-1 . --target=azure-agent --no-cache
docker run -it --rm azcp-1

I have replicated code even as part of multi build but want to make a point here that even having all the dependency in place still it does not work.

gounthar commented 1 week ago

Same for me.

docker build .
[+] Building 3.5s (7/7) FINISHED                                                                                                                      docker-container:asciidoctor
 => [internal] load build definition from Dockerfile                                                                                                                          0.0s
 => => transferring dockerfile: 584B                                                                                                                                          0.0s
 => [internal] load metadata for ghcr.io/maxmind/geoipupdate:v7.0.1                                                                                                           0.8s
 => [auth] maxmind/geoipupdate:pull token for ghcr.io                                                                                                                         0.0s
 => [internal] load .dockerignore                                                                                                                                             0.1s
 => => transferring context: 2B                                                                                                                                               0.0s
 => CACHED [1/3] FROM ghcr.io/maxmind/geoipupdate:v7.0.1@sha256:80c57598a9ff552953e499cefc589cfe7b563d64262742ea42f2014251b557b0                                              0.0s
 => => resolve ghcr.io/maxmind/geoipupdate:v7.0.1@sha256:80c57598a9ff552953e499cefc589cfe7b563d64262742ea42f2014251b557b0                                                     0.0s
 => [2/3] RUN apk update &&     apk add --no-cache libc6-compat wget &&     wget https://aka.ms/downloadazcopy-v10-linux &&     tar -xvf downloadazcopy-v10-linux &&     cp   2.3s
 => ERROR [3/3] RUN ls -artl /usr/bin/azcopy && file /usr/bin/azcopy && /usr/bin/azcopy --version                                                                             0.1s
------
 > [3/3] RUN ls -artl /usr/bin/azcopy && file /usr/bin/azcopy && /usr/bin/azcopy --version:
0.090 -rwxr-xr-x    1 root     root      33664744 Sep  3 14:33 /usr/bin/azcopy
0.090 /bin/sh: file: not found