search

sgerrand / alpine-pkg-glibc

A glibc compatibility layer package for Alpine Linux
2.05k stars 280 forks source link

Distribute through the official Alpine APK repo so we can `apk add glibc` #1

Closed chriskilding closed 8 years ago

chriskilding commented 9 years ago

It looks like quite a few people are using your build of glibc in their minimal Docker base images, for things which have it as a hard dependency (like Oracle Java).

At the moment it is necessary to curl the apk from your build server's artifact store, and install it manually (necessitating the use of a technically untrusted package source, which isn't too good).

So it would be great if you could rig your build server to automatically push the compiled APK to the official Alpine repos, so we could just install it with apk add glibc, or alternatively find some other process to release it there.

ricoli commented 8 years ago

yes please, what @themasterchef said. Also, would be great to have glibc 2.5 or greater. Ping @andyshinn

andyshinn commented 8 years ago

I'll look at this package again soon. There are a couple problems with it, most notably that it doesn't build on latest Arch linux glibc.

ricoli commented 8 years ago

That's awesome news @andyshinn let us know how it goes :+1:

tromantic commented 8 years ago

:+1:

moritzheiber commented 8 years ago

:+1:

bryanlatten commented 8 years ago

:+1:

andyshinn commented 8 years ago

I talked with ncopa about this the other day and the general consensus is that:

The first two are are more should be but the third one is probably more must have and is not trivial at all. So, it is unlikely that we will see this existing in the main (or testing) Alpine repositories anytime soon.

I'd be more interested in pushing harder on the evangelism side. Lets make Alpine Linux a popular choice for developers and the container ecosystem. We can then push on software maintainers to start building binaries for musl and/or supporting musl as build targets.

If you are not already filing issues about software failing to build or not supporting musl libc and Alpine Linux, please do so! It never hurts to ask.

tromantic commented 8 years ago

Thank you for responding.

Hopefully as more companies leverage Alpine, more developers realize the need for what you proposed to become reality.

ghostbar commented 8 years ago

Hey @andyshinn, no need to add them into official repos to be able to do apk add to it. I made some packages and just took it to the gh-pages branch and wrote an index.html on each directory.

You can see an example in https://github.com/ghostbar/alpine-pkg-nodejs; check the listing.sh script and the structure inside v3.2/pkgs. I point users to v3.2/pkgs on their /etc/apk/repositories and that basically has $ARCH/APKINDEX.tar.gz which you are already doing. If they add your key to their /etc/apk/keys they won't need to add --allow-untrusted.

But you probably already knew this.

andyshinn commented 8 years ago

Someone in Glider Labs made a small proxy to be able to install packages from GitHub releases: apk -X apkproxy.heroku.com/andyshinn/alpine-pkg-glibc add glibc glibc-bin. Longer term, this could be fleshed out into a more global repository where community members can submit links to their package GitHub repos to be included (when they don't fit for Alpine community, testing, or main repos).

ghostbar commented 8 years ago

That's really cool! What about support for different alpine releases? :D

andyshinn commented 8 years ago

I think it really depends on if the tool becomes an actuality. I think it is an awesome first step to being able to manage an external distributed Alpine package repository. But questions like support for multiple Alpine releases and multiple package versions (maybe (ab)using package pinning as the GitHub release pointer) will be better asked once there is a proper repository.

CC @mattaitchison

arthurtsang commented 8 years ago

+1

ecliptik commented 8 years ago

@andyshinn thank you for the apk -X command, and I tried it in a new Dockerfile but it's giving me an error

docker build --no-cache -t glibc .
Sending build context to Docker daemon   191 kB
Step 1 : FROM alpine
 ---> 3372fee49a30
Step 2 : RUN apk -X apkproxy.heroku.com/andyshinn/alpine-pkg-glibc add glibc glibc-bin
 ---> Running in 2f003c2d2209
WARNING: Ignoring apkproxy.heroku.com/andyshinn/alpine-pkg-glibc/x86_64/APKINDEX.tar.gz: No such file or directory
WARNING: Ignoring APKINDEX.d3812b7e.tar.gz: No such file or directory
WARNING: Ignoring APKINDEX.bb2c5760.tar.gz: No such file or directory
ERROR: unsatisfiable constraints:
  glibc (missing):
    required by: world[glibc]
  glibc-bin (missing):
    required by: world[glibc-bin]
The command '/bin/sh -c apk -X apkproxy.heroku.com/andyshinn/alpine-pkg-glibc add glibc glibc-bin' returned a non-zero code: 2

Dockerfile: 

FROM alpine:3.3

RUN apk -X apkproxy.heroku.com/andyshinn/alpine-pkg-glibc add glibc glibc-bin

CMD /bin/sh

Am I doing something wrong? I also tried curling the *.apk files from the Github release and installing the local packages in the Dockerfile and get the same ERROR: unsatisfiable constraints: error.

ghostbar commented 8 years ago

@ecliptik add an --update after the apk bit; your system does not knows about what's inside that repo you're sending.

The -X is just a shortcut, so you don't have to add it into /etc/apk/repositories.

So RUN apk --update -X apkproxy.heroku.com/andyshinn/alpine-pkg-glibc add glibc glibc-bin

mattatcha commented 8 years ago

@ecliptik The following should work.

FROM alpine:3.3

RUN apk --allow-untrusted --no-cache -X http://apkproxy.heroku.com/andyshinn/alpine-pkg-glibc add glibc glibc-bin

CMD /bin/sh
ecliptik commented 8 years ago

Thank you @ghostbar and @mattaitchison , it was the --allow-untrusted flag I was missing and now it's installing correctly.

moritzheiber commented 8 years ago

Please do not use --allow-untrusted. The packages all carry valid signatures, the procedure to install @andyshinn's key is documented in another thread. This --allow-untrusted nonsense has to stop :smiley:

sgerrand commented 8 years ago

I think the previous answers have covered the original request. If not, please comment. 😸

carlomas commented 7 years ago

Hi @andyshinn I've tried to create the image, but it's still failing

carlo@carlo-ThinkPad-W541:~/temp/docker$ docker build --no-cache -t alpine-glibc .
Sending build context to Docker daemon 2.048 kB
Step 1 : FROM alpine
 ---> ee4603260daa
Step 2 : RUN apk --update -X apkproxy.heroku.com/andyshinn/alpine-pkg-glibc add glibc glibc-bin
 ---> Running in 703142ace960
WARNING: Ignoring apkproxy.heroku.com/andyshinn/alpine-pkg-glibc/x86_64/APKINDEX.tar.gz: No such file or directory
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/community/x86_64/APKINDEX.tar.gz
ERROR: unsatisfiable constraints:
  glibc (missing):
    required by: world[glibc]
  glibc-bin (missing):
    required by: world[glibc-bin]
The command '/bin/sh -c apk --update -X apkproxy.heroku.com/andyshinn/alpine-pkg-glibc add glibc glibc-bin' returned a non-zero code: 2

The Docker file is:

FROM alpine

RUN apk --update -X apkproxy.heroku.com/andyshinn/alpine-pkg-glibc add glibc glibc-bin

CMD /bin/sh

Any suggestion?

sgerrand commented 7 years ago

@carlomas: 👋 I've taken on maintenance of this repository. Your Dockerfile is failing for a few reasons:

  1. You need to import the private key used to sign the index before trying to install a package from outside the Alpine repositories.
  2. If you want to use apkproxy.heroku.com as a proxy to install this package, then you need to prefix it with a protocol (i.e. http://).

The following Dockerfile, based on your simplified example,:

FROM alpine

RUN apk --no-cache add ca-certificates openssl && \
    wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://raw.githubusercontent.com/sgerrand/alpine-pkg-glibc/master/sgerrand.rsa.pub && \
    apk --no-cache -X http://apkproxy.heroku.com/sgerrand/alpine-pkg-glibc add glibc glibc-bin

CMD /bin/sh

builds successfully:

docker build .
Sending build context to Docker daemon 2.048 kB
Step 1 : FROM alpine
 ---> ee4603260daa
Step 2 : RUN apk --no-cache add ca-certificates openssl &&     wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://raw.githubusercontent.com/sgerrand/alpine-pkg-glibc/master/sgerrand.rsa.pub &&     apk --no-cache -X http://apkproxy.heroku.com/sgerrand/alpine-pkg-glibc add glibc glibc-bin
 ---> Running in 40766fa49bbe
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/community/x86_64/APKINDEX.tar.gz
(1/2) Installing ca-certificates (20160104-r4)
(2/2) Installing openssl (1.0.2j-r0)
Executing busybox-1.24.2-r11.trigger
Executing ca-certificates-20160104-r4.trigger
OK: 6 MiB in 13 packages
fetch http://apkproxy.heroku.com/sgerrand/alpine-pkg-glibc/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/community/x86_64/APKINDEX.tar.gz
(1/3) Installing glibc (2.23-r3)
(2/3) Installing libgcc (5.3.0-r0)
(3/3) Installing glibc-bin (2.23-r3)
Executing glibc-bin-2.23-r3.trigger
OK: 12 MiB in 16 packages
 ---> b5c2e9efcc31
Removing intermediate container 40766fa49bbe
Step 3 : CMD /bin/sh
 ---> Running in 430a9b583f24
 ---> f9ab794e1c3c
Removing intermediate container 430a9b583f24
Successfully built f9ab794e1c3c
carlomas commented 7 years ago

@sgerrand thanks, it worked perfectly!

So, correct me if I'm wrong, there will be no "official" support from Alpine. I suppose the original request is not fulfilled, I still have to import the private key used to sign the package, to install it.

sgerrand commented 7 years ago

@carlomas: Apologies for the delayed response. Yes, you're correct. This package is unlikely to ever make it into the official Alpine package repositories.

In the meantime, you can install the package as per the README or the method above. 👍

carlomas commented 7 years ago

@sgerrand thanks for the reply

sgerrand commented 7 years ago

For anyone arriving here in search of apkproxy.heroku.com, it looks like that application is no longer running. 😭

anuragw commented 6 years ago

@sgerrand Thanks, this helped me a lot!

mattatcha commented 6 years ago

@sgerrand It looks like heroku changed the url to https://apkproxy.herokuapp.com/.

sgerrand commented 6 years ago

Thanks for the tip!

gfenn-newbury commented 6 years ago

In case anyone is still running into the issue, use: apk --no-cache --allow-untrusted -X https://apkproxy.herokuapp.com/sgerrand/alpine-pkg-glibc add glibc glibc-bin For one reason or another, the certificate is currently untrusted.

xxx7xxxx commented 6 years ago

I don't get it why hasn't this apk added to the official index, which can fix Error relocating XXX: __fprintf_chk: symbol not found. kind of errors.

sgerrand commented 6 years ago

I don't get it why hasn't this apk added to the official index, which can fix Error relocating XXX: __fprintf_chk: symbol not found. kind of errors.

Alpine Linux is based on musl-libc, not the GNU C library. That's the reason why it hasn't been added to any of Alpine's repositories.

Adding this package to an official repository index won't fix those errors—I don't think you understand what the cause of those errors are.

earcam commented 5 years ago

You don't need --allow-untrusted with the "correct" URL.

The following works perfectly:

apk --no-cache -X https://apkproxy.herokuapp.com/sgerrand/alpine-pkg-glibc add glibc glibc-bin

But if the above does not work for you - then simply try accessing the fetched APKINDEX.tar.gz in your browser, e.g.

https://apkproxy.herokuapp.com/sgerrand/alpine-pkg-glibc/x86_64/APKINDEX.tar.gz

It'll likely tell you what domains the certificate is valid for - so just switch to an equivalent.

cytown commented 4 years ago

You don't need --allow-untrusted with the "correct" URL.

The following works perfectly:

apk --no-cache -X https://apkproxy.herokuapp.com/sgerrand/alpine-pkg-glibc add glibc glibc-bin

But if the above does not work for you - then simply try accessing the fetched APKINDEX.tar.gz in your browser, e.g.

https://apkproxy.herokuapp.com/sgerrand/alpine-pkg-glibc/x86_64/APKINDEX.tar.gz

It'll likely tell you what domains the certificate is valid for - so just switch to an equivalent.

Just tested and still need the --allow-untrusted flag...