Support Alpine

[wernight]

The direction is going towards using Docker, and on Docker the direction is towards using Alpine base images (because they're smaller), and most main packages now support Alpine (Nginx included).

It seems that it almost compiles on alpine:edge. I got past the traceback by installing libunwind@testing package but some missing references remain:

psol/lib/Release/linux/x64/pagespeed_automatic.a(1.time.o.o): In function `base::Time::FromDoubleT(dtime.cc:(.text._ZN4base4Time11FromDoubleTEd+0x2a): undefined reference to `__isnan'
psol/lib/Release/linux/x64/pagespeed_automatic.a(96.string_util.o.o): In function `net_instaweb::Str
string_util.cc:(.text._ZN12net_instaweb14StringToDoubleEPKcPd+0x14): undefined reference to `__strtod_internal'
pagespeed_automatic.a(132.ucnv.o.o): In function `pagespeed_ol_ucnv_conveucnv.c:(.text.ucnv_convertEx_46+0x6db): undefined reference to `__rawmemchr'
pagespeed_automatic.a(132.ucnv.o.o): In function `ucnv_internalConvert':
ucnv.c:(.text.ucnv_internalConvert+0x276): undefined reference to `__rawmemchr'
pagespeed_automatic.a(132.uloc.o.o): In function `pagespeed_ol_uloc_accep
uloc.c:(.text.uloc_acceptLanguageFromHTTP_46+0x214): undefined reference to `__strtod_internal'
uloc.c:(.text.uloc_acceptLanguageFromHTTP_46+0x452): undefined reference to `__strtod_internal'
pagespeed_automatic.a(132.uniset_props.o.o): In function `icu_46::Unicode
uniset_props.cpp:(.text._ZN6icu_4610UnicodeSet18applyPropertyAliasERKNS_13UnicodeStringES3_R10UErrorCode+0x35e): undefined reference to `__str
uniset_props.cpp:(.text._ZN6icu_4610UnicodeSet18applyPropertyAliasERKNS_13UnicodeStringES3_R10UErrorCode+0x462): undefined reference to `__str

It's worth finding how to make it compile and document it.

[hgl]

@robov in @nberlee's Dockerfile add

mv /tmp/incubator-pagespeed-ngx-1.12.34.3-stable /tmp/ngx_pagespeed-1.12.34.3-stable && \

below

curl -L https://github.com/pagespeed/ngx_pagespeed/archive/v${NGX_PAGESPEED_VERSION}-stable.tar.gz | tar -zx && \

[hgl]

@nberlee a quick question, why is this line necessary? I tried to compile without it, and saw no error.

[oschaaf]

With regard to the renaming of the repo, https://github.com/apache/incubator-pagespeed-ngx/commit/e733a838c35a0f2108768c2d1ead6fb7ee806cf8 was merged yesterday to unbreak the install script.

[oschaaf]

(The repo may be renamed again at some stage later on, and this change hopefully is resilient to that)

[hgl]

@oschaaf Thanks for the head up. But it seems Dockerfiles from most unofficial alpine images don't use this script. So they will continue to fail to build until fixed.

Another reason to have an official nginx + pagespeed alpine image. :)

[robov]

@oschaaf @hgl : with the merge to unbreak the install script... do I still have to rename the dir

[hgl]

Yes, that script is never used in the Dockerfile

[hgl]

But since the name is tentative, that mv command could easily fail when that happens. A better approach:

mkdir -p /tmp/ngx_pagespeed-${PAGESPEED_VERSION}-stable \
    && cd /tmp/ngx_pagespeed-${PAGESPEED_VERSION}-stable \
    && curl -L https://github.com/pagespeed/ngx_pagespeed/archive/v${PAGESPEED_VERSION}-${PAGESPEED_BRANCH}.tar.gz \
    | tar -xj --strip-components=1

You do have to install tar though, because busybox tar doesn't support strip-components

[nberlee]

@hgl The sed line was a workaround for a Alpine 3.7 busybox wait bug. (which never stopped waiting) This seems to be fixed since, making the line now obsolete.

Fixed my branch

[robov]

@nberlee Is there intentionally the hardcoded version in

 https://github.com/We-Amp/ngx-pagespeed-alpine/blob/master/mod-pagespeed-beta-1.12.34.3.tar.

[nberlee]

@robov Currently yes, once a new mod-pagespeed stable (1.12.34.3) is released, with @ashishk-1 commit: https://github.com/apache/incubator-pagespeed-mod/pull/1640/commits/faf212ccb5ee42ed3a5deaa188259d1a04f5e8b8 this can use the ${PAGESPEED_VERSION} with an official url. For now we use this specific version because its the only one with an updated GRPC and therefor building on Alpine >3.5. Also its the only version in the ngx-pagespeed-alpine branch on oschaaf his We-Amp repository.

[oschaaf]

@nberlee @robov maybe we should inventorise if it is viable to pull maintenance of the alpine image into the project to centralize efforts. Would you be up for that, and be willing to help out maintaining it?

[robov]

I would definately be interested, but I am not bringing much to this table. My experience in linux is very low and I consider myself a newbie in this space

[robov]

@nberlee : I am trying to understand your code, especially this part confuses me : LD_LIBRARY_PATH=/tmp/modpagespeed-${PAGESPEED_VERSION}/usr/lib ./configure \

When I look into the image I cannot find that path ..usr/lib , am I missing something ? What I am trying to accomplish is to create an pagespeed object in a separate dockerbuild, then copy the needed files out and into a new dockerfile and build the nginx in a that new dockerfile

This reduces my compiletime of my main image dramatically (as I do not have to recompile pagespeed every time). But I seem to be not able to find the references to : /tmp/modpagespeed-${PAGESPEED_VERSION}/usr/lib Or is this a temp direcory generated by the nginx compilation ?

[robov]

Holy Smoly... my bad... I was looking at the wrong branch... I was looking at 3.4 instead of 3.7 for this last post

[robov]

Is there someone who can help me out ? I want to build (tedious long) pagespeed in a separate container and include only the necessary files in the main build.

This is what I did I used the : nberlee/ngx-pagespeed-alpine/3.7/Dockerfile

A) Created a new dockerfile with the code up to : # Build Nginx with support for PageSpeed: B) Extracted from that container : /tmp/ngx_pagespeed-${PAGESPEED_VERSION}-stable

1) Created a new docker file from # Build Nginx with support for PageSpeed: 2) Copied the extracted files IN this container

Result : Compiling breaks with : with-ld-opt= ... not found .. but it does not mention what is not found

[robov]

I guessed the libpng was missing (only present in the temp build container, so I added that to the main one. But hten the compiling broke on "missing pagespeed optimization library"

[nberlee]

@robov I made a multi-stage Dockerfile see https://github.com/nberlee/ngx-pagespeed-alpine/tree/3.7-dev but haven't tested it yet! It should build fine

@oschaaf Yes I think I would like to be a maintainer, I think there is still some work to be done, like verifying sigatures of download files, but an official dockerhub image would be great!

[robov]

@nberlee : I am testing now..

[robov]

Step 8/26 : COPY mod-pagespeed-beta-1.12.34.3.tar.bz2 patches/*.patch /tmp/ COPY failed: stat /var/lib/docker/tmp/docker-builder857321033/mod-pagespeed-beta-1.12.34.3.tar.bz2: no such file or directory

[robov]

oops... my bad... forgot to pull your entire sources

[robov]

@nberlee it compiles nice (I have not tested the actual functionality, just compiling) I have a personal preference to have separate files instead of one multistage (this allows me to re-compile just one section) . One thing I am currently struggling with is to copy : /usr/lib/libpng* out of the container (as they contain symlinks) Any ideas ?

[robov]

When I do something like this on a container : docker cp extractfromcontainer:/usr/lib/libpng* compiled/usr/lib then it cannot find anything... when I step in I see that they are symlinks

[nberlee]

@robov https://github.com/moby/moby/issues/7710

[robov]

aha,.... and when copying the entire folder it complains ( i think it was about the symlinks)

[robov]

After 5 days of a very steep learning curve I am finally getting somewhere... @nberlee , FROM on you multistage image I create my own derivative which I then start as a container. however, now the nginx complains that 'modsecurity' is an unknown directive. I linked this in in the nginx compiling step, but when starting it up, it is not recognized. Am I missing a copy somewhere

It is linked in like this and it compiles well... any ideas ?

--add-module=${WRKDIR}/modsecurity/nginx/modsecurity \     
--add-module=${WRKDIR}/ngx_cache_purge-${CACHE_PURGE} \           
        --add-module=/tmp/ngx_pagespeed\

[robov]

Everything is working now.... BUT I receive a very wierd error On loading the page everything goes well on RE-Loading the page with ctrl-f5 I get a 404 (just pressing f5 again and it loads nicely again) ANY ideas where this problem originates or how to fix it ?

[robov]

I have setup pagespeed on my proxy that passes the requests to the backend app server. The 404 error is generated from the app server. I think the request is looking for the pagespeed cache in the backend (where it isn;t)... but.. how to prevent that ....is the next question

[nberlee]

I just pushed https://github.com/We-Amp/ngx-pagespeed-alpine/tree/dev

• Pagespeed 1.13.35.2 (mod and ngx)
• Builds on Alpine 3.7 and edge
• Works with Nginx 1.12.2 and 1.13.8
• Drop in replacement for nginx:alpine image. It should be exactly the same experience, making migration seamless. Butthis one contains a newer LibreSSL, Alpine, and of course Pagespeed.

[tobsch]

@nberlee great thing! do you have an idea how to convince pagespeed to link libpng statically? In my case, I get trouble with other libraries requiring a newer version later

[nberlee]

@tobsch Are you having problems with my Dockerfile as-is? If this is the case I would like to know.

If you only want libpng 1.6 (which i would understand), @ashishk-1 made a commit recently in incubator-pagespeed-mod:master which would make it possible. You can use patch (I haven't tested it) https://github.com/apache/incubator-pagespeed-mod/commit/7ea8d5fc141c7067c54ffa2b9fbc552c15089ca6.patch (just save it with the other patches in my branch)

Make sure to add libpng-dev in the apk add section of modpagespeed This patch will not be incorporated in the Dockerfile sine its not part of 1.13.35.2 and I am not backporting when its not absolutely necessary

[tobsch]

@nberlee it's not "as it is" but in combination with php & such. thanks a lot for the png patch tip: works out, after modifying the patch a bit (does not work with 1.13.35-2). Much better than a static version of libpng12!

[oschaaf]

@tobsch could you perhaps share the tweaked patch for 1.13?

[tobsch]

@oschaaf here you go: https://gist.github.com/tobsch/c59b0c9b89f51a412a57c2bf8a4cbb31

I had to change the location of zlib.h, as it is now a submodule.

[tobsch]

@oschaaf maybe you can help me with https://github.com/apache/incubator-pagespeed-ngx/issues/1535 in exchange :-) ? this prevents me from using ps in production currently

[oschaaf]

@tobsch Thanks for the patch! And sure, I will take a look, but it will be next week as I am on vacation this week. The required change is not super straight forward (I already took a quick stab at it) On Mon, 26 Feb 2018 at 08:01, Tobias Schlottke notifications@github.com wrote:

@oschaaf https://github.com/oschaaf maybe you can help me with #1535 https://github.com/apache/incubator-pagespeed-ngx/issues/1535 in exchange :-) ? this prevents me from using ps in production currently

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/apache/incubator-pagespeed-ngx/issues/1181#issuecomment-368408109, or mute the thread https://github.com/notifications/unsubscribe-auth/ACIsRKDqZwX0p3VfRbl-9FAWGwk4LBFlks5tYlbQgaJpZM4IN6ge .

[tspicer]

curious if there has been any progress. seems like things were close but got stuck.

[nberlee]

https://github.com/We-Amp/ngx-pagespeed-alpine All changes of dev are now merged in master. I was running dev in production for some time.... (also with nginx 1.14.0

I will contact @oschaaf to discuss to create something on Dockerhub..

[oschaaf]

@nberlee can you ping me at oschaaf@we-amp.com?

[tspicer]

Any update? Also, will these changes be packaged into the base ngx build or is the model to provide a docker image with patches? Would prefer a stable ngx alpine build without docker wrapper.

[nberlee]

https://hub.docker.com/r/pagespeed/nginx-pagespeed/

[tspicer]

@nberlee so the solution is to use the actual docker build and all the associated patches from https://github.com/We-Amp/ngx-pagespeed-alpine rather than getting a ngx_pagespeed that can be used outside of that setup? In the case of someone having their own nginx docker build process or using an alternate project, we will need to disassemble the docker file and assets from We-Amp to understand how to get this functional. Am understanding that correctly?

[nberlee]

@tspicer Your aim was to have an apk in the alpine store if I read your comment correctly? I think that would not be to hard to create or get it accepted in the alpine-community repo, so feel free to do so. It is not uncommon for packages in the alpine repository to have patches because of musl. I don't expect from the ngx_pagespeed to incorporate the patches, but feel free to create PR's to ngx_pagespeed ot make it possible to build it by default on alpine without sacrificing libc distributions.

I am sorry I raised your expectation I would make or maintain such a thing. However, as docker is build on layers you can easily add stuff to the compiled docker image.

FROM pagespeed/nginx-pagespeed:stable
apk add --no-cache <my-packages>

[oschaaf]

@tspicer @nberlee So our initial goals with the patches was to incorporate them into mod_pagespeed, but priorities changed and we didn't wrap up. But actually some of the more complex changes already actually landed to make building possible/easier for Alpine (& FreeBSD) (e.g: upgrades for grpc / libpng).

Looking at them, the remaining patches seem pretty small and mundane. One of them would have to be upstreamed to Chromium, the others can probably be merged into mod_pagespeed without any problems --- except for the Makefile patch, which may need some consideration. I can't promise any timelines, but I'll see what I can do, hopefully in the next couple of weeks, to get the required level of patching for Alpine down to 0. One concern I have going forward though is that we Alpine is not tested in CI.

[tspicer]

@nberlee not the APK store, but just being able to compile as I would for another OS. At the moment the process to install is tightly coupled into a specific Dockerfile. If you are compiling from nginx source with alternate modules/packages and libraries, your choice is to 1) copy the docker file as provided and modify the nginx build as applicable, then customize any other parts that are warranted or 2) deconstruct the Dockerfile, grab the patches and attempt to incorporate into your own build.

So no expectations were set by you :) I just want to make sure what you have referenced is the fix and in basically closes this issue.

thanks @oschaaf for clarifying

[csandanov]

PSOL compilation takes way too much time (~35 min on travis), so I moved it out to a separate repository before the official release become available and just download the archive (100M of objects required for nginx module compilation) during nginx build, see example. Useful if you maintain your own alpine-based nginx image and want to add pagespeed module.

[nberlee]

@csandanov If you would implement docker layer caching in travis (which is ok, with a tagged versions in a multi-stage docker file), builds would fly if you change something on the nginx.

[oschaaf]

Accidentally closed this while merging https://github.com/apache/incubator-pagespeed-ngx/pull/1598. Re-opening this, while the bar has been substantially lowered, this isn't completely done yet.