GLIBC error when deploying version 0.14.1+ to Vercel or CloudFlare

[dbanty]

Bug Report

Environment

Zola version: 0.15.2

Expected Behavior

Zola sites can deploy with the framework presets on the documented providers. I don't actually know if this is a thing the maintainers of Zola can fix or if this needs to be an issue opened with each provider.

Current Behavior

I have tried deploying both to Vercel and CloudFlare and get similar issues. Here is the output on CloudFlare:

15:26:35.839 | Executing user command: zola build
-- | --
15:26:35.842 | zola: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.27' not found (required by zola)
15:26:35.842 | zola: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by zola)
15:26:35.842 | zola: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by zola)
15:26:35.843 | Failed: build command exited with code: 1

And on Vercel:

Installing Zola version 0.14.1
--
15:15:23.499 | zola: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by zola)
15:15:23.500 | zola: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by zola)
15:15:23.500 | zola: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by zola)
15:15:23.503 | Error: Command "zola build" exited with 1

Note that I tried on the latest 0.15.2 on both platforms as well as the older 0.14.1 on Vercel and got the same results. 0.14.0 seems to build on both, though I'm going to have to adjust my source to work with that.

Step to reproduce

Follow the guide to create a basic Zola site, then deploy to Vercel or CloudFlare using version 0.14.1 or newer. For example, follow this guide and set the ZOLA_VERSION environment variable to 0.15.2.

[flother]

I saw the same error this evening while trying out Cloudflare Pages. I used Zola v0.15.2 by setting the ZOLA_VERSION=0.15.2 environment variable. Here's the full build log.

21:23:19.222    Initializing build environment. This may take up to a few minutes to complete
21:25:23.045    Success: Finished initializing build environment
21:25:23.045    Cloning repository...
21:25:26.528    Success: Finished cloning repository files
21:25:26.723    Installing dependencies
21:25:26.725    Python version set to 2.7
21:25:27.794    v12.18.0 is already installed.
21:25:28.252    Now using node v12.18.0 (npm v6.14.4)
21:25:28.295    Started restoring cached build plugins
21:25:28.299    Finished restoring cached build plugins
21:25:28.416    Attempting ruby version 2.7.1, read from environment
21:25:29.370    Using ruby version 2.7.1
21:25:29.612    Using PHP version 5.6
21:25:29.643    5.2 is already installed.
21:25:29.648    Using Swift version 5.2
21:25:29.649    Installing Hugo 0.54.0
21:25:30.292    Hugo Static Site Generator v0.54.0-B1A82C61A/extended linux/amd64 BuildDate: 2019-02-01T10:04:38Z
21:25:30.293    Installing Zola 0.15.2
21:25:31.624    Started restoring cached go cache
21:25:31.628    Finished restoring cached go cache
21:25:31.650    go version go1.14.4 linux/amd64
21:25:31.655    go version go1.14.4 linux/amd64
21:25:31.656    Installing missing commands
21:25:31.656    Verify run directory
21:25:31.656    Executing user command: zola build
21:25:31.657    zola: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.27' not found (required by zola)
21:25:31.657    zola: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by zola)
21:25:31.657    zola: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by zola)
21:25:31.658    Failed: build command exited with code: 1

Several people on the Cloudflare community site have had the same problem. No resolution yet.

[Keats]

Hmm must be one of the updated dependencies but not sure which. Looking at the thread it seems to happen since 0.14 so that's been that way a long time 🤔

[flother]

From the thread:

This is a known issue with the 0.14.x builds of Zola. 0.13.x builds should work fine. I should re-poke the team about this issue

Not sure who it is that knows about the issue though. Someone in the Cloudflare dev team?

[dbanty]

I opened https://github.com/cloudflare/cloudflare-docs/issues/3043 to request the max Zola version be documented there. I also commented that it'd be nice to get the update but I'm not sure how to request that.

No idea either where to ask for the Vercel update. I'd assume that Netlify has a similar issue but I haven't tried yet.

[mikekasprzak]

Has anyone figured out how to contact the CloudFlare pages team? We're 6 months and 6 versions behind now.

[sqwxl]

I can confirm the same issue happens on Netlify. Rolling back to 0.14.0 was necessary there as well.

[Keats]

Have you tried using Netlify latest image? I deployed my personal site with it and 0.15.3 yesterday

[HanKruiger]

Thanks @Keats, I can confirm that building with the newer image on Netlify solved this problem for me! :rocket:

To change the build image in Netlify, go to Deploy settings -> Build image selection, and select the newer one (Ubuntu Focal).

[Keats]

I need to update the docs, it was failing as well for me 16.04

[sqwxl]

Thanks @Keats bumping the image to latest fixed it for me too 🎉

[mamcx]

How about build zola with musl? I have good success with my projects using https://github.com/messense/cargo-zigbuild

[Keats]

Pretty sure libsass (and potentially other libs) are not going to work on musl

[btwiuse]

Vercel's builders run in an amazonlinux container, which uses outdated glibc. I tried to build zola 0.15.3 from source in that environment, and it works fine.

Here is my compiled zola binary for vercel environment. You can also build it yourself if you can't trust it.

[mamcx]

Thanks this work for now!

[doamatto]

Pretty sure libsass (and potentially other libs) are not going to work on musl

This should be a non-issue, actually. Zola is in the community repos on Alpine Linux, which musl, and has worked fine for me in the past. The package is even for 0.15.3 on all architectures.

[bemyak]

I can confirm that building with musl just works. However, as @Keats pointed out here it's probably not a good idea to ship a binary build with musl.

The issue is caused by this commit. The simplest fix would be to revert the base image back to ubuntu-16.04.

[Keats]

ubuntu-16.04 is not supported anymore on Azure pipelines

[dstaley]

@Keats Azure Pipelines supports running jobs in containers with a small modification to the workflow syntax. As explained in this blog post (opperating under the assumption that GitHub Actions and Azure Pipelines are two slightly different veneers over the same infrastructure), this can allow you to build a binary release of Zola that supports a wider range of glibc versions by using a container that ships with an older version of glibc. You could use a container image of ubuntu:16.04, amazon:2 (for AWS Lambda and Vercel compatibility), or debian:stretch.

[Keats]

That wouldn't work with the current rustc version required by Zola afaik?

[dstaley]

@Keats I think it does!

I created builds using container: amazonlinux:2 and the same rustup invocations used in the Azure Pipelines workflow; it built successfully. Additionally, I dumped the GLIBC symbols from the binary and confirmed the highest usage was 2.25. This was on the master branch, but I also confirmed the same on the next branch.

[Keats]

Has it been updated? See https://github.com/getzola/zola/pull/2179#issuecomment-1511205022

[dstaley]

@Keats Vercel has not been updated, and continues to use glibc 2.26 (which is the same glibc version currently used by AWS Lambda, the infrastructure provider for Vercel's build system).

> ldd --version && node --experimental-wasi-unstable-preview1 build.mjs
ldd (GNU libc) 2.26

The v0.17.2 binary of zola contains references to 2.27, 2.28, and 2.29.

$ objdump -T zola | grep GLIBC | sed 's/.*GLIBC_\([.0-9]*\).*/\1/g' | sort -Vu
2.2.5
2.3
2.3.2
2.3.4
2.4
2.6
2.7
2.9
2.10
2.12
2.14
2.15
2.17
2.18
2.25
2.27
2.28
2.29

[hannobraun]

Cloudflare Pages is working now. I just successfully deployed a page using Zola 0.17.2.

[Keats]

Ok I'm going to close this issue then since there's nothing actionable on my end and it will work again automatically when providers upgrade their images.

[mikekasprzak]

Thank you @hannobraun for letting us know, and thank you @Keats for everything you do!

[wewlad-dev]

Is this solved on Vercel? if not? how to? Or should I look for netifly or cloudflare instead?

zola: /lib64/libm.so.6: version GLIBC_2.27 not found (required by zola) zola: /lib64/libm.so.6: version GLIBC_2.29 not found (required by zola) zola: /lib64/libc.so.6: version GLIBC_2.28 not found (required by zola) Error: Command "zola build" exited with 1

[elchead]

Is this solved on Vercel? if not? how to? Or should I look for netifly or cloudflare instead?

zola: /lib64/libm.so.6: version GLIBC_2.27 not found (required by zola) zola: /lib64/libm.so.6: version GLIBC_2.29 not found (required by zola) zola: /lib64/libc.so.6: version GLIBC_2.28 not found (required by zola) Error: Command "zola build" exited with 1

netlify works, vercel is still broken

[mjovanc]

Vercel is using 0.13.0, I hope they will update to at least 0.17.2. I run into problems compiling sass using the older version...

[dstaley]

Very pleased to say that Zola 0.17.2 works wonderfully on Vercel by updating the Node.js version in settings to v20.

[21:26:52.606] Running build in Washington, D.C., USA (East) – iad1 (Hive)
[21:26:52.699] Cloning github.com/dstaley/dstaley.com (Branch: node-v20-zola, Commit: 65c5bf3)
[21:26:53.321] Previous build caches not available
[21:26:53.384] Cloning completed: 683.855ms
[21:26:53.608] Running "vercel build"
[21:26:54.099] Vercel CLI 32.5.5
[21:26:54.733] Installing Zola version 0.17.2
[21:26:55.237] Building site...
[21:26:55.311] Checking all internal links with anchors.
[21:26:55.311] > Successfully checked 0 internal link(s) with anchors.
[21:26:55.312] -> Creating 9 pages (0 orphan) and 1 sections
[21:26:56.735] Done in 1.5s.
[21:26:56.735] 
[21:26:56.758] Build Completed in /vercel/output [2s]
[21:26:56.799] Deploying outputs...
[21:26:57.435] Injected preview comments middleware, you can disable this in your project settings.
[21:26:57.983] 
[21:26:58.262] Deployment completed
[21:26:59.198] Uploading build cache [29.00 B]...
[21:26:59.594] Build cache uploaded: 395.685ms