Tracker for Build issues

[wonglkd]

Build status

Clicking on a badge will show you the recent builds for that OS. If your target OS's build is failing, you may wish to check recent issues and PRs for known workarounds.

• 
• 
• 
• 
• 
• 
• 
• 
• 

Issues

• [x] Debian (Docker image), all (CMake) - #200
• [x] CentOS 8.1, 8.5, RockyLinux-8.6 (XDCHECK) - #201
• [x] Fedora, Rocky, Arch (numa deps) - #197
• [x] all (fmt) - #196

1. Debian fails at stage update-submodules. fatal: detected dubious ownership in repository at '/__w/CacheLib/CacheLib'

• See https://github.com/actions/runner-images/issues/6775, https://github.com/actions/checkout/issues/1169
• Verified in our builds that /usr/bin/git config --global --add safe.directory /__w/CacheLib/CacheLib is run
• Workaround available: Run - run: git config --system --add safe.directory /__w/CacheLib/CacheLib after https://github.com/facebook/CacheLib/blob/6ad7a318b43de25ba41067c3d6a851d5a60d1633/.github/workflows/build-cachelib-debian-10.yml#L53

2. CentOS fails CacheLib build in XDCHECK

/__w/CacheLib/CacheLib/cachelib/../cachelib/cachebench/runner/AsyncCacheStressor.h:306:7: internal compiler error: in cp_build_addr_expr_1, at cp/typeck.c:5965
       XDCHECK(req->sizeBegin + 1 != req->sizeEnd);

https://github.com/facebook/CacheLib/blob/6ad7a318b43de25ba41067c3d6a851d5a60d1633/cachelib/cachebench/runner/AsyncCacheStressor.h#L306

• ~Still figuring this out.~
• Tried but didn't work: req->sizeBegin with std::next(req->sizeBegin), replacing XDCHECK with XDCHECK_NEQ, XCHECK
• The build last passed on Jan 3, then broke due to fmt/folly. AsyncCacheStressor.h itself hasn't changed in 2023.
• Figured that all the failing builds used GCC-8.5, while the others used other versions (7.5, 9.4, 11.3.1).
• ~Didn't see any relevant issues/commits on the folly repo.~ Definition of XDCHECK. Two commits since last passing build and now: https://github.com/facebook/folly/commit/1aafad45f316896a7504396f421dacd6c10d7d5f and https://github.com/facebook/folly/commit/e6d09f66b9fc473bc108361d4c8dce8f29f7bcaf (gcc-8.x internal compiler error)
• There is a similar line in question in CacheStressor.h. So the main difference is that this is in a lambda, which could be the cause of the issue. Suspecting it might be a folly regression.
• Commenting out AsyncCacheStressor.h#L306 fixed the build. Planning to submit a fix that gates it on compiler version and replace XDCHECK with an assert() for gcc-8.5. https://github.com/wonglkd/CacheLib-1/blob/ab175a207215ca58ac9936db72f519af69a0cf20/cachelib/cachebench/runner/AsyncCacheStressor.h#L307-L314. Gating style adapted from https://github.com/facebook/folly/commit/e6d09f66b9fc473bc108361d4c8dce8f29f7bcaf

3. RPM builds

• https://pagure.io/meta/cachelib/blob/packit/f/cachelib.spec
• Probably the above needs to be updated with the right packages, including numactl-devel
• https://src.fedoraproject.org/rpms/cachelib/pull-request/1#request_diff
• Not sure if it needs to be updated here too
• Builds seem to be failing due to dependency issues: see https://download.copr.fedorainfracloud.org/results/packit/facebook-CacheLib-201/fedora-rawhide-x86_64/05579210-cachelib/builder-live.log.gz

[wonglkd]

All builds are passing for now (except rpm builds). Could put the status badges (which filter on scheduled builds) with the build instructions or even the readme:

• https://github.com/facebook/CacheLib/blob/main/BUILD.md
• https://github.com/facebook/CacheLib/blob/main/website/docs/installation/installation.md
• https://github.com/facebook/CacheLib/blob/main/README.md (if it doesn't make it too bloated)

## Build status

Clicking on a badge will show you the recent builds for that OS. If your target OS's build is failing, you may wish to check recent issues and PRs for known workarounds.

- [![CentOS 8.1](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-centos-8-1.yml/badge.svg?event=schedule)](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-centos-8-1.yml?query=event%3Aschedule)
- [![CentOS 8.5](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-centos-8-5.yml/badge.svg?event=schedule)](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-centos-8-5.yml?query=event%3Aschedule)
- [![Debian 10](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-debian-10.yml/badge.svg?event=schedule)](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-debian-10.yml?query=event%3Aschedule)
- [![Fedora 36](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-fedora-36.yml/badge.svg?event=schedule)](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-fedora-36.yml?query=event%3Aschedule)
- [![Rocky Linux 8](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-rockylinux-8.yml/badge.svg?event=schedule)](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-rockylinux-8.yml?query=event%3Aschedule)
- [![Rocky Linux 9](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-rockylinux-9.yml/badge.svg?event=schedule)](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-rockylinux-9.yml?query=event%3Aschedule)
- [![Ubuntu 18](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-ubuntu-18.yml/badge.svg?event=schedule)](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-ubuntu-18.yml?query=event%3Aschedule)
- [![Ubuntu 20](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-ubuntu-20.yml/badge.svg?event=schedule)](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-ubuntu-20.yml?query=event%3Aschedule)
- [![Ubuntu 22](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-ubuntu-22.yml/badge.svg?event=schedule)](https://github.com/facebook/cachelib/actions/workflows/build-cachelib-ubuntu-22.yml?query=event%3Aschedule)

[wonglkd]

@jaesoo-fb for your consideration (re build badges, above, if you want to put them somewhere)

closing this issue out for now, since the rpm-build issues are being handled internally. Thanks @jaesoo-fb @jiayuebao @therealgymmy for working with me on the PRs and reviewing them!

[jaesoo-fb]

@wonglkd Thanks for working on this; the build looks all green now. For the build badges, well, having said that we can see the status of all build actions in the actions tab, not sure whether it would be useful somewhere else too (yeah, it is fancy though!!) Let us discuss internally.

[wonglkd]

@jaesoo-fb Sounds good, yup, a matter for y'all to discuss. (The main reason I found it useful was that I couldn't see at one glance the status of all the builds on the tab, as for some OSes, we alternate which version is scheduled to run every day, and (potentially red) builds from PRs can quickly clutter up the actions tab.)

[jaesoo-fb]

Thanks @wonglkd for the suggestion and the code snippet. I just landed the commit e679b72d5b78 (README.md: added build status badges to main README.md) which adds the build status to the bottom of main README file https://github.com/facebook/CacheLib