ppggff / vagrant-qemu

Use Vagrant to manage machines using QEMU. Test with Apple Silicon / M1 and CentOS aarch64 image
MIT License
424 stars 33 forks source link

How did you create the centos7 image? #15

Closed itinneed2022 closed 2 years ago

itinneed2022 commented 2 years ago

I need to setup a Fedora 35 or 36 (or equivalent RHEL7/8) image for testing. How did you guys create your centos7 box for this plugin?

ppggff commented 2 years ago

The basic steps are:

  1. Build a kernel with 4K page size to be compatible with apple m1 (CentOS 7 also need a small patch, it's based on kernel commits "Revert "arm64: Remove unnecessary ISBs from set_{pte, pmd, pud}")
  2. Install this "new" kernel to the official cloud image
  3. Turn the altered image to a vagrant box

Sorry I didn't remember so many details, there are some notes, hope it will help you.

  1. Build kernel:
    • I build it on a x86 machine, with qemu-static + binfmt-misc (do you want more info about this?) this could be simplified by doing it on a arm vm, or apple m1 vm without 'accel=hvf'

At last is the recored script about compile the kernel, it may not complete, use it as a hint.

# prepare
yum groupinstall "Development Tools"
yum install ncurses-devel
yum install hmaccalc zlib-devel binutils-devel elfutils-libelf-devel
yum install asciidoc audit-libs-devel bash bc binutils binutils-devel bison diffutils elfutils
yum install elfutils-devel elfutils-libelf-devel findutils flex gawk gcc gettext gzip hmaccalc hostname java-devel
yum install m4 make module-init-tools ncurses-devel net-tools newt-devel numactl-devel openssl
yum install patch pciutils-devel perl perl-ExtUtils-Embed pesign python-devel python-docutils redhat-rpm-config
yum install rpm-build sh-utils tar xmlto xz zlib-devel
yum install libcap-devel python3-devel devtoolset-8-build devtoolset-8-binutils devtoolset-8-gcc devtoolset-8-make python3-rpm-macros dwarves libbpf-devel libbabeltrace-devel libcap-ng-devel llvm-toolset-7.0 openssl-devel
yum install centos-release-scl
yum install devtoolset-8
yum install llvm-toolset-7.0
yum install wget
yum install libarchive
yum install popt-devel
yum install vim
yum install linux-firmware

# prepare
su - test
scl enable devtoolset-8 bash
scl enable llvm-toolset-7.0 bash
export PATH=$PATH:/sbin/

curl -O https://ftp.gnu.org/gnu/bash/bash-5.0.tar.gz
./configure
make
make install # with root

wget https://cbs.centos.org/kojifiles/packages/rpm/4.13.0.2/1.el7.c8/aarch64/python2-rpm-4.13.0.2-1.el7.c8.aarch64.rpm
wget https://cbs.centos.org/kojifiles/packages/rpm/4.13.0.2/1.el7.c8/aarch64/rpm-build-4.13.0.2-1.el7.c8.aarch64.rpm
wget https://cbs.centos.org/kojifiles/packages/rpm/4.13.0.2/1.el7.c8/aarch64/rpm-devel-4.13.0.2-1.el7.c8.aarch64.rpm
wget https://cbs.centos.org/kojifiles/packages/rpm/4.13.0.2/1.el7.c8/aarch64/rpm-build-libs-4.13.0.2-1.el7.c8.aarch64.rpm
wget https://cbs.centos.org/kojifiles/packages/rpm/4.13.0.2/1.el7.c8/aarch64/rpm-libs-4.13.0.2-1.el7.c8.aarch64.rpm
wget https://cbs.centos.org/kojifiles/packages/rpm/4.13.0.2/1.el7.c8/aarch64/rpm-plugin-ima-4.13.0.2-1.el7.c8.aarch64.rpm
wget https://cbs.centos.org/kojifiles/packages/rpm/4.13.0.2/1.el7.c8/aarch64/rpm-plugin-selinux-4.13.0.2-1.el7.c8.aarch64.rpm
wget https://cbs.centos.org/kojifiles/packages/rpm/4.13.0.2/1.el7.c8/aarch64/rpm-plugin-syslog-4.13.0.2-1.el7.c8.aarch64.rpm
wget https://cbs.centos.org/kojifiles/packages/rpm/4.13.0.2/1.el7.c8/aarch64/rpm-plugin-systemd-inhibit-4.13.0.2-1.el7.c8.aarch64.rpm
wget https://cbs.centos.org/kojifiles/packages/rpm/4.13.0.2/1.el7.c8/aarch64/rpm-sign-4.13.0.2-1.el7.c8.aarch64.rpm
wget https://cbs.centos.org/kojifiles/packages/rpm/4.13.0.2/1.el7.c8/aarch64/rpm-debuginfo-4.13.0.2-1.el7.c8.aarch64.rpm
rpm -Uvh *.rpm # with root

# prepare source code
mkdir -p ~/rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
echo '%_topdir %(echo $HOME)/rpmbuild' > ~/.rpmmacros

rpm -i kernel-4.18.0-305.10.2.el7.src.rpm 2>&1 | grep -v 'exist'
cd ~/rpmbuild/SPECS
rpmbuild -bp --target=$(uname -m) kernel.spec

# configure the kernel
cd ~/rpmbuild/BUILD/kernel-*/linux-*/
cp configs/kernel-4.18.0-aarch64.config .config

make oldconfig
make menuconfig # "Kernel Features" -> "Page size (64KB)" -> "4KB"
# add to the top of `.config`: # arm64

cp .config configs/kernel-4.18.0-aarch64.config
cp configs/* ~/rpmbuild/SOURCES/

# complie
cd ~/rpmbuild/SPECS/
cp kernel.spec kernel.spec.distro
vi kernel.spec

# fix a minor problem in the spec file
42c42
< # define buildid .local
---
> %define buildid .gzq4K
1216,1218c1216,1218
< cp $RPM_SOURCE_DIR/kernel-*.config .
< cp %{SOURCE41} .
< VERSION=%{version} NAME=%{name} ./generate_all_configs.sh
---
> cp $RPM_SOURCE_DIR/kernel-%{version}-*.config .
> #cp %{SOURCE41} .
> #VERSION=%{version} NAME=%{name} ./generate_all_configs.sh

rpmbuild --with baseonly --without debug --without debuginfo --without kabichk -bb --target=`uname -m` kernel.spec 2> build-err.log | tee build-out.log

# install the kernel
cd /home/test/rpmbuild/RPMS/aarch64/
yum localinstall kernel-*.rpm # with root

# check
grub2-editenv list
leifliddy commented 1 year ago

I built a 4k kernel today. These are quick + dirty instructions on how I did that.

You don't need install rpm >= 4.13.0.1-19 Unless you're building the kernel with debuginfo

%if %{with_debuginfo}
BuildRequires: rpm-build, elfutils
BuildConflicts: rpm < 4.13.0.1-19
yum install centos-release-scl

yumdownloader --source kernel

rpm --ivh kernel-4.18.0-348.20.1.el7.src.rpm

cd /root/rpmbuild/SPECS

# I built these from the opencsd el8 SRPM
rpm -ivh https://leifliddy.com/epel/7/aarch64/opencsd-1.0.0-4.el7.aarch64.rpm
rpm -ivh https://leifliddy.com/epel/7/aarch64/opencsd-devel-1.0.0-4.el7.aarch64.rpm

yum-builddep kernel.spec

rpmbuild -bp kernel.spec

cd /root/rpmbuild/BUILD/kernel-4.18.0-348.20.1.el8_5/linux-4.18.0-348.20.1.4k.el7.aarch64
cp configs/kernel-4.18.0-aarch64.config .config

make menuconfig   (if cross compiling then specify the arch: make ARCH=arm64 menuconfig)

Kernel Features --> Page Size --> select 4KB
[ save ]
vi .config 
--insert this verbatim on the first line of this file
# arm64

cp .config ~/rpmbuild/SOURCES/kernel-aarch64.config

cd /root/rpmbuild/SPECS

# this command "should" work but I haven't tested it 
rpmbuild --with baseonly --without debug --without debuginfo -bb kernel.spec

# This is because I modified the kernel.spec file to set the options I wanted 
# With my srpm + spec file, I just need to run:
rpmbuild -bb kernel.spec

I also rebased that patch: https://patchwork.kernel.org/project/linux-arm-kernel/patch/20190827131818.14724-2-will@kernel.org/

You can find my el7 4k kernel rpms here: https://leifliddy.com/epel/7/aarch64/

And the SRPM here: https://leifliddy.com/epel/7/SRPMS/

Thanks!! @ppggff

update: The 4.19 kernel in the link above has virtio-9p support

leifliddy commented 1 year ago

In case anyone's interested -- I created a centos7 boot.iso that contains the 4k kernel here: https://leifliddy.com/epel/7/images/

update: I just discovered this repo which contains lts (4k) kernels: http://mirror.centos.org/altarch/7/kernel/x86_64/Packages/