Is there a menuconfig for Photon OS 4.0 Rev2 to recompile the kernel?

JonathanVQP commented 1 year ago

Is your feature request related to a problem? Please describe.

There is no problem. Just wondering if one can recompile the Photon OS 4.0 Rev 2 kernel. There doesn't appear to be a menuconfig.

Describe the solution you'd like

To be able to recompile the Photon OS kernel.

Describe alternatives you've considered

No response

Additional context

No response

sshedi commented 1 year ago

Yes you can. Download the kernel src.rpm from https://packages.vmware.com/photon/4.0/photon_srpms_4.0_x86_64/ and get started.

dcasota commented 1 year ago

+1

@sshedi please provide an instruction recipe on how kernel option modification should be repackaged. How to live test e.g. the following Ph4 rev2 kernel header source change? How to repackage it smart with Photon OS coding guidelines for versioning, sha512, etc.?

# modify CONFIG_MEDIA_SUPPORT
cp /usr/src/linux-headers-5.10.168-4.ph4-secure/.config /usr/src/linux-headers-5.10.168-4.ph4-secure/.config.0
sed "s/# CONFIG_MEDIA_SUPPORT is not set/CONFIG_MEDIA_SUPPORT=y/" /usr/src/linux-headers-5.10.168-4.ph4-secure/.config.0 > /usr/src/linux-headers-5.10.168-4.ph4-secure/.config

As Linux + Photon OS beginner in 2019, in 2022 I've promised in this thread to come back with a functioning example, but I stuck in #1370. Thanks alot!

JonathanVQP commented 1 year ago

@sshedi I downloaded and installed linux-5.10.168-4.ph4.src.rpm but no menuconfig. Please advise.

dcasota commented 1 year ago

Hi @sshedi,

Yes you can. Download the kernel src.rpm from https://packages.vmware.com/photon/4.0/photon_srpms_4.0_x86_64/ and get started.

Can you provide an example about "and get started" ? The following make menuconfig snippet is for the Linux kernel' menuconfig (Photon 5).

tdnf install -y wget tar build-essential ncurses ncurses-devel
wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.1.10.tar.gz
tar xzvf linux-6.1.10.tar.gz
cd linux-6.1.10
make menuconfig

However, to recompile the Photon OS 4.0 Rev 2 kernel, how to get started e.g. with https://packages.vmware.com/photon/4.0/photon_srpms_4.0_x86_64/linux-5.10.168-4.ph4.src.rpm ? Any quick win?

sshedi commented 1 year ago

tdnf install -y wget rpm-build
mkdir -p linux-build && cd linux-build
mkdir RPMS SOURCES
cd SOURCES
rpm2cpio https://packages.vmware.com/photon/4.0/photon_srpms_4.0_x86_64/linux-5.10.168-4.ph4.src.rpm | cpio -idm
cd ..
rpmbuild -bb SOURCES/linux.spec -D '%_topdir .'

You can modify the SOURCES/config_x86_64 as needed.

JonathanVQP commented 1 year ago

@dcasota Thanks. It works! As of today, I see the latest kernel is 5.10.176. Can I configure it to work with Photon OS 4 Rev2 or does it need to have special or specific changes since I see that the Ph4 kernels end with a .ph4 suffix?

dcasota commented 1 year ago

Hi @JonathanVQP, Actually 5.10.168 is available for public download:

for a VMware Hypervisor-optimized installation of x86_64 Photon OS 4, download https://packages.vmware.com/photon/4.0/photon_srpms_4.0_x86_64/linux-esx-5.10.168-2.ph4.src.rpm
for a generic installation of x86_64 Photon OS 4, download https://packages.vmware.com/photon/4.0/photon_srpms_4.0_x86_64/linux-5.10.168-4.ph4.src.rpm

During provisioning of minimal or full iso, the built-in Photon OS installer asks about [ generic / VMware Hypervisor optimized ] selection.

The VMware team publishes newer kernel files quite frequently, but dependencies safety first.

@sshedi thanks for sharing! Learning alot.

JonathanVQP commented 1 year ago

@sshedi I couldn't get it to work. Probably because of the directory structure.

dcasota commented 1 year ago

@sshedi rpmbuild creates a temporary script file e.g. rpmbuild_tmpscript.txt. There are multiple lines with

# Using autosetup is not feasible
cd '/BUILD'

If I've understood the script, each of those lines could be changed to cd './BUILD to make it work, right? Now this is out of the context, so it has to be verified in the source which creates the temporary file.

sshedi commented 1 year ago

Please create those dirs.

mkdir -p $PWD/{RPMS,SRPMS,SOURCES,SPECS,LOGS,BUILD,BUILDROOT}

dcasota commented 1 year ago

@sshedi rpmbuild_user_output, this one fails at the end rpmbuild-photon-minimal-4.0-c001795b8_user_output, this one fails with a poor permission workaround rpmbuild-photon-minimal-4.0-c001795b8_user_output_withfailedworkaround

aroodgar commented 7 months ago

I got the same errors as pointed out by @dcasota but the following workaround worked for me. (I tested for linux-5.10.194-2.ph4.src.rpm.) In @sshedi 's suggestion, instead of naming the root directory linux-build, simply name it rpmbuild and remove the -D '%_topdir .' option from the last command as the default root directory is now considered to be ~/rpmbuild. Important note: The rpmbuild directory should be in your home directory.

This is the final look of the stages after these changes:

tdnf install -y wget rpm-build
mkdir  ~/rpmbuild && cd ~/rpmbuild
mkdir -p $PWD/{RPMS,SRPMS,SOURCES,SPECS,LOGS,BUILD,BUILDROOT}
cd SOURCES
rpm2cpio https://packages.vmware.com/photon/4.0/photon_srpms_4.0_x86_64/linux-5.10.194-2.ph4.src.rpm | cpio -idm
cd ..
cp SOURCES/linux.spec SPECS/linux.spec
rpmbuild -bb SPECS/linux.spec

oliverkurth commented 7 months ago

In Photon 5.0 it's much easier:

tdnf --enablerepo=photon-srpms --source install linux
tdnf --enablerepo=photon-srpms --builddeps install linux
tdnf install build-essential rpm-build
rpmbuild -bb --define "dist .ph5" /usr/src/photon/SPECS/linux.spec

Files will be in /usr/src/photon/.

Should we backport the source/builddeps feature back to 4.0?

aroodgar commented 7 months ago

I got the same errors as pointed out by @dcasota but the following workaround worked for me. (I tested for linux-5.10.194-2.ph4.src.rpm.) In @sshedi 's suggestion, instead of naming the root directory linux-build, simply name it rpmbuild and remove the -D '%_topdir .' option from the last command as the default root directory is now considered to be ~/rpmbuild. Important note: The rpmbuild directory should be in your home directory.

This is the final look of the stages after these changes:
tdnf install -y wget rpm-build
mkdir  ~/rpmbuild && cd ~/rpmbuild
mkdir -p $PWD/{RPMS,SRPMS,SOURCES,SPECS,LOGS,BUILD,BUILDROOT}
cd SOURCES
rpm2cpio https://packages.vmware.com/photon/4.0/photon_srpms_4.0_x86_64/linux-5.10.194-2.ph4.src.rpm | cpio -idm
cd ..
cp SOURCES/linux.spec SPECS/linux.spec
rpmbuild -bb SPECS/linux.spec

After I did this, I wanted to use menuconfig to change a few configs. I copied the new generated .config file into the SOURCES directory as the new config file and tried to get a new build with rpmbuild -bb SPECS/linux.spec. But I got this error:

+ echo 'Check for .config applicability'
Check for .config applicability
+ make LC_ALL= olddefconfig -j8
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  LEX     scripts/kconfig/lexer.lex.c
  YACC    scripts/kconfig/parser.tab.[ch]
  HOSTCC  scripts/kconfig/preprocess.o
  HOSTCC  scripts/kconfig/util.o
  HOSTCC  scripts/kconfig/symbol.o
  HOSTCC  scripts/kconfig/lexer.lex.o
  HOSTCC  scripts/kconfig/parser.tab.o
  HOSTLD  scripts/kconfig/conf
#
# No change to .config
#
+ sed -i 3d .config
+ diff -u .config.old .config
diff: .config.old: No such file or directory
error: Bad exit status from /var/tmp/rpm-tmp.GzncwC (%prep)

RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.GzncwC (%prep)

aroodgar commented 6 months ago

I got the same errors as pointed out by @dcasota but the following workaround worked for me. (I tested for linux-5.10.194-2.ph4.src.rpm.) In @sshedi 's suggestion, instead of naming the root directory linux-build, simply name it rpmbuild and remove the -D '%_topdir .' option from the last command as the default root directory is now considered to be ~/rpmbuild. Important note: The rpmbuild directory should be in your home directory. This is the final look of the stages after these changes:
tdnf install -y wget rpm-build
mkdir  ~/rpmbuild && cd ~/rpmbuild
mkdir -p $PWD/{RPMS,SRPMS,SOURCES,SPECS,LOGS,BUILD,BUILDROOT}
cd SOURCES
rpm2cpio https://packages.vmware.com/photon/4.0/photon_srpms_4.0_x86_64/linux-5.10.194-2.ph4.src.rpm | cpio -idm
cd ..
cp SOURCES/linux.spec SPECS/linux.spec
rpmbuild -bb SPECS/linux.spec
After I did this, I wanted to use menuconfig to change a few configs. I copied the new generated .config file into the SOURCES directory as the new config file and tried to get a new build with rpmbuild -bb SPECS/linux.spec. But I got this error:
+ echo 'Check for .config applicability'
Check for .config applicability
+ make LC_ALL= olddefconfig -j8
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  LEX     scripts/kconfig/lexer.lex.c
  YACC    scripts/kconfig/parser.tab.[ch]
  HOSTCC  scripts/kconfig/preprocess.o
  HOSTCC  scripts/kconfig/util.o
  HOSTCC  scripts/kconfig/symbol.o
  HOSTCC  scripts/kconfig/lexer.lex.o
  HOSTCC  scripts/kconfig/parser.tab.o
  HOSTLD  scripts/kconfig/conf
#
# No change to .config
#
+ sed -i 3d .config
+ diff -u .config.old .config
diff: .config.old: No such file or directory
error: Bad exit status from /var/tmp/rpm-tmp.GzncwC (%prep)

RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.GzncwC (%prep)

Does anyone have any thoughts on this?

dcasota commented 6 months ago

edited 03/18/2024 @aroodgar ~no news, stil weired error(s).~ seems to work now.

fyi actually, the visibility per package is weired in packages.vmware.com/photon/4.0/photon_srpms_4.0_x86_64/. For instance, the 5.10.210 srpm is downloadable but not visible.

Here the actual recipe.

# Naming scheme VMware By Broadcom ESXi optimized: linux-esx
# Naming scheme generic                          : linux

# variables
InstalledKernelReleaseName=""
KernelRelease=""
KernelReleaseNumber=""
DownloadUrl=""
NamingScheme=""

# get installed Linux Kernel Release
tdnf install -y awk
InstalledKernelReleaseName=`uname -a | awk '{print $3}'`
# 5.10.210-3.ph4-esx

KernelRelease=`tdnf list linux-api-headers | grep photon-updates | awk '{ print $2}'`
# 5.10.210-1.ph4
KernelReleaseNumber=`echo $KernelRelease | awk -F "-" '{print $1}'`
# 5.10.210

# Assemble DownloadUrl for Linux Kernel srpm
DownloadUrl=""
NamingScheme=""
if [[ $InstalledKernelReleaseName == *"-esx"* ]]; then
  DownloadUrl="https://packages.vmware.com/photon/4.0/photon_srpms_4.0_x86_64/linux-esx-$KernelRelease.src.rpm"
  NamingScheme="linux-esx"
else
  DownloadUrl="https://packages.vmware.com/photon/4.0/photon_srpms_4.0_x86_64/linux-$KernelRelease.src.rpm"
  NamingScheme="linux"
fi
# https://packages.vmware.com/photon/4.0/photon_srpms_4.0_x86_64/linux-esx-5.10.210-1.ph4.src.rpm

# Download and configure source rpm of Linux Kernel
tdnf install -y wget rpm-build 
cd /usr/local/src
mkdir -p $PWD/{RPMS,SRPMS,SOURCES,SPECS,LOGS,BUILD,BUILDROOT}
cd SOURCES
rpm2cpio $DownloadUrl | cpio -idm

# prepare for menuconfig
export TERM=linux
tdnf install -y tar build-essential ncurses ncurses-devel openssl-devel
tar xf linux-$KernelReleaseNumber.tar.xz
cd linux-$KernelReleaseNumber/
make clean

# start menuconfig and configure (enable CONFIG_MODULES!), save configuration and exit. 
make menuconfig
# or copy your own .config
# cp .config .config.old
# cp <your path>/.config .config

## make
chmod 777 .config
tdnf install -y elfutils-devel
make -j$(nproc) KCONFIG_CONFIG=.config
# if make fails, additional packages may have to be added to the tdnf command above (tdnf install -y elfutils-devel)
make modules_install -j$(nproc)
make install -j$(nproc)
cd ..

# exchange with a new tar
mv linux-$KernelReleaseNumber.tar.xz linux-$KernelReleaseNumber.tar.xz.old
tar -czf linux-$KernelReleaseNumber.tar.xz linux-$KernelReleaseNumber/*
rm -r -f linux-$KernelReleaseNumber/
rm linux-$KernelReleaseNumber.tar.xz.old
cd ..

## rpmbuild
cp SOURCES/$NamingScheme.spec SPECS/$NamingScheme.spec

# workaround for: rpmbuild fails with 
# `error: Unable to open /usr/src/photon/SOURCES/modify_kernel_configs.inc: No such file or directory` `error: line 469: Unclosed %if`
ln -s -d /usr/local/src /usr/src/photon

# solve package dependencies for SPECS/$NamingScheme.spec
# Linux-PAM-devel is needed by linux-esx-5.10.210-1.x86_64
#         elfutils-libelf-devel is needed by linux-esx-5.10.210-1.x86_64
#         gdb is needed by linux-esx-5.10.210-1.x86_64
#         glib-devel is needed by linux-esx-5.10.210-1.x86_64
#         kmod-devel is needed by linux-esx-5.10.210-1.x86_64
#         libdnet-devel is needed by linux-esx-5.10.210-1.x86_64
#         libmspack-devel is needed by linux-esx-5.10.210-1.x86_64
#         procps-ng-devel is needed by linux-esx-5.10.210-1.x86_64
#         xerces-c-devel is needed by linux-esx-5.10.210-1.x86_64
#         xml-security-c-devel is needed by linux-esx-5.10.210-1.x86_64
tdnf install -y linux-PAM-devel elfutils-libelf-devel gdb glib-devel kmod-devel libdnet-devel libmspack-devel procps-ng-devel xerces-c-devel xml-security-c-devel

rpmbuild -bb SPECS/$NamingScheme.spec

rpmbuild takes a while.

See output in /usr/local/src/RPMS/x86_64.

-rw-r----- 1 root root  15771428 Mar 18 13:10 linux-esx-5.10.210-1.x86_64.rpm
-rw-r----- 1 root root 365321243 Mar 18 13:10 linux-esx-debuginfo-5.10.210-1.x86_64.rpm
-rw-r----- 1 root root  13965485 Mar 18 13:10 linux-esx-devel-5.10.210-1.x86_64.rpm
-rw-r----- 1 root root  10286334 Mar 18 13:10 linux-esx-docs-5.10.210-1.x86_64.rpm

However, these packages standalone are rubbish without quality assurance, testing modules, limitations to avoid sha/naming/numbering mismatch, etc.

aroodgar commented 6 months ago

@dcasota I believe you should also change the config file generated in the SOURCES directory. Without doing it, the default config file is copied as the .config in the build source.

When I tried to do this, I encountered an error indicating difference between .config and .config.old as a result of running diff -u .config.old .config in the SOURCES/check_for_config_applicability.inc directory.

I added a simple cp .config .config.old as a temporary workaround in the file and then went through your solution again. It passed that git diff step successfully but encountered another error. Following are the logs:

+ info LD .tmp_vmlinux.kallsyms1
+ '[' '' '!=' silent_ ']'
+ printf '  %-7s %s\n' LD .tmp_vmlinux.kallsyms1
  LD      .tmp_vmlinux.kallsyms1
+ shift
+ '[' .tmp_vmlinux.kallsyms1 '!=' 1 ']'
+ strip_debug=-Wl,--strip-debug
+ '[' x86 '!=' um ']'
+ objects='--whole-archive                                              arch/x86/kernel/head_64.o arch/x86/kernel/head64.o arch/x86/kernel/ebda.o arch/x86/kernel/platform-quirks.o init/built-in.a usr/built-in.a arch/x86/built-in.a kernel/built-in.a certs/built-in.a mm/built-in.a fs/built-in.a ipc/built-in.a security/built-in.a crypto/built-in.a block/built-in.a io_uring/built-in.a lib/built-in.a arch/x86/lib/built-in.a  lib/lib.a  arch/x86/lib/lib.a drivers/built-in.a sound/built-in.a net/built-in.a virt/built-in.a arch/x86/power/built-in.a                                          --no-whole-archive           --start-group                                                                                                    --end-group                                                     '
+ ld -m elf_x86_64 -z noexecstack --emit-relocs --discard-none -z max-page-size=0x200000 --build-id=sha1 --orphan-handling=warn --strip-debug -o .tmp_vmlinux.kallsyms1 -T ./arch/x86/kernel/vmlinux.lds --whole-archive arch/x86/kernel/head_64.o arch/x86/kernel/head64.o arch/x86/kernel/ebda.o arch/x86/kernel/platform-quirks.o init/built-in.a usr/built-in.a arch/x86/built-in.a kernel/built-in.a certs/built-in.a mm/built-in.a fs/built-in.a ipc/built-in.a security/built-in.a crypto/built-in.a block/built-in.a io_uring/built-in.a lib/built-in.a arch/x86/lib/built-in.a lib/lib.a arch/x86/lib/lib.a drivers/built-in.a sound/built-in.a net/built-in.a virt/built-in.a arch/x86/power/built-in.a --no-whole-archive --start-group --end-group
ld: init/main.o: in function `start_kernel':
main.c:(.init.text+0x72e): undefined reference to `fips_integrity_init'
ld: arch/x86/kernel/alternative.o: in function `apply_alternatives':
alternative.c:(.text+0xbc4): undefined reference to `__canister_ealtinstructions'
ld: alternative.c:(.text+0xbcd): undefined reference to `__canister_saltinstructions'
+ on_exit
+ '[' 1 -ne 0 ']'
+ cleanup
+ rm -f '.btf.*'
+ rm -f .tmp_System.map
+ rm -f '.tmp_vmlinux*'
+ rm -f System.map
+ rm -f vmlinux
+ rm -f vmlinux.o
make: *** [Makefile:1207: vmlinux] Error 1
error: Bad exit status from /var/tmp/rpm-tmp.nfo1SI (%build)

RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.nfo1SI (%build)

/var/tmp/rpm-tmp.nfo1SI

Edit: Also, the last commands run leading to the error belong to this file: BUILD/linux-5.10.210/scripts/link-vmlinux.sh.

dcasota commented 6 months ago

@aroodgar the code successfully finished in a non-fips environment, not hardened, etc. Hence, e.g. a left config.old isn't traced as security issue. The purpose of config.old as you mentioned it, was intended for comparison purposes only. In addition, your code must run in a more secure environment and prevent issues e.g. undefined reference to fips_integrity_init. Honestly I have no experience with fips implementation. The Photon OS team in India for sure can give a hint, but now it's evening, and in US it's too early. With a bit of luck I'll have time for tests again next weekend.

aroodgar commented 6 months ago

@dcasota I am just asking because I am not really knowledgeable on this matter. Does a zero value of /proc/sys/crypto/fips_enabled imply a non-fips environment or do I need to check for other things?

dcasota commented 6 months ago

@aroodgar Here a few research findings.

What is fips? If an organization does any work for the United States federal government, it is likely that cryptography applications will be required to be in compliance with Federal Information Processing Standards. FIPS 140-2 is a security accreditation program for validating cryptographic modules produced by private companies.

VMware Export Control Policy VMware has an export policy, please read https://www.vmware.com/help/export-control.html.

value of /proc/sys/crypto/fips_enabled? If the output is 0, it means that FIPS mode is not enabled.

The recipe in https://github.com/vmware/photon/issues/1412#issuecomment-2003615673 contains make menuconfig. I started it, closed it, and a .config file has been created automatically. The rest of the recipe finished successfully. If the rpmbuild fails, the .config file specified with modules does not work. This leads not to the same issue as the question if recompiling the kernel is supported.

@JonathanVQP You are the initiator of the case. Your most recent feedback is one year old. Has this case been answered successfully?

vmware / photon