Is photon-os-installer bits injection into a broken (Ph4 rev1) installation possible?

[dcasota]

Hi,

I'm struggling with utilizing the 2.0 bits. On photon-4.0-ca7c9e933.iso the setup before the installer ui phase breaks on a tiny, physical x86_64 workstation with mbr boot. Nothing new (https://github.com/vmware/photon/issues/1042), and the workaround of modifying the kernel commandline with photon.media=UUID= works flawlessly.

The photon-os-installer 2.0 approach seems somewhat interesting (the wording 'secondary ks device' nails it perfectly!), hence, I've build the photon-os-installer 2.0 bits accordingly to the recipe (btw. I had to install python3-devel, too) and copied ./dist/photon-os-installer into the /bin directory of the broken installation. I thought that simply a copy would be enough. Starting the executable, now it throws 'No module named curses'. I thought it makes sense to share with you this finding. A simple copy of photon-os-installer wasn't mentioned as eligible method though.

ps.1 On photon-4.0-ca7c9e933-aarch64.iso with UEFI boot the setup enters the installer ui phase without the described photon-os-installer 1.x issue.

ps.2 Running make image on photon-4.0-ca7c9e933 (Azure virtual machine)

git clone -b 4.0 https://github.com/vmware/photon.git
cd photon
make image IMG_NAME=iso

throws out

Traceback (most recent call last):
  File "/root/photon/./build.py", line 17, in <module>
    import docker
ModuleNotFoundError: No module named 'docker'
make: *** [Makefile:6: image] Error 1

Please, clarify the step "Building Photon Cloud images using Photon OS Installer".

[gpiyush-dev]

Hi @dcasota,

Thanks for reporting this. You need not make your own binary for consuming 2.0 installer bits as the changes are merged in the photon 4.0 branch. In order to consume 2.0 bits, you need to follow these steps.

git clone -b 4.0 https://github.com/vmware/photon.git
cd photon
make iso

docs to build photon from source - https://vmware.github.io/photon/assets/files/html/3.0/photon_installation/build-photon.html

In order to make your own custom installer binary, you can install the BuildRequires: packages first mentioned here and then you need to build the binary as mentioned in the readme steps. You can use this custom binary at run time to debug the installer or install photon on a secondary attached disk. After building the binary, you can invoke the binary through this command:-

photon-installer -i iso -o <build-options-file> -r <rpms-path>

build-options-file - https://github.com/vmware/photon/blob/4.0/common/data/build_install_options_all.json rpms-path - Clone photon, make packages, rpms will be built in photon-dir/stage/RPMS.

If you are building any other image than iso. You can follow the steps mentioned in the readme under the heading Building Photon Cloud images using Photon OS Installer.

I will update the readme to add the required packages to install before building the binary. Let me know if you find any more issues.

Thanks, Piyush

[dcasota]

fyi After 14 hours in progress and somewhere in step 3, I've changed my virtual hardware specs from 2vcpu to 4vcpu + 32gb ram, and restarted the process with make -j$((`nproc`+1)) image IMG_NAME=iso. A recipe for a distributed workload of make image would be nice.

[gpiyush-dev]

It's already present. Try this- make iso THREADS=10

[dcasota]

Hi @gpiyush-dev, the make image failed during kafka.spec. Here's the protocol snippet. Kind regards, Daniel photon-iso-build-issues.txt

[gpiyush-dev]

It's a known issue that is intermittent in nature, seen from Gradle side. If you retrigger it again you might not see it. Github issue - https://github.com/gradle/gradle/issues/3777

[dcasota]

Thanks, the iso has been successfully created! Here's the custom recipe just as is.

1. Install Photon OS 3.0 (eg. on Azure photon-azure-3.0-9355405.vhd), login as root user
2. Use the latest photon-os-installer: run the following script to install photon-os-installer and to run 'make image'

   if [ `cat /etc/yum.repos.d/photon.repo | grep -o "packages.vmware.com/photon" | wc -l` -eq 0 ]; then
   cd /etc/yum.repos.d/
   sed -i 's/dl.bintray.com\/vmware/packages.vmware.com\/photon\/$releasever/g' photon.repo photon-updates.repo photon-extras.repo photon-debuginfo.repo
   fi
   # run distro-sync as first test if the process breaks
   tdnf distro-sync -y
   tdnf install -y kpartx git bc build-essential createrepo_c texinfo wget python3-pip tar dosfstools cdrkit
   pip3 install docker==2.3.0
   cd /
   # use latest photon-os-installer
   pip3 install git+https://github.com/vmware/photon-os-installer.git
   # specify the photon os branch
   git clone -b 4.0 https://github.com/vmware/photon.git
   cd /photon
   make -j$((`nproc`+1)) image IMG_NAME=iso THREADS=10

3. Sometimes the process breaks and has to be restarted. Here some troubleshooting info. "Failed during building package": This occured eg. on 'Building rpm is failed /usr/src/photon/SPECS/kafka.spec'. The workaround is to restart the process with less parallel threads. make -j4 image IMG_NAME=iso THREADS=4 Keep in mind: Just because the process does not break anymore it doesn't mean that there aren't issues. Analyze the logfile and make decisions.
4. Grab the iso from ./stage.

   root@ph3 [ /photon ]# find . -name *.iso
   ./stage/photon-4.0-2c3bc6f88.debug.iso
   ./stage/photon-4.0-2c3bc6f88.iso

   Eg. configure Photon to allow ssh access and download the iso.

Hope this helps for your readme, too. Thanks for the journey so far! I'll share the findings about photon-os-installer 2.0 as soon as possible.

The build process with multiple runs [hardware-limited first run, process limitation aware next run(s)] is somewhat slow, and the more branches the slower. Using more performant hardware and fully run the build virtual machine in memory may be an approach for optimization. Not sure if there is a different, affordable build process approach. Simply building all ~2'000 packages and the iso of a branch in an hour would be nice though.

[dcasota]

okay, here some findings.

Booting from a helper-utility called ventoy, the photon-4.0-2c3bc6f88.iso (with included installer 2.0 bits) and with modified kernel commandline photon.media=/dev/mapper/ventoy starts flawlessly.

Booting directly from the photon-4.0-2c3bc6f88.iso and with modified kernel commandline photon.media=/dev/sdb1 starts flawlessly.

Booting directly from the photon-4.0-2c3bc6f88.iso without modifying the kernel commandline, the setup before the installer ui phase still breaks, but it enters now the 'retry mount the cd' code lines 160++ because of 'no medium found'. Medium /dev/sr0 as a detection result from kernel commandline photon.media=cdrom for this particular case is wrong.

[gpiyush-dev]

Yep, booting from photon.media=cdrom by default takes /dev/sr0 as the media to mount - https://github.com/vmware/photon-os-installer/blob/b8efd0d75f1db2b282e83d766b4060b927148a1b/photon_installer/isoInstaller.py#L153 and that's expected behavior if iso is not attached to cdrom.

[dcasota]

Indeed. First, the recipe on how to consume the photon-os-installer bits is much more better than the copy-based injection attempt. That said, many thanks for the support. -> close "issue".

photon-os-installer is promising. On goal might be to allow choice for boot devices (usb, internal hd, cdrom), uefi/bios boot, for filesystems (fat32, exfat,) and driver support (usb 2.0), and for mount/unmount virtual media from same or different device. Photon OS should easily run on legacy hardware. In an old esxi kickstart source in the usb section I've used parted. The source is from 2011, nothing to refactor. The kernel commandline cdrom-detect/try-usb=true option in Ubuntu reflects a similar approach.

MountUSBDevice()
{
usbdevice=`parted -l|grep -i "fat32" --before-context=6|grep -i "Disk /dev/"|awk '{print $2}'|sed  's/://'`
if [ "$usbdevice" == "" ]; then
    echo "Could not find usb device. Please check if usb device is present"
    echo "and ready to mount. Check following partition table as additional help:"
    cat /proc/partitions
    echo -n "Enter device path manually (example: /dev/sde1) : "
    read usbdevice
fi
mkdir /mnt/usbdisk 1>/dev/null 2>/dev/null
mount ${usbdevice}1 /mnt/usbdisk 1>/dev/null 2>/dev/null
if [ $? -ne 0 ]; then
    echo "Path $usbdevice is not recognized. Please check:"
    echo "- The $usbdevice must be the usb stick."
    echo "- The usb stick must be formatted with FAT32."
    echo "Press a key."
    usbdevice=
    read
fi
}

[dcasota]

Hi @gpiyush-dev

Yep, booting from photon.media=cdrom by default takes /dev/sr0 as the media to mount -

https://github.com/vmware/photon-os-installer/blob/b8efd0d75f1db2b282e83d766b4060b927148a1b/photon_installer/isoInstaller.py#L153

and that's expected behavior if iso is not attached to cdrom.

For the use case that iso is not attached to cdrom, but Photon OS has started with default kernel cmdline photon.media=cdrom, here a suggestion. Instead of aligning it fixed to /dev/cdrom , doing a pre-detection of bootable, syslinux-marked partitions helps to install Photon OS on much more baremetal (legacy) hardware.

The bash code snippet below lists detected, bootable, syslinux-marked medias. As output you get eg. /dev/sdb1.

for i in $(fdisk -l ¦ grep '^/dev/[a-z]*[0-9]' ¦ grep '*' ¦ cut -b 1-10); do (if [ "$(dd if=$i bs=1 count=62 status=none ¦ hexdump -C ¦ grep SYSLINUX)" ]; then echo $i;fi);done

In case of one or more detected medias with supported filesystems, ask the user to select his Photon OS boot and repo media.

Hope this helps. Daniel

[gpiyush-dev]

Thanks, @dcasota for the suggestion. I will try to implement this.

[dcasota]

~Not sure if this https://github.com/dcasota/photon-os-installer/blob/master/photon_installer/isoInstaller.py works. I might pretest it this weekend. The version proposed allows one single valid boot media. If none or more than one is detected, it raises an exception.~ The test code is poor quality.