edk2* directories in downloaded "source code zip" are empty

[mi-hol]

Wanted to compile firmware with X86 emulation enabled, but all edk2* directories in downloaded "source code zip" are empty

This issue seems to exist likely since several (source code) releases. Related to https://github.com/pftf/RPi4/issues/203

[pbatard]

These are git submodules, so they will be empty if you just clone the project as is.

You need to familiarize yourself with git and submodule usage if you want to use this project.

Please look online, as you should be able to find plenty of resources about this (or run git submodule update --init --recursive).

[pbatard]

Also the source code zip is a default artefact produced by GitHub (we can't remove it, else we would do so), and should NOT be used to compile the firmware. Instead you should clone the project using git.

[mi-hol]

Thanks for the hint, not everybody is familiar with git submodules. Would you consider merging a PR to add this detail to project's readme ?

[pbatard]

As with any PR, that will depend of its content, but I see no reason why you shouldn't be able to submit one. I guess a #Compilation entry might prove useful for people who want to produce their own firmware, yes.

[mi-hol]

@pbatard thanks to the command git submodule update --init --recursive it seems I got over the initial hurdles :) Issues I encountered in following https://github.com/tianocore/edk2-platforms#using-uefi-tools-helper-scripts are:

1. the missing target rpi4

~/RPi4$ ./uefi-tools/edk2-build.sh rpi4
unknown arg rpi4
usage:
uefi-build.sh [-b DEBUG | RELEASE] [ all | juno | fvp | tc2 | overdrive | overdrive1000 | cello | hikey | hikey960 | d03 | d05 | d06 | armada70x0 | armada80x0 | mcbin | chaoskey | beagle | sni-box | capsuleapp | androidboot | androidfastboot | armvirtqemu | armvirtqemu64 | helloworld | sgi575 | ovmfx64 | ovmfia32 | rpi3 ]

2. when I tried to build listed target rpi3, it failed. Any hint how I could fix these issue?

/home/user/RPi4
Target: AARCH64
Build: X64
Building Raspberry Pi 3 (AARCH64) - AARCH64
CROSS_COMPILE="aarch64-linux-gnu-"
rpi3_BUILDFLAGS=' -D FIRMWARE_VER=cc6efb9'
PLATFORM_TOOLCHAIN is
Toolchain prefix: _AARCH64_PREFIX=aarch64-linux-gnu-
Build environment: Linux-5.10.60.1-microsoft-standard-WSL2-x86_64-with-glibc2.31
Build start time: 21:30:55, Mar.07 2022

build.py...
 : error 1005: Not supported target [platform/raspberrypi/rpi3/rpi3.dsc].
        Please select one of: all genc genmake modules libraries fds clean cleanall cleanlib run

- Failed -
Build end time: 21:30:55, Mar.07 2022
Build total time: 00:00:00

------------------------------------------------------------
                       Raspberry Pi 3 (AARCH64) RELEASE fail
------------------------------------------------------------
pass    0
fail    1

[pbatard]

I wasn't aware uefi-tools/edk2-build.sh even existed, and as mentioned in the edk2-platforms readme, "uefi-tools is a completely unofficial set of helper-scripts".

At this stage, I'm not sure if someone is ever going to update that helper tool for the Pi 4 platforms (or fix it for Pi 3 if it doesn't work), since it's unofficial.

The official way to build the platform is to follow the method that you can find in the build script from this very repository, which is also how you are supposed to build platforms that aren't covered by uefi-tools.

This basically boils down to make sure that your system has the acpica-tools gcc-aarch64-linux-gnu python3-distutils uuid-dev packages installed, and in a directory where you recursively cloned the edk2, edk2-platforms, edk2-non-osi repositories (recursively meaning that you have invoked git submodule update --init --recursive in each repository (though I think this may only be needed for edk2), running something like:

make -C edk2/BaseTools # NOTE: This command usually only need to be run once
export GCC5_AARCH64_PREFIX=aarch64-linux-gnu-
export WORKSPACE=$PWD
export PACKAGES_PATH=$WORKSPACE/edk2:$WORKSPACE/edk2-platforms:$WORKSPACE/edk2-non-osi
source edk2/edksetup.sh
build -a AARCH64 -t GCC5 -b RELEASE -p edk2-platforms/Platform/RaspberryPi/RPi4/RPi4.dsc

And once you have that working, then you can look into adding some options such as -D INCLUDE_TFTP_COMMAND=TRUE -D NETWORK_ISCSI_ENABLE=TRUE -D SMC_PCI_SUPPORT=1 or some PCDs such as --pcd gEfiMdeModulePkgTokenSpaceGuid.PcdFirmwareVersionString=L"My RPi4 UEFI Test Firmware".

But please bear in mind that what I'm giving you here is pretty much the same as what you'll find from the edk2-platform's documentation, so if you have trouble, please give it another look.

[mi-hol]

This basically boils down to make sure that your system has the acpica-tools gcc-aarch64-linux-gnu python3-distutils uuid-dev packages installed

Apparently I had not installed all of the required packages, after I fixed that uefi-tools/edk2-build.sh rpi3 worked correctly :)

But please bear in mind that what I'm giving you here is pretty much the same as what you'll find from the edk2-platform's documentation

you are certainly right, but there seems a lot of background knowhow required in order to follow this documentation correctly. I remember for example that installing ARM compiler was hindered by many references to web sites obsolete by now. Hence a newbie can get lost easily, I'll recheck on why I missed/incorrectly applied above package dependencies with the goal to improve documentation.

[mi-hol]

Thanks to your helping hand building for RPi4 succeeded

GUID cross reference file can be found at /home/user/RPi4/Build/RPi4/RELEASE_GCC5/FV/Guid.xref

FV Space Information
FVMAIN [99%Full] 6247040 (0x5f5280) total, 6247016 (0x5f5268) used, 24 (0x18) free
FVMAIN_COMPACT [61%Full] 1769472 (0x1b0000) total, 1088752 (0x109cf0) used, 680720 (0xa6310) free

- Done -
Build end time: 13:30:47, Mar.08 2022
Build total time: 00:03:29

[mi-hol]

look into adding some options such as -D INCLUDE_TFTP_COMMAND=TRUE -D NETWORK_ISCSI_ENABLE=TRUE -D SMC_PCI_SUPPORT=1

building with above option succeeded too, but I'm unsure about benefits of iSCSI & SMC_PCI_SUPPORT versus potentially associated risks. I'd assume these options are related to networking capabilities.

1. iSCSI is mentioned in Wiki but very little information is provided.
2. SMC_PCI_SUPPORT is only referenced in edk2-non-osi/Platform/RaspberryPi/RPi4/TrustedFirmware/Readme.md

Maybe you'd know pointers to documents explaining the background?