Support for HiFive Unmatched

[Dbof]

Hello everyone!

Since the HiFive Unleashed is discontinued (and pretty much unavailable in Europe) I was wondering if the successor, the HiFive Unmatched, was being considered. Are there any efforts regarding support of that board or the U740 RISC-V processor? I am looking for hardware to natively run Keystone on, and this seems the most promising one so far, but any other suggestions are appreciated.

How does one create these platform-specific files as found in sm/plat/sifive/fu540/ of the SM project?

*update: just saw a similar request in the SM repo: https://github.com/keystone-enclave/sm/issues/13#issue-934729754

[samwick07]

I too am interesting in extending Keystone to run on the HiFive Unmatched!

I'm currently attempting to make the required platform files (using the fu540 files as an example); but development has been slow-going as I'm not a software developer. Has anyone made a generic guide as for what to do to? I'm willing to put in the effort to get this going, but I'm not entirely sure where to start.

Thanks!

[dayeol]

Could you try to compile our sm with the generic platform first? I believe OpenSBI's generic platform may cover HiFive Unleashed / Unmatched

[samwick07]

Dayeol, Absolutely! I will give that a try and see where it leads. Thanks for the suggestion!

[samwick07]

All,

Some context: I intend to build a bootable image for the Unmatched with Keystone Enclave with Ubuntu 22.04 LTS as the OS. Ideally, I'd like to be able to boot the Unmatched from the onboard flash (QuadSPI) and from an SD card. From what I can tell, Canonical only provides pre-built server images for the Unmatched.

After examining the generic platform files included within the most recent OpenSBI version, I can confirm that the U740 is supported, and thus it should be possible to get Keystone on the Unmatched. Unfortunately, I have been unsuccessful at building a bootable SD card. I am unfamiliar with the build steps and directory hierarchies required to build the bootloader with Keystone Enclave.

My confusion revolves around the appropriate directory structure and my lack of understanding about the build process: Where does the /opensbi directory need to reside? Should it belong inside the /keystone directory? Or the other way around? I'm new to all of this...

From the Keystone SM GitHub page, for the make step: What is expected for the build directory path and Linux image path? Can the build directory path be an arbitrary build directory from my current working directory? Will the pre-built Ubuntu 22.04 image file suffice for the Linux image? I'm lost.

Kindly, could anyone assist in building my desired bootable image? I'm happy to learn and build the image myself, but I'm hitting some walls.

Much thanks!

[dayeol]

If you do make from the build directory of the top-level keystone repo, the build system automatically generates firmware binary files under sm.build/platform/generic/firmware directory. These files are various OpenSBI binaries, so you can use them to replace the original OpenSBI binaries they provide. For example, if you are writing the entire boot image (OpenSBI + Linux kernel) into the SD card, you can just burn fw_payload.bin into the card and boot from there. I'm not super certain what boot options Unmatched provides, but if you are able to boot from OpenSBI then probably it will boot OpenSBI+SM without any issues.

[dayeol]

And thank you for doing this! Please keep me posted how it goes. :D

[samwick07]

Thanks for reaching out!

I continue to hit errors within the make step. Particularly, for this command: make -C opensbi O=<build dir> PLATFORM_DIR=$(pwd)/plat/generic CROSS_COMPILE=riscv64-unknown-elf- FW_PAYLOAD_PATH=<path/to/linux/image> FW_PAYLOAD=y.

Most notably, make cannot run rom the top-level Keystone repo as there is no /opensbi directory. From within the /sm directory, the /opensbi directory exists. I then modified the /plat and /platform directories to contain the most recent sifive_fu740.c configuration file. When executed, the make command attempts to run, but I have been unsuccessful with this also; likely due to my confusion about the expected parameter types.

What file type is expected for the Linux image? Is my use of the Ubuntu Preinstalled Server Image <image>.img appropriate? Or does the <path/to/linux/image> need to be the desired Linux Kernel? I seem to be misunderstanding the intent for this parameter.

Thanks! Hopefully, I'll be up and running soon with Keystone on the Unmatched!

[dayeol]

What I meant is running make after cmake: http://docs.keystone-enclave.org/en/latest/Getting-Started/QEMU-Compile-Sources.html

mkdir <build directory>
cd <build directory>
cmake ..
make

We already have written CMakeListst.txt which generates makefile commands for building SM. If you run make after cmake in your build directory, you should be able to see the Linux image in linux.build directory and the OpenSBI firmware image in sm.build directory.

I'm recommending you build the top-level repo, instead of the SM separately.

[samwick07]

Dayeol,

Would it be possible to update Keystone's OpenSBI git submodule to the latest version? It would help streamline my testing process by allowing me to use OpenSBI's most recent platform configuration files (including the sifive_fu740.c) without having to manually move it over for each build attempt.

Thanks.

[samwick07]

Update:

I've nearly gotten it working... But not quite there yet.

I have followed the Keystone Documentation to build Keystone.

After running the ./fast-setup.sh script, I updated the platform configuration files within keystone/sm/plat/generic directory to match the most recent OpenSBI generic platform. From within the Keystone top-level directory, I've successfully replaced the opensbi and linux directories with their latest builds from their respective repos. I then applied unmatched patches to opensbi and to linux using this repo as a guide.

From there, I created the keystone build directory and successfully completed cmake ... Following this, I ran make and didn't succeed...

make failed at Performing build step for 'eryie-test-eryie' where I received the following message: Makefile:7: *** KEYSTONE_SDK_DIR is undefined. Stop. I'm not sure how to get past this issue, especially because I have already set the relevant environment variables and added them to my shell's startup file.

Since I am particularly focused on building the sm, I also attempted make sm. This is still in progress. Of all the Keystone submodules, which run in M-mode? If just the SM runs in M-Mode, I'm thinking I'll just build the firmware with the SM and then try to build the Keystone SDK directly on the Unmatched after the fact.

Any advice? Thanks!

[samwick07]

Success! Mostly.

After commenting out the add_patch macro from the CMakeLists.txt and all references to the included patches, I was able to build the SM. (I previously patched the linux and opensbi directories as described earlier.)

I now have the appropriate binaries to proceed with building U-Boot. I'll keep posting as I build in case anyone is interested.

[samwick07]

Just curious, has anyone else had luck getting Keystone Enclave running on the Unmatched? I can't seem to get past the insmod linux-keystone-driver.ko step on the Unmatched. I've successfully built the modified OpenSBI + Keystone bootloader and built several modern Linux Kernels. Is there a particular Kernel version required for enclave compatibility?

[dayeol]

What is your current Linux version? and what error message insmod gives you?

[samwick07]

I've built for Linux versions 5.11.0-1020-generic and for 5.13.0.1007-generic. I get insmod: error inserting 'linux-keystone-driver.ko': -1 Invalid module format. I've tried this step with Ubuntu 22.04, 21.10, 21.04, and OpenEmbedded 20.04, each with their appropriate matching Linux Kernel versions... I can't seem to get the driver to work though.

[samwick07]

So, I've just now realized that Canonical provides their own distribution kernels for Ubuntu that are based on upstream Linux kernel versions. Because of this difference in kernel versioning, my linux-keystone-driver.ko will not be compatible with any viable version of Ubuntu for the Unmatched.

I've built Keystone with a few versions of the 5.16.0-rc kernel; however, the only compatible Ubuntu images available for the Unmatched are prebuilt server images for Ubuntu 21.04, 21.10, and 22.04. This means that I can't feasibly modify my Ubuntu kernel to match the kernel I compiled for use with Keystone. Additionally, without the source for Canonical's distribution kernel, I can't conceivably build keystone to target their precise kernel distributions 5.13.0-1007-generic or 5.11.0-1023-generic. It seems like the only path forward is to switch to another open-source Linux distribution.

Side note, I've been able to successfully build linux, sm, and driver using make, but I have not been able to build the other individual components. I'd be interested to hear if anyone else has had success in building tests.

Thanks!

[hsrakesh]

Hello all,

Is keystone driver supports for latest kernel. If yes, is any changes required in kernel.

Is anyone tried for hifive-unmatched board. Please send me the steps/procedure.

Thanks