Configuring Hummingboard fTPM for connection to Azure

[kiwikissling]

Our Hummingboard/iot-core efforts to-date have involved using the IOT Dashboard to build and test images. As part of this, we've used the sTMP to connect to our IoT Hub.

I'm now in the process of scripting the FFU build from top to bottom and it's time to (hopefully) move from the sTPM to the Hummingboard's fTPM. I've struggled to find good information on how to do this, and so hope that someone here can help.

Even though this didn't appear to be Hummingboard-specific, I started following Setting up TPM on Suggested Platforms, but got stuck at (2). Hitting F2 on boot got me to a Hummingboard (UEFI?) boot menu, and on selecting 'Device Manager' I see an empty 'Devices List'. If that's the mechanism needed to work through the Setting Up TPM guidance then it's kinda a dead end.

I have pulled and built Limpet, and have T2T.exe and Limpet included as part of my scripted FFU build.

Has anyone successfully configured the fTPM on a Hummingboard for connection to Azure? I'd love to learn from you! Many thanks.

PS. I'm not sure if this is the 'right' forum for asking these questions. Apologies if it's not, only I've received very helpful responses to earlier questions.

[dmcilvaney]

Looks like that document is indeed very specific to the Minnow Board, although the testing with t2t and urchin should work the same. I'll see about getting it updated with a note for iMX.

The modern fTPM on iMX bypasses the UEFI steps and communicates with Windows automatically.

The driver side was updated in https://github.com/ms-iot/imx-iotcore/pull/78/files, but we had a hold up on updating the actual binaries which are included (nothing technical). The binaries are normally located in UEFI at compile time. In hind sight I probably should have rolled the driver changes back.

The easiest way forward is to build your own TPM binary (and get a new authenticated variable store TA for UEFI as well). Running make update_tas in the firmware folder will trigger an update. You might need to clone the MSRSec repo if you have not already done so (its listed as optional in the firmware build document).

I think that's definitely the best path forward, but if its not suitable/doesn't fix it there are ways to debug further. I'll see what I can do to get the TAs updated soon regardless.

RPMB CLEAR Depending on what was run before, there may be old artifacts left in the secure RPMB storage. I think you should be in the clear but if you run into issues, re-compile OP-TEE with CFG_RPMB_RESET_FAT=y

[kiwikissling]

@dmcilvaney , Thanks for this. This is a journey for me... from the relatively fluffy, light world of UI apps down into very uncharted depths of UEFI and boot firmware. Evidence? I don't know what TA means :) I may have to wait until you've been able to take a further look. Regards, Andrew

[dmcilvaney]

@kiwikissling No worries, firmware is not a simple endeavor at the best of times.

Quick question: Are you running the pre-built firmware included in the imx-iotcore repo, or have you customized your own at all? If you have customized it I should be able to get you up and running quite quickly.

As for TAs and OP-TEE: Our firmware stack goes SPL->OP-TEE->U-Boot->UEFI->Windows. OP-TEE is a secure kernel which remains in memory for the life of the system, and is protected by ARM Trust Zone (hardware enforced access control, etc). OP-TEE runs programs called Trusted Applications (TAs). These are sometimes also called enclaves (Intel nomenclature from Intel SGX which provides similar security).

For a concrete example, our firmware TPM (fTPM) TA is bundled with UEFI. UEFI requests that OP-TEE starts the TA (identified by a UUID), and when OP-TEE asks for the program, passes it the TA binary (lots of details omitted here). OP-TEE is responsible for measuring and verifying the signing of the TA, etc. Once OP-TEE is happy it loads the TA binary into secure memory and allows it to process requests.

UEFI can then make requests to the fTPM through OP-TEE (specifically, each non-secure OS runs an OP-TEE "supplicant" which handles switching to/from secure world, passing data, and finding TA binaries among other things). The fTPM it is a "persistent" TA, which means even when UEFI closes its session the TA remains loaded in memory by OP-TEE.

Windows then connects to the fTPM from its OP-TEE driver/supplicant (https://github.com/ms-iot/imx-iotcore/blob/public_preview/driver/TrEE/TrEE/FtpmService.c). Since the TA is persistent OP-TEE can provide an immediate connection instead of re-loading it. Windows will then send all TPM requests through this connection. This is actually where I think you are running into issues, since the OP-TEE drivers were updated to expect a new UUID, but the old fTPM has a different one.

There is lots of OP-TEE documentation here https://optee.readthedocs.io/ , but ideally it should all just work.

[kiwikissling]

@dmcilvaney , That's a really helpful explanation. Thank you - I've learned something today :)

Yes, I'm running pre-built firmware directly from the imx-core repo. This is work I'm doing to test the end-to-end scripting of a Retail build using off-the-shelf iMX-based hardware ... while a separate boot issue is being worked on. I have an email discussion in-play with NXP (@mvarecha ) and Microsoft (@hbeberman) about that issue. Mentioning this for clarity as this thread, and that boot issue are not related.

[dmcilvaney]

Ok, I was planning on making a concerted effort to get the firmware updated next week, so this is extra impetus to do so.

If you are blocked and need a fast fix I think I know a work around but the best option is probably to wait on the updated firmware.

[kiwikissling]

Not blocked -- I have a full backlog of independent tasks. Happy to wait. Many thanks

[dmcilvaney]

New Binaries

Working to get the binaries updated (more on that later), but here is a UEFI which should include a fixed TPM if you want to get started.

uefi.zip

How to update

Replace the uefi.fit in HERE with the new one from the zip. This will add the updated fTPM which can talk to the newer Windows drivers to the FFUs you build.

This new fTPM has seperate storage from the old fTPM so there should be no conflicts, and it will seem like a brand new TPM on the first boot (more on this later) .

Reseting NV storage if needed

If you do run into any issues, I've also attached a 'factory reset' firmware which will clear the non-volatile (NV) storage used by the fTPM. It will boot until it reaches UEFI, clear all the NV storage, print a message to serial, then hang. Personally I have a second SD card sitting around which I keep this firmware on.

reset_NV.zip

Two options for deploying it:

1. Write it directly to an existing SD card via: Deploying firmware

   This will require grabbing the dd.exe tool and figuring out which physical drive your SD card is, but is MUCH quicker. You will need to either use dd.exe to put the normal firmware back again, or completly re-flash the SD card if you want to use it to boot normally again.

2. Replace the boot loader Here and build a completely new FFU.

   Much slower, but doesn't require any extra setup. Just don't forget to swap back to the normal firmware when you are done.

UUID and storage conflicts

The new fTPM uses a UUID (0x53BAB89C...), while the old one was (0xBC50D971...). Unfortunately I'm going to have to roll that change back shortly.

If you use this new version, it will work fine until I roll back the UUID in the drivers. The rolled back UUID TPM will be unable to access the storage of the one I just gave you, so beware.

Explanation

Each TA has its own storage, unique to its UUID. I chose a new UUID when I updated the fTPM since it involved restructuring the underlying storage and I didn't want any conflicts with old data.

Unfortunately (for this discussion anyways, otherwise quite exciting), there is interest in using this fTPM with Linux, and a driver was written to do so. The driver was written against the OLD UUID, and they don't want to move to the new UUID since it is so late in the upstreaming proccess. So... we are going to have to roll the UUID back on our end, which means possible storage conflicts with older versions.

We are working on two solutions for this:

1. The fTPM will, in the future, have an option to recover from corrupted storage by clearing its own data (maybe not ideal for production, but really helpful during development).
2. Provide a purpose built factory reset option, probably in UEFI, to clear the storage.

The reset firmware I attached is a stopgap version of 2.

Sorry for the confusing mess, please let me know if you have any questions or need any help with the process.

[kiwikissling]

Many thanks for these files, and for the comprehensive explanation. I'm going to be travelling for the next three weeks ... and sneaking my IoT kit into the suitcase would be a dogbox-able offence if I was caught by my lovely wife ... so I'll have a good look at all this on my return. Cheers.