Microcode update from AMD available - possible to include in coreboot build?

[fhloston]

There is updated microcode from AMD and it can be found in the wild:

https://github.com/platomav/CPUMicrocodes/blob/master/AMD/cpu00730F01_ver07030106_2018-02-09_88EDFAA0.bin

I have successfully loaded that microcode in my debian installation.

Can you include that into the coreboot builds, please?

[pietrushnic]

@fhloston thanks for this information. We will try to address that in next release although I have to consult if using microcode published "in the wild" would be ok. Have you got any pointers, if anyone tried it and can confirm that issues were mitigated?

[fhloston]

@pietrushnic I tried loading it during initrd and in a live system myself and it does contain some microcode enhancements which weren't there before:

• Indirect Branch Prediction Barrier (IBPB)
  • PRED_CMD MSR is available: YES
  • CPU indicates IBPB capability: YES (IBPB_SUPPORT feature bit)

Can't you just request the microcode from AMD?

[fhloston]

I should add, the raw microcode needs to be formatted properly into microcode_amd_fam16h.bin

https://github.com/groeck/amd-ucodegen

(link fixed)

[gretel]

@fhloston could you possibly post the commands .. script you are using to load the firmware at runtime? regards

[fhloston]

@gretel easy:

amd-ucodegen cpu00730F01_ver07030106_2018-02-09_88EDFAA0.bin -o microcode_amd_fam16h.bin

backup old file in /lib/firmware/amd-ucode (linux) or /usr/local/share/cpucontrol (freebsd) put new file there echo 1> /sys/devices/system/cpu/microcode/reload (linux) cpucontrol -v -u /dev/cpuctl[0-3] (freebsd)

you might have to install devcpu-data on freebsd first, and activate the respective service

[gretel]

@fhloston thanks! for freebsd this is what i did:

$ git clone --depth 1 https://github.com/groeck/amd-ucodegen
$ cd amd-ucodegen
$ make
$ curl -O https://github.com/platomav/CPUMicrocodes/blob/master/AMD/cpu00730F01_ver07030106_2018-02-09_88EDFAA0.bin
$ ./amd-ucodegen cpu00730F01_ver07030106_2018-02-09_88EDFAA0.bin -o microcode_amd_fam16h.bin

[pietrushnic]

@fhloston decision was made to not include unofficial microcode binaries in official PC Engines firmware releases. I believe we can prepare unofficial firmware release based on v4.6.9 and/or v4.0.17 if there is interest in that.

Meanwhile, we are in contact with AMD and trying to obtain official microcode release.

[gretel]

@pietrushnic thanks - i am interested.

[fhloston]

@pietrushnic thanks for the update! also interested!

[pietrushnic]

@miczyg1 let's prepare unofficial releases version with new microcode. @fhloston @gretel I think, because of long weekend in Poland, it would be ready before end of next week. Sorry for delay.

[gretel]

@pietrushnic cool.

[miczyg1]

@fhloston we have been able to include microcode updates in coreboot, yet we still need some information about new microcode features You listed:

Indirect Branch Prediction Barrier (IBPB)

• PRED_CMD MSR is available: YES
• CPU indicates IBPB capability: YES (IBPB_SUPPORT feature bit)

Where did You get these information from? Is there any possibility to verify these features?

[fhloston]

@miczyg1 I loaded the updated microcode on a Debian Stretch Installation and used spectre-meltdown-checker to verify changes. It is also packaged in stretch-backports.

https://github.com/speed47/spectre-meltdown-checker

[pietrushnic]

@miczyg1 please publish here links to custom binaries tomorrow EOD.

[miczyg1]

@fhloston @gretel sorry for the delay. Binaries are available here. Keep in mind that the microcode comes from wild source. We will not take any responsibility for any damage these binaries will cause to Your boards. We are in contact with AMD and will try to obtain official microcode patch.

[gretel]

@miczyg1 thanks for getting back to us! going to be back from travels on friday and report back.

[nicklowe]

The latest AMD microcode/ucode is available from a reputable source:

https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/amd-ucode

https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/log/amd-ucode

[nicklowe]

Could this be built in to 4.6.10?

[miczyg1]

@nicklowe unfortunately linux demands microcode in special container designed for kernel. We could use only raw binaries delivered by AMD. Second thing is we finished development cycle for v4.6.10 last Friday, the binaries should be available soon. Depending on result of our negotiations with AMD we may obtain the raw microcode binary, however we cannot assure the microcode will be included in next release.

[gretel]

still didnt get a chance to uprade mine :(

[nicklowe]

Couldn't you use the appropriate file from linux-firmware.git and use https://github.com/platomav/MCExtractor to strip the container?

[fhloston]

guys, what is the issue with the container?

@gretel posted on April 24th how that container is built from raw microcode.

[nicklowe]

I flashed the APU2 4.6.9 ucode image supplied by @miczyg1 and observed the following:

root@OpenWrt:~# dmesg | grep microcode [ 1.541701] microcode: CPU0: patch_level=0x07030105 [ 1.558224] microcode: CPU1: patch_level=0x07030105 [ 1.570174] microcode: CPU2: patch_level=0x07030105 [ 1.575078] microcode: CPU3: patch_level=0x07030105 [ 1.580070] microcode: Microcode Update Driver: v2.2.

I don't think this is working currently therefore.

[miczyg1]

@nicklowe it may be that kernel is overriding the microcode patchlevel. I am 100% sure that MSR register responsible for storing current patch level reported 0x07030106 in coreboot. I would have to investigate it.

[nicklowe]

Hmm, well, OpenWRT's amd64-microcode package is not installed so there would be nothing to downgrade it with.

[paravoid]

That git log from linux-firmware above shows commits back in May 2018 with microcode updates, but if you look closer these were just for families 15h/17h. The log for 16h (which the APU2 uses) is: https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/log/amd-ucode/microcode_amd_fam16h.bin …and that was last updated in November 2014.

@miczyg1 if you're in contact with AMD, it might be worth to ask them to also update linux-firmware like they've done with the other families and/or get the person who made those commits there (who seems to have an @amd.com email address :) in the loop. Thanks for your responsiveness here and your efforts in general regardless!

[paulmenzel]

Judging from this discussion, the linux-firmware files are also outdated for Family 15h (or the metadata wasn’t updated).

Sadly AMD microcodes at linux-firmware repository are seriously outdated:

• microcode_amd_fam15h.bin <-- CPU ID 0x610F01 microcode - version 0x6001119 [2012-07-13]
• microcode_amd_fam16h.bin <-- CPU ID 0x700F01 microcode - version 0x700010F [2014-02-19]

[pietrushnic]

@paulmenzel this is because AMD doesn't give permission to publicly release binaries. I'm pretty sure they have ucode with fixes for internal/debugging use cases. I also think that have no issue with providing code for big IBV (like AMI, Insyde, Phoenix etc.).

[kolargol]

Any update on this issue?

[pietrushnic]

Cześć @kolargol , unfortunately, no. AMD do not reply to our queries. I can say binaries definitely exist, but w have no rights to distribute. Vendor wonders about placing an official statement about AMD support so stay tuned.

What I can advise is getting binaries from not the official source (if you trust those) and building firmware yourself, there is make menuconfig option to include external binary.

@miczyg1 do we have some guide about that? Maybe publishing guide and/or blog post would help people watching this thread.

The last resort we probably can produce community firmware with unofficial binaries with a note that it can be potentially dangerous and we are not responsible for any damage.

If you have other suggestions please let us know we would be glad to help, but lack of silicon vendor support just ties our hands. Blame AMD not PC Engines.

[kolargol]

Hi, I see 4.8.0.5 is tagged, any news on microcode for APU2?

[miczyg1]

@kolargol unfortunately not. I plan to send upstream the code that will enable loading microcode patches in official coreboot for apu platforms and prepare a guide how to build PC Engines firmware image with microcode this month. But that is all we can do.

Since we cannot publish any microcode blobs (not even mentioning about obtaining any), we can not release firmware containing any microcode updates.

[paravoid]

If you're in contact with them… my suggestion would be to ask AMD to push those in upstream linux-firmware, like they've done multiple times in the past for other CPU families (commits by AMD's Sherry Hurwitz, who you should probably Cc!).

These would get them licensed under linux-firmware's LICENSE.amd-ucode which is a very permissive license and would in turn give you permission to distribute as well :) Probably easier to go with something that's already set up, rather than trying to convince them to do a vendor-specific arrangement…

[pietrushnic]

@paravoid I will ping them again according to Your suggestion, but please note that Linux community was not able to force AMD to release correct firmware binaries for this family, so I doubt my complaints will have any results. More to that according to my experience this platform is treated like legacy hardware and they don't want to spend too much budget to support it.

[gretel]

update: https://github.com/pcengines/apu2-documentation/blob/master/docs/microcode_patching.md

[pietrushnic]

@gretel in next iteration blog post coming how to patch and validate based on this guide.

[kolargol]

I confirm all works great and on OpenBSD i see mitigation works as expected. Thanks for making this possible! It is improtant addition.

[gretel]

@gretel in next iteration blog post coming how to patch and validate based on this guide.

@pietrushnic just built as described on very recent osx. worked fine. had to patch out the use of sudo in build.sh. going to flash later..

[kolargol]

@gretel in next iteration blog post coming how to patch and validate based on this guide.

@pietrushnic just built as described on very recent osx. worked fine. had to patch out the use of sudo in build.sh. going to flash later..

actually you have to do some more changes to work on macOS: md5sha->md5 and remove realpath reliance build.txt

[gretel]

@kolargol good point, have coreutils installed, guess that made it compile.

[kolargol]

yes, but on clean macOS you need that changes. If someone want to try binary build for APU2: https://blog.onefellow.com/post/181047767608/apu2-spectre-mitigation

[pietrushnic]

@kolargol @gretel Many thanks for testing effort. Feel free to spread the word about 3mdeb and PC Engines :)

[pietrushnic]

@miczyg1 I think we can close that issue or we planning something more here?

[miczyg1]

@pietrushnic I do not think we have more to do here.

[kolargol]

Hi, guys. Microcode load vanished from v4.9.0.2 release - this is intended or bug ?

[Firefishy]

UPDATE: DO NOT USE THIS WORKAROUND AS PER https://github.com/pcengines/apu2-documentation/issues/75#issuecomment-462976047 . It works for me, but maybe I am just lucky ;-)

@kolargol I was able to load in the microcode with v4.9.0.2, with an extra step. After step 4 of this: https://github.com/pcengines/apu2-documentation/blob/master/docs/microcode_patching.md do the following, edit file release/coreboot/src/cpu/Kconfig and the setting under "config SUPPORT_CPU_UCODE_IN_CBFS" change "default n" to "default y", then continue the guide from step 5.

[miczyg1]

@kolargol the microcode update has been changed since it did not patch all cores previously. Current microcode update procedure works well for all cores. The guide is available in https://github.com/pcengines/apu2-documentation/blob/master/docs/microcode_patching.md

Please do not follow @Firefishy guide since it is wrong way to do that. There is a special Kconfig option under microcode inclusion menu: Add microcode patch for AMD fam16h (EXPERIMENTAL). It will correctly include the microcode binary into coreboot. Any other method will patch only 1 core and lead to platform instability.

[Firefishy]

@miczyg1 Try follow https://github.com/pcengines/apu2-documentation/blob/master/docs/microcode_patching.md with v4.9.0.2. The Add microcode patch for AMD fam16h (EXPERIMENTAL) option does not display because Include CPU microcode in CBFS is hidden because SUPPORT_CPU_UCODE_IN_CBFS is set to false.

[kolargol]

@miczyg1 the menu option you mention vanished from v4.9.0.2 and @Firefishy seems right. I am building firmware right now and will let you know if all cores are patched.