Spectre mitigation (retpoline)

[Esokrates]

Hi Matthias,

Thanks for your amazing effort. I know you have plenty to do right now, but let me ask, do you plan to backport the spectre mitigations (retpoline) too?

(https://lkml.org/lkml/2018/1/3/780)

[minipli]

I take what lands upstream in the v4.9 stable branch -- probably the microcode and MSR based solution.

[theLOICofFRANCE]

CVE-2017-5715: Mitigation 1 -- solution: Use CONFIG_X86_MSR and add also the support for IBRS Mitigation 2 -- solution: Add CONFIG_RETPOLINE

I hope you'll make a backport if need it.

[biergaizi]

Retpoline going to be landed in 4.9. https://www.phoronix.com/scan.php?page=news_item&px=Linux-4.9-4.14-Retpoline

Perhaps it's easier to backport Retpoline first? Provide it doesn't depend on some precursor commits from KPTI... At least we can fix the userspace first with the new compiler now...

[theLOICofFRANCE]

@biergaizi The initial retpoline support is in 4.9.77. See diff.

There is a bit of breakage with "pax_indirect_call" and "pax_ret" but nothing comparable with KPTI's breakage. Conflicts: _arch/x86/entry/entry_64.S arch/x86/entry/entry_32.S arch/x86/crypto/aesni-intel_asm.S arch/x86/kernel/irq_32.c arch/x86/kernel/mcount_64.S arch/x86/crypto/camellia-aesni-avx-asm_64.S arch/x86/crypto/camellia-aesni-avx2-asm_64.S arch/x86/lib/checksum32.S

[biergaizi]

arch/x86/crypto/aesni-intel_asm.S
arch/x86/crypto/camellia-aesni-avx-asm_64.S
arch/x86/crypto/camellia-aesni-avx2-asm_64.S
arch/x86/lib/checksum_32.S

These conflicts are trivial, just some indirect replacements.

arch/x86/entry/entry_64.S
arch/x86/entry/entry_32.S
arch/x86/kernel/irq_32.c
arch/x86/kernel/mcount_64.S

Don't know if PaX/grsecurity introduced any extra assembly code here. But probably still easy to backport. 😄

Haven't looked into it, still don't know if any code introduced by retpoline depend on some precursor commits from KPTI. Probably not?

[theLOICofFRANCE]

Haven't looked into it, still don't know if any code introduced by retpoline depend on some precursor commits from KPTI. Probably not?

You can add retpoline support without adding KPTI with the commit https://github.com/torvalds/linux/commit/76b043848fd22dbf7f8bf3a1452f8c70d557b860 (patch here). You'll just have to fix a few Hunks by hand.

However, it does not appear to be compatible with the following optimizations: export CONCURRENCY_LEVEL=3 export KCFLAGS="-O3 -march=x86-64" export KPPCFLAGS="-O3 -march=x86-64"

[miroR]

You can add retpoline support without adding KPTI with the commit torvalds/linux@76b0438 (patch here).

Truth is, IIUC, then AMD64 owners would have a lot less to worry, and grsecuoff kernels would already make sense, for them, installing and running... (Truth is also, we're a minority...) Of course, minipli, and others here, might know better, and I, as is due, just continue to patiently wait.

[klikevil]

Do we have a timeline for getting the generic retpoline patches in to this?

[theLOICofFRANCE]

@klikevil My patch which is not perfect, but it's better than nothing : retpoline_for_unofficial_grsec-4.9.74.patch

[miroR]

HacKurx wrote:

My patch which is not perfect, but it's better than nothing : retpoline_for_unofficial_grsec-4.9.74.patch

I recompiled 4.9.74 with your retpoline patch and uploaded it to: https://www.croatiafidelis.hr/gnu/deb/linux-image-4.9.74-grsecunoff-180203-22/

It is being tested here: $ uname -r 4.9.74-unofficial+grsec180203-22

and it is likely to be tested by others from Debian/Devuan,

[miroR]

I thought I'd open a new issue and name it: enable amd64-microcode for 4.9.74 retpoline-patched? but I'm not sure it would be worth it. So asking you guys here about it first. Installing: https://packages.debian.org/sid/amd64-microcode would not get me the microcode for 4.9.74 grsecunoff- and retpoline- patched. And here's why: In the list of files: https://packages.debian.org/sid/amd64/amd64-microcode/filelist there's the /usr/share/doc/amd64-microcode/changelog.gz (but I couldn't fine it online), which says:

amd64-microcode (3.20171205.1) unstable; urgency=high

• New microcode updates (closes: #886382): sig 0x00800f12, patch id 0x08001213, 2017-12-05 Thanks to SuSE for distributing these ahead of AMD's official release!
• Add IBPB support for family 17h AMD processors (CVE-2017-5715)
• README: describe source for faml17h microcode update
• Upload to unstable to match IBPB microcode support on Intel in Debian unstable.

• WARNING: requires at least kernel 4.15, 4.14.13, 4.9.76, 4.4.111 (or a backport of commit f4e9b7af0cd58dd039a0fb2cd67d57cea4889abf "x86/microcode/AMD: Add support for fam17h microcode loading") otherwise it will not be applied to the processor.

  -- Henrique de Moraes Holschuh hmh@debian.org Mon, 08 Jan 2018 12:19:57 -0200

Pls. note that fat WARNING in bottom of the above paste... And this is asking for help. I, myself am extremely unlikely to be able to help much with the code at all, not yet at least (even though I will try and look it up). But I'm not asking just for me. If this is fixed, I the packages above are likely to make sense, if only for AMD owners.

Because that's the part that's now most missing for security, retpoline-patch being there. ( Apparently, the compiler and missing toolchain for retpolines do not seem to be such great security issue, according to: https://github.com/torvalds/linux/commit/b8b9ce4b5aec8de9e23cabb0a26b78641f9ab1d6 )

[miroR]

Maybe not extremely unlikely. I just took the same patch, actually I git clone'd the linux.git and did:

git diff e3811a3f74bd1ad773667b78323f396166891f3a..f4e9b7af0cd58dd039a0fb2cd67d57cea4889abf

and diff did it complaining very little:

miroR@some-amd-machine:/somewhere/src/linux-4.9.74$ patch -p1 < \ ../microcode_amd_c_fam17h.patch patching file arch/x86/kernel/cpu/microcode/amd.c Hunk #1 succeeded at 592 (offset 122 lines). Hunk #2 succeeded at 604 (offset 122 lines). miroR@some-amd-machine:/somewhere/src/linux-4.9.74$

I'll tell you if I get the microcode to load, but it'll take longer. I compile with only one thread (no -j2 or -j4, just default -j1 for make), feels more reliable, I want it to really work for others too... I'm compiling the all-modules-compiled but separately from kernel, modules proper, for any hardware.

[miroR]

microcode_amd_c_fam17h.txt

Would not accept the extension .patch. But it should be named: microcode_amd_c_fam17h.patch

[klikevil]

Thanks apparently my attempt at it didn't touch several files and it was failing to boot (just hanging). Attempting compiling yours in now.

net/netfilter/xt_RATEEST.c: In function ‘xt_rateest_put’: net/netfilter/xt_RATEEST.c:66:26: error: ‘struct xt_rateest’ has no member named ‘rate_est’ gen_kill_estimator(&est->rate_est); ^~ net/netfilter/xt_RATEEST.c:66:3: error: too few arguments to function ‘gen_kill_estimator’ gen_kill_estimator(&est->rate_est); ^~~~~~~~~~~~~~~~~~ In file included from net/netfilter/xt_RATEEST.c:15:0: ./include/net/gen_stats.h:59:6: note: declared here void gen_kill_estimator(struct gnet_stats_basic_packed *bstats, ^~~~~~~~~~~~~~~~~~ net/netfilter/xt_RATEEST.c: In function ‘xt_rateest_tg_checkentry’: net/netfilter/xt_RATEEST.c:135:50: error: ‘struct xt_rateest’ has no member named ‘rate_est’ ret = gen_new_estimator(&est->bstats, NULL, &est->rate_est, ^~ net/netfilter/xt_RATEEST.c: At top level: net/netfilter/xt_RATEEST.c:165:3: error: ‘struct xt_target’ has no member named ‘usersize’; did you mean ‘targetsize’? .usersize = offsetof(struct xt_rateest_target_info, est), ^~~~~~~~ targetsize

[miroR]

Klikevil wrote:

Thanks apparently my attempt at it didn't touch several files and it was failing to boot (just hanging). Attempting compiling yours in now.

net/netfilter/xt_RATEEST.c: [...]

I can't understand much of it. minipli, HackKurx and others might, but you should try to introduce newlines in your paste (look it up on the web, it never even got your paste into email, Github's fault <- if others got what I got)...

Corrections to my previous post (which can't be corrected in the email already sent 20 hours ago):

--- /Cmn/mr/Github_180205_minipli_COR_1.txt 2018-02-05 16:41:48.514779652 +0000 +++ /Cmn/mr/Github_180205_minipli_COR_2.txt 2018-02-05 16:43:27.037541053 +0000 @@ -1,3 +1,3 @@ -I compile with only one core +I compile with only one thread (no -j2 or -j4, just default -j1 for make)

-compiling the all-modules-compiled in kernel +compiling the all-modules-compiled but separately from kernel, modules proper, for any hardware

Yes, it's likely Github's fault, I tried to add backticks to start and end of the primitive diff just above, (under escape on many keyboards, but that pronto removed newlines...)

[miroR]

However, for my AMD machines, it does not sort the microcode... OTOH, neither can new microcode get with the latest:

# apt-cache policy linux-image-4.14.0-0.bpo.3-amd64 linux-image-4.14.0-0.bpo.3-amd64: Installed: 4.14.13-1~bpo9+1 Candidate: 4.14.13-1~bpo9+1 Version table: 4.14.13-1~bpo9+1 100 100 tor+https://pkgmaster.devuan.org/merged ascii-backports/main amd64 Packages 100 tor+https://pkgmaster.devuan.org/merged testing-backports/main amd64 Packages 100 /var/lib/dpkg/status

(no backticks for the above explained reason, and shows still poorly: nothing should be striken-through... if others see like I see: newlines removed when backticks). I couldn't get new microcode mitigating Vuln 2 to load for my Athlon based machines.

# cat /proc/cpuinfo | grep 'model name|microcode' model name : AMD Athlon(tm) 64 X2 Dual Core Processor 4200+ microcode : 0x4d

And that's the proc on the old machine. Will paste from the new machine... In a while.

[miroR]

On the new machine: # cat /proc/cpuinfo | grep 'model|microcode' model : 4 model name : AMD Phenom(tm) II X4 965 Processor microcode : 0x10000c8 [and 4 times so] # Retpolines on only our 4.9.74, meltdown mitigation on only Debivuan's 4.14.13-1bpo9+1, but Debivuan (Debian/Devuan) non support compiler toolchain (Gentoo supports, among the very few, having gcc-7.3 available, but I'm not running Gentoo currently), and microcode not supported in either, see for yourself (if I manage to post images of spectre-meltdown-checker.sh runs):

[miroR]

Sorry, I mistakenly claimed Debian/Devuan has retpoline support, it doesn't. Correcting it online.

[miroR]

@Klikevil, maybe just one thing, regarding your post above:

Thanks apparently my attempt at it didn't touch several files and it was failing to boot (just hanging). Attempting compiling yours in now.

net/netfilter/xt_RATEEST.c: [...]

If you used my https://github.com/miroR/grsec-dev1-compile script, and maybe patched it not at the correct stage, open an issue there, and I might try and find time to improve the script. In case it is something more complex, I'm not of much use.

[miroR]

There I have received (sent:

Date: Thu, 15 Feb 2018 13:58:59 +0000 (UTC)
From: klikevil <notifications@github.com>

[copied from the email headers]) email, which said in bottom: https://github.com/minipli/linux-unofficial_grsec/issues/26#issuecomment-365934833 but it does not open any post, only the issue 26. Just to state the fact of it from my part. Don't know why, and whether others see (i.e. do not see) like me.

[g3ngr33n]

@HacKurx

Thanks for the retpoline patch, far better than my

One question, how long do you evaluate the life of this kernel before to be deprecated ("less" secure than the mainline) ?

I'm planing to stick with this for the next few years (~ 5 years), do you think this is a reasonable plan ?

[klikevil]

@miroR i had apparently made a lot of changes to the source, i fetched the zip reapplied the patch and rebuilt w/o issue

edit:

Finally got the system up and running with @HacKurx 's retpoline patches, if you are on redhat 7 you'll have to literally rebuild systemd from source because systemd sucks and misdetects the architecture.

[root@dnocc1337 ~]# dmesg |& grep -i spect [ 0.004798] Spectre V2 mitigation: Mitigation: Full generic retpoline

[root@dnocc1337 ~]# uname -a Linux dnocc1337 4.9.74-retpoline+pax-hardened #12 SMP Tue Mar 20 17:43:45 GMT 2018 x66_64 x66_64 x66_64 GNU/Linux

Other update: see my post below, you don't have to rebuild systemd just comment 2 lines from the patch that arch/x86/kernel/cpu/bugs.c /IF/ you run in to an issue with systemd not detecting the correct architecture.

[klikevil]

@HacKurx

After some debugging and several recompiles i figured out why my kernel was booting as x66_64

/*      init_utsname()->machine[1] =
                '0' + (boot_cpu_data.x86 > 6 ? 6 : boot_cpu_data.x86);
*/
        char **ragedebug_info = '0' + (boot_cpu_data.x86 > 6 ? 6 : boot_cpu_data.x86);
        alternative_instructions();
        pr_info("What the fuck is this string? : %s\n", &ragedebug_info);

[ 0.013920] What the fuck is this string? : 6

 init_utsname()->machine[1] =
            '0' + (boot_cpu_data.x86 > 6 ? 6 : boot_cpu_data.x86);

That was making utsname()-> machine[1] come back as x66_64 >.>

She is better now though

[root@dnocc1337 linux]# uname -a Linux dnocc1337 4.9.74-retpoline+pax-hardened #28 SMP Thu Apr 5 19:47:23 GMT 2018 x86_64 x86_64 x86_64 GNU/Linux

This is in arch/x86/kernel/cpu/bugs.c

[biergaizi]

Good to know that RedHat just invented the x66 architecture :joy::joy:

[miroR]

I like the enthusiasm @klikevil , but I'm not advanced. When it comes to C code (which I can edit and use only basic stuff of yet), I need hours to maybe figure out... (And I'm sure some among the other followers would benefit from the... read on)... Pls. some guidance, where that code is, and as much as possible of the details...

I haven't compiled grsec since almost two months now, eagerly awaiting a savior, that it be @minipli or anybody else, to save, essentially GNU/Linux to remain feasible to be modified to become secure (which it is not as it comes out by the dictator and his lieutenants)...

And so... not having a few hours to recall and refresh my previous work and understanding, pls. a schoolish, slow, reminder... if possible, of course...

[klikevil]

@miroR The offending lines can be found here.

Only do this if you run in to the same issue that i was having where it was detecting the architecture as x66_64 instead of x86_64 despite being compiled with a proper CHOST/ARCH etc set.

Comment out line 42 and 43 and build again and it will detect the proper architecture. Why in God's name it's detecting it as that I don't have an answer but when i made it print the string to the kernel ring buffer with

char **ragedebug_info = '0' + (boot_cpu_data.x86 > 6 ? 6 : boot_cpu_data.x86);
...
pr_info("What the fuck is this string? : %s\n", &ragedebug_info);

it printed a 6 and so i just took a shot in the dark and commented out those lines that fuck with init_utsname->()machine[1] (had a bit of help in figuring out what that was from @HacKurx by getting me pointed in the right direction as to what actually determines uname lol). Either way solved my problem :D . Also it is important to note I was using gentoo's gcc which essentially pretends it's a cross compiler and i had it in a major non-standard path and a bunch of other unique oddities.

[theLOICofFRANCE]

@klikevil Well done. Do you have the same problem on a vanilla kernel 4.9.74 ?

[theLOICofFRANCE]

@minipli interesting for after spectre: warn about potential spectre vulnerabilities.txt

[klikevil]

Been out of work for personal reasons for a bit so I haven't tested the vanilla kernel on the same vm if I am able to make it back in the forseeable future I'll let you know @HacKurx

[miroR]

@klikevil

@miroR The offending lines can be found here.

A very belated thanks. ( Unfortunately, I'm not up to even my own speed at this work currently. Was busy for too long in non-computing matters. Will try to catch up... )