Closed zenmonkeykstop closed 3 years ago
Still awaiting hardware - deferring for current sprint/1.5.0, we'll see where we are mid-sprint.
Hardware obtained. For the 8/20-9/2 sprint, @zenmonkeykstop committed to step through and document the install procedure on Ubuntu 16.04, in the second half of the sprint.
Initial testing done against NUC10i5FNH, Ubuntu 16.04.6 server install:
Discussed in sprint planning today. We've got NUC8s on order and will test those in late September. The NUC7s are still available in retail (including online: Amazon, Newegg, etc.) but getting stale.
I have a NUC8 and have (briefly) gone over the initial testing with @zenmonkeykstop. I'll plan to step through and document results as well during the coming sprint.
Per above @rocodes will do some testing on the NUC8 during the sprint, but we're not aiming to QA 1.6.0 on it yet.
Partial update on NUC8 hardware testing:
[x] Ubuntu 16.04 LTS installed successfully. NIC was not recognized during installation wizard, nor was the USB ethernet adapter (Tripp-Lite USB 3.0 Adapter U336-000-GBW, and I also tried an Asus adapter I had lying around). However, once Ubuntu was installed, a quick lspci
indicates that the ethernet adapter device is actually detected, but the interface wasn't configured, so this is quickly fixable and the USB ethernet adapter is usable in Ubuntu. (I didn't spend any time trying to figure out if it was possible to get the NIC recognized in vanilla Ubuntu since I wanted to focus on grsec, but I could do).
[ ] securedrop-grsec-4.14.188 kernel installation not completed. Installing the linux-headers-4.14.188-securedrop-grsec
and linux-image-4.14.188-securedrop-grsec
occurs without issue, but the securedrop metapackage requires a fallback kernel, and in the apt repo, we only seem to have linux-image-4.14.175
but not the headers
package, so the installation of 4.14.175 is not successful. Therefore, the securedrop-grsec-4.14.188
metapackage cannot successfully be installed (it complains about a missing dependency).
[ ] Booting into the grsec 4.14.188 kernel from the GRUB menu, it currently hangs. This sounds like an issue similar to what @zenmonkeykstop encountered historically with the NUC7s, which required a kernel update to mitigate. Conducting further diagnostics to gather more information.
[x] Ubuntu 20.04 Focal installs and boots successfully and ethernet connection is detected successfully without the use of a USB adapter.
[ ] Compiling and installing a grsec-5.4.68
kernel (back on Xenial on the NUC8) is also unsuccessful, and hangs at initrd
in the same way the 4.14.188 kernel did. (Thanks to @conorsch for much assistance with this and other steps!) Also under conor's watchful eye, the problem persists after pulling in the two Kaby Lake patches from https://github.com/freedomofpress/ansible-role-grsecurity-build/tree/main/files and recompiling this kernel. I can confirm that the kernel boots on a Xenial-based qube, so it appears to have been compiled correctly.
Unfortunately, even after building a new 5.4.68 kernel based on a new kernel config provided by @emkll today, I'm unable to boot into it on the NUC8, though I can boot into it on a Xenial qube, same as with the kernel I built yesterday.
I do notice the following console output when installing the kernel:
[...] Ignoring non-Xen Kernel on Xen domU host: vmlinuz-5.4.68-grsec
Some quick searching tells me this is grub trying to be clever, however, a) the kernel still shows up in the grub menu and b) this message is also displayed in the Xenial qube when installing the kernel, which then goes on to boot just fine.
Stay tuned folks... Sorry it's not a happier update.
@rocodes and @zenmonkeykstop will continue their work on this in the 10/1-10/15 sprint, aiming for no more than 16 person hours total time spent during this sprint. Our goal right now is to get the NUC8s running on a 4.14 series grsec kernel, so we can potentially ship support in a point release or 1.7.0. Additional 5.4 testing is out scope for now, but we'll revisit 5.4+ kernel support as part of the Focal migration (#4768) at the latest.
At @emkll's suggestion I tried compiling a 4.14 series kernel without grsec patches to see if it would boot on the NUC8, thereby helping us determine whether the issue is with the grsec patches or not. Unfortunately, a 4.14.188 kernel without grsec patches does boot successfully on the NUC8, while the 4.14.188 kernel from the apt repo that includes grsecurity patches does not.
But wait there's more! In both cases (the kernel that successfully boots and the one that does not), during installation I see the following warning:
W: Possible missing firmware /lib/firmware/i915/kbl_guc_ver9_14.bin for module i915
W: Possible missing firmware /lib/firmware/i915/bxt_guc_ver8_7.bin for module i915
However, installing the appropriate drivers has not enabled the grsec kernel to boot.
tested on the NUC8i5BEK as provided by @rocodes:
dpkg
and rebooted: boot frozen after loading initial ramdisk
debug ignore_loglevel earlyprintk=efi,keep
to grsec kernel parameters for more info - found boot freezes after ACPI: 8 ACPI AML tables successfully acquired and loaded
mds=full,nosmt noefi
flags: boot successful!tried out an SD install with the dongle:
./securedrop-admin install
sshed to app server and switched default interfaces in /etc/network/interfaces
With a working USB adaptor, it looks like the NUC8s are a viable install option, though it would be worth going through a full install run as above to verify and document the process.
(One thing to note, paxtest tests pass but one meltdown test (for the Foreshadow SGX vulnerability mitigation) fails.)
(Per most recent findings it sounds like only docs additions may be needed fro NUC8 support, so migrated to securedrop-docs repo.)
IMO this should move back to SD core - NUC10 support will require a kernel build.
(Retitled and checklist updated for clarity.)
One observation from NUC10s is that the BIOS instructions now recommend downloading a CAP file for the F7 update method, not a BIO file as stated in our docs. I've added a checkbox to the epic to ensure that we update this as warranted.
Except for the above minor detail, no issues migrating an Ubuntu 16.04 instance on Mac Minis to Ubuntu 20.04 on NUC10s :tada:
The NUC8i5s in our hardware docs were officially discontinued on Oct 2020 (https://www.intel.ca/content/www/ca/en/support/articles/000016234/intel-nuc.html), although they'll get security updates til Oct 2023. A further round of NUC8s were discontinued Feb 2021, getting security updates til early 2024.
Would be great to add NUC10s (and LIbrems?) to our hardware recommendations to get a bit ahead of this.
I ran the testinfra tests from Conor's #5848 branch (i.e. including paxtest
) using USE_FOCAL=1 ./securedrop-admin --force verify
on my NUC10. The results were as expected (i.e. only successes or xfail), full details below. Next up is the meltdown check.
meltdown results on NUC10 looking good, detailed output below. Seeing the same test failure reported in #5040, likely for the same reason (also seeing the grsec denied write to CPU MSR
directly caused by the script in syslog
).
Beyond that, I see three grsec
lines in syslog
/ kern.log
on boot, for setting time via systemd
and for setting rwxmap_logging
to 0
and grsec_lock
to 1
, as expected. Nothing else notable in logs, let me know if you'd like me to poke around further.
Description
7-series NUCs were discontinued in mid-April and are no longer officially available from Intel (tho probably still sporadically available at retail for a while). The recommended hardware list should be updated to include NUC models that can be reasonably expected to be available long-term. Given that 10-series NUCs are now available, it may make sense to add both 8-series and 10-series NUCs to recommendations.
The last update had issues with Ethernet chipset support in the kernel. The 8- and 10-series NUCs use the same chipset (Intel i219-v) as the 7-series, so this will probably work, but hardware testing is still required to be safe..
Tasks
User Research Evidence
(Feedback from folks looking into setting up instances)