Support for EFI boot

[marmarek]

Reported by marmarek on 4 Feb 2014 02:53 UTC EFI is more and more popular, soon there will be some machines with EFI-only boot, so we need badly support for it. Adding such support may be non-trivial, because Xen 4.1 do not support direct EFI boot (AFAIR it is supported in >=4.2 or even >=4.3). But perhaps some workaround like grub-efi can be used.

Migrated-From: https://wiki.qubes-os.org/ticket/794

[ideologysec]

I have an "EFI\qubes\xen-4.6.0.efi" boot option in rEFInd, but it just kicks me back to the rEFInd boot menu very very quickly - so fast I have to take multiple photos to even see the error.

Starting xen-4.6.0.efi Using load options ' ' Xen 4.6.0 (c/s) EFI loader Allocation failed for xen.cfg: Out of resources

Any configuration files or parameters I should be or can be editing?

[marmarek]

Installation image with redind instead of grub as boot loader: http://ftp.qubes-os.org/~marmarek/Qubes-20151226-x86_64-DVD-refind.iso http://ftp.qubes-os.org/~marmarek/Qubes-20151226-x86_64-DVD-refind.iso.asc

Not tested thoroughly, but bug about 'user' username should be fixed there too. (signed by my code signing key)

[marmarek]

It was built from https://github.com/marmarek/qubes-installer-qubes-os/tree/efi-refind-wip-live

[ideologysec]

Downloaded, installed. Starting the installer works without any extra hassle on my part - gives me a textual rEFInd window with options, I select "install," away it goes.

Still can't boot afterwards, no matter what I try - the default reboot after installation still drops me to a grub shell, and using rEFInd booting externally leads to the same outcomes as above.

One thing I did notice is that xen.cfg is 0 bytes in size. Should there be any information in here?

[vantonio]

@marmarek I tried your latest ISO with rEFInd boot manager, but still have the same result. Booted in rEFInd text mode, frozen at the screen with 4 penguins.

[marmarek]

Strange, it should be something like this:

[global]
default=4.1.13-7.pvops.qubes.x86_64

[4.1.13-7.pvops.qubes.x86_64]
options=loglvl=all
kernel=vmlinuz-4.1.13-7.pvops.qubes.x86_64 root=/dev/mapper/core3-root vconsole.font=latarcyrheb-sun16 rd.lvm.lv=core3/root rhgb quiet
ramdisk=initramfs-4.1.13-7.pvops.qubes.x86_64.img
[3.19.8-100.fc20.x86_64]
options=loglvl=all
kernel=vmlinuz-3.19.8-100.fc20.x86_64 root=/dev/mapper/core3-root vconsole.font=latarcyrheb-sun16 rd.lvm.lv=core3/root rhgb quiet
ramdisk=initramfs-3.19.8-100.fc20.x86_64.img

[ideologysec]

Progress!

I had to copy your example xen.cfg to my xen.cfg, and rename xen-4.6.0.efi to just "xen.efi", and use rEFInd on a usb drive to select the proper file, but it boots!

...until it gets to the screen where it asks for the disk password with a very small progress bar on the bottom, and then won't accept any input. I've tried internal and external USB keyboards, mice, plugged in before and after booting, and nothing is accepted. I don't know if it's a consequence of the newness of the hardware, or the stage at which input devices are loaded - and I can't get to a console because I don't have any input!

Update 1: Seems to be related to https://groups.google.com/forum/#!topic/qubes-users/sU5UY2ZPI3c I do have two graphics cards on this laptop, and one of them is indeed an AMD Radeon (R9 M370X 2GB).

Update 2: removed "rghb quiet" from the boot option for the 4.1 vmlinuz option in xen.cfg, and it boots to a text screen asking me for the luks disk password (after several lines about initializing the radeon driver). no input is accepted even at this screen, though - internal or external usb keyboard. not sure if any characters or asterisks should show up, but they don't. Not sure why; keyboard worked in the installer...

[marmarek]

...until it gets to the screen where it asks for the disk password with a very small progress bar on the bottom, and then won't accept any input. I've tried internal and external USB keyboards, mice, plugged in before and after booting, and nothing is accepted.

Does any of those devices is PS/2, or all are USB? In case of USB, there may be missing driver...

[ideologysec]

Unfortunately, this being an Apple laptop, the only keyboards I have are USB - both internal and external. I thought it might be related to the old bug where Mac keyboards didn't work in the installer, but that was closed a long time ago, and most importantly, the internal keyboard worked during install. Possibly just not loading the USB driver early enough.

Oh well. Don't want to take up your time with this unecessarily; probably going to end up getting PC hardware to run Qubes for compatibility and security.

[ideologysec]

Purchased the Dell XPS 15" 9550 from Microsoft, bought the i7 configuration. On paper looks ideal, supports vt-d and TPM, and has plenty of RAM and processor power.

http://www.microsoftstore.com/store/msusa/en_US/pdp/Dell-XPS-15-9550-Signature-Edition-Laptop/productID.326871600

Attempting to boot Qubes in EFI mode fails to load X, likely due to the noveau driver not working. Installing via the textual script works up until actual installation, when it throws the error

(anaconda : 1184): Gdk-ERROR \ : error : XDG_RUNTIME_DIR not set in the environment . Pane is dead

I asked on the Google Groups thread relating to this, and it seems like if I enable i915 preliminary support and disable noveau, it will work. (kernel options 'i915.preliminary_hw_support=1 rd.blacklist.drivers=nouveau nouveau.modeset=0')

I can do this in legacy mode, but no bootable partitions are created afterwards - I get an error after restarting that the UUID of the /boot partition can't be found, and gives me a GRUB rescue shell. Also looking into the partition after the fact with a Fedora LiveUSB shows another xen.cfg that is 0 bytes - completely empty. Copying a working xen.cfg to that partition manually does not allow for successful boot in EFI or Legacy/BIOS mode.

Is there any way to edit the options passed to the xen kernel during installer startup in EFI mode? attemping to edit the boot options just gives me the "chainloader" line, and I can't edit the xen.cfg on the installer USB because it appears as a read-only 9660 filesystem.

I also don't have an old copy of 3.1rc1 where I could edit the USB installer, but if anyone does, I'd be happy to test it. or an updated/experimental build. Or instructions on how to regenerate xen.cfg during the install or repair process so this installation will boot correctly.

[marmarek]

What exactly have you selected in disk layout, which would result in that unbootable system? Just defaults, or some custom partitions?

As for editing boot options - if you use USB as installation media (probably you do), you can mount the second partition, which has FAT filesystem, and edit xen.cfg there - there is kernel command line. You can do it even booting from the same USB stick (in legacy mode).

Also, try with just i915.preliminary_hw_support=1 (but no disabling nouveau). If that's enough, we may consider enabling it by default.

[ideologysec]

1) found the second partition on the USB installer, mounted. not sure how I missed it before. I used Automatic partitioning during install, selected LVM for the main partitioning scheme and encrypted my data. I gave it the whole 500 GB SSD (which is a PCIe NVME drive), and it created three partitions.

The Qubes installer now boots, and successfully completes installation.

2) But not without all three strings I mentioned above. Merely enabling i915.preliminary_hw_support=1 is not sufficient. I need to also enable rd.blacklists.drivers=nouveau nouveau.modeset=0 for the graphical installer to boot.

3) Qubes does not reboot correctly after installation. I added all three boot options to xen.cfg after the fact, and manually created a UEFI boot entry in the system BIOS that pointed to xen-4.6.0.efi as the default boot. That worked.

4) However, I think because the kernel in domU is an older 3.19, it doesn't support the intel graphics adapter very well. Rebooting after install, I get a flickering/flashing screen with vertical lines during the setup, and eventually when the final progress bar appears it switches to a warm black screen and stays there, no matter how long I leave it.

is there any way to update the kernel in domU post install? I thought about trying the 4.1pvops kernel, but I didn't know how well that would work.

[marmarek]

On Fri, Jan 15, 2016 at 11:30:43PM -0800, Aktariel wrote:

1) found the second partition, mounted. not sure how I missed it before. Thank you. The Qubes installer now boots, and successfully completes installation.

2) But not without all three strings I mentioned above. Merely enabling i915.preliminary_hw_support=1 is not sufficient. I need to also enable rd.blacklists.drivers=nouveau nouveau.modeset=0 for the graphical installer to boot.

Hmm, we may provide additional option like "Boot with nouveau driver disabled". Unfortunately current Xen version in UEFI mode doesn't provide a way to pass kernel parameters other than xen.cfg, so not possible to edit it directly from bootloader.

3) Qubes does not reboot correctly after installation. I manually created a UEFI boot entry in the system BIOS that pointed to xen-4.6.0.efi as the default boot, and that worked. I added

There should be also xen.efi created (and boot entry pointing at it), wasn't there? Anyway those additional boot parameters (disabling nouveau, i915.preliminary_hw_support=1) currently also needs to be manually added. After installation, you can switch to text console (ctrl+alt+f2) and edit xen.cfg from there - at that stage EFI partition should be still mounted.

4) However, I think because the kernel in domU is an older 3.19, it doesn't support the intel graphics adapter very well. rebooting after install, I get a flickering/flashing screen with vertical lines during the setup, and eventually when the final progress bar appears it switches to a warm black screen and stays there, no matter how long I leave it.

Kernel in domU should have nothing to do with graphics - it doesn't have access to it.

Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?

[ideologysec]

There should be also xen.efi created (and boot entry pointing at it), wasn't there? Anyway those additional boot parameters (disabling nouveau, i915.preliminary_hw_support=1) currently also needs to be manually added. After installation, you can switch to text console (ctrl+alt+f2) and edit xen.cfg from there - at that stage EFI partition should be still mounted.

It does appear to be created. I have not attempted to boot from it; will do so. What happens when I reboot immediately after installation is that the UEFI bios cannot find any bootable volumes. I have to enter setup, manually create a UEFI boot option and point it to the xen-4.6.0.efi file, and then it boots.

I've been editing xen.cfg post-install with a Fedora 23 LiveUSB and gedit, which is working fine. Good to know I can do it post-install.

Kernel in domU should have nothing to do with graphics - it doesn't have access to it.

which xen.cfg should I be editing? the only one I can find has two lines for 4.1 pvops kernels, and one 3.19 entry for domU. is there a separate config for xen itself? should I be adding all three graphics kernel options to all three lines?

[marmarek]

On Sat, Jan 16, 2016 at 03:46:07PM -0800, Aktariel wrote:

There should be also xen.efi created (and boot entry pointing at it), wasn't there? Anyway those additional boot parameters (disabling nouveau, i915.preliminary_hw_support=1) currently also needs to be manually added. After installation, you can switch to text console (ctrl+alt+f2) and edit xen.cfg from there - at that stage EFI partition should be still mounted.

It does appear to be created. I have not attempted to boot from it; will do so. What happens when I reboot immediately after installation is that the UEFI bios cannot find any bootable volumes. I have to enter setup, manually create a UEFI boot option and point it to the xen.efi file, and then it boots.

Check logs in /var/log/anaconda (in installed system) for any errors related to efibootmgr. It should add xen.efi boot option...

which xen.cfg should I be editing? the only one I can find has two lines for 4.1 pvops kernels, and one 3.19 entry for domU. is there a separate config for xen itself? should I be adding all three graphics kernel options to all three lines?

At the top you can see what is default section - you need to edit only that section (but shouldn't harm editing all of them).

Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?

[ideologysec]

At the top you can see what is default section - you need to edit only that section (but shouldn't harm editing all of them).

The default section ("Global") is for the 3.19 kernel (and adding i915 hw support and disabling nouveau still results in crazy flickering during the post-install user & vm creation steps). Does that have support for the new Intel hardware? It looks like the installer is booting a newer 4-series kernel than what xen.cfg is using for day-to-day booting. What kernel is Xen attempting to boot/what does it use for graphical management?

[marmarek]

On Sat, Jan 16, 2016 at 05:02:11PM -0800, Aktariel wrote:

At the top you can see what is default section - you need to edit only that section (but shouldn't harm editing all of them).

The default section ("Global") is for the 3.19 kernel (and adding i915 hw support and disabling nouveau still results in crazy flickering during the post-install user & vm creation steps). Does that have support for the new Intel hardware? It looks like the installer is booting a newer 4-series kernel than what xen.cfg is for day-to-day booting. What kernel is Xen attempting to boot/what does it use for graphical management?

Xen uses whatever is set in default section, so if you have 3.19 there, you need to change that to 4.1 to get newer graphics driver.

Actually this looks like a bug in the installer - it should set default to be the same as the one used to install the system... #1650

Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?

[ideologysec]

The only other kernel image looks like the pvops one. It's alright to use that?

[marmarek]

On Sat, Jan 16, 2016 at 06:33:55PM -0800, Aktariel wrote:

The only other kernel image looks like the pvops one. It's alright to use that?

Yes.

Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?

[noseshimself]

On 01/17/2016 03:12 AM, Marek Marczykowski-Górecki wrote:

Actually this looks like a bug in the installer - it should set default to be the same as the one used to install the system... #1650

While we're at discussing the installer: It should try to detect if CSM is turned on and remind the user to turn it off if possible; on certain machines it succeeds installing Qubes (tunning on the EFI frame buffer driver and hits a wall on the next reboot hard enough to drop dead). Or at least add a big fat warning. Blinking.

Achim

[marmarek]

On Thu, Jan 21, 2016 at 04:03:04AM -0800, noseshimself wrote:

While we're at discussing the installer: It should try to detect if CSM is turned on and remind the user to turn it off if possible; on certain machines it succeeds installing Qubes (tunning on the EFI frame buffer driver and hits a wall on the next reboot hard enough to drop dead). Or at least add a big fat warning. Blinking.

Do you have an idea how to detect it?

Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?

[noseshimself]

On 01/21/2016 01:07 PM, Marek Marczykowski-Górecki wrote:

On Thu, Jan 21, 2016 at 04:03:04AM -0800, noseshimself wrote:

While we're at discussing the installer: It should try to detect if CSM is turned on and remind the user to turn it off if possible; on certain machines it succeeds installing Qubes (tunning on the EFI frame buffer driver and hits a wall on the next reboot hard enough to drop dead). Or at least add a big fat warning. Blinking.

Do you have an idea how to detect it?

I haven't looked into things too deeply but I always assumed if you had an INT 19 (BIOS boot) handler around after having been booted via EFI there was an activated CSM layer but it seems that this assumption is not valid. I'll ask Rod Smith.

Achim

[ag106428]

Would it be possible to release an official build (otherwise the same as the mainline build) with rEFInd and the edited name to get it booting on tricky systems? I'm currently trying to get Qubes installed on a UEFI system (Intel 5300U, American MegaTrends firmware) fully verified on multiple systems and rEFInd plus file renaming while maintaining that is tricky with the DVD burn I'm using to verify the base ISO...

[ag106428]

Ideally signed by the official release key and on the main site. (Double post because I don't know how to edit)

[HardenedArray]

Hello All:

1. I was not aware of the existence of this (lengthy), painful, thread prior to my documentation of a proposed EFI booting solution. I offer the following comments in the hope that others might find them useful.
2. I can feel your Qubes EFI boot pain: been there, done that, so, now, let's: ManUp!
3. The first, critical, question you must be able to address is:

Three (3). a. (and, again, as previously noted, a most 'heartfelt and warm' welcome/return to GitHub's completely FUBARed list numbering 'solution').

Can I EFI boot, another/any OS, from the exact same HDD I am now trying to boot Qubes from?

Three (3). b. Test, and verify. Any copy of a verified Fed 23 *.iso (not coincidentally, one of the same OSes Qubes rc2 uses) makes an excellent (and recommended) EFI boot test candidate solution.

Three (3). c. If you cannot EFI boot: a hugely popular, known-to-EFI-boot-across-multiple-disparate-systems, and a simply-kick-ass distro, like/similar to Fed 23, why on God's Green Earth would you expect to be able to EFI boot an OS with a currently tiny user-base, likes Qubes? Just to be clear: Fedora/RHEL have supported EFI booting for their millions of users for at least 3 years (in addition to all of the implied: implications). ;D

Three (3). d. Have you, satisfactorily, addressed the query posed in: Three (3). a.?

Four (4). If not, repeat, on another computer: rinse, and re-lather.

Five (5). I have not installed: http://ftp.qubes-os.org/~marmarek/Qubes-20150930-x86_64-DVD.iso so I cannot comment on its viability as a solution.

Six (6). If you are not running/attempting to run: https://mirrors.kernel.org/qubes/iso/Qubes-**R3.1-rc2-x86_64.iso, we can stop all further 'discussion.' Do yourself a favor, and download, verify, and install only this latest released version of Qubes. Qubes-rc2 is known to correctly EFI boot following installation. Furthermore, as far as I can tell, rc2 is very stable, and all operations are working as expected.

Seven (7). If you still experience EFI booting problems, especially those problems involving USB HDD boot issues, consider deploying my Qubes rc2 EFI booting solution, which I have documented at:

https://github.com/QubesOS/qubes-issues/issues/1658

Cheers, and from a very long time user/proponent of: Gentoo Hardened, i2p and Whonix, I recommend:

Do not give up, nor, in, ever. ;-)

HardenedArray

[ideologysec]

Gave up on the dell - 4K support in Qubes is pretty bad (when do we expect KDE 5 to be included?), and the wireless card is poorly supported - refused to boot any time it was enabled, whether assigned to a VM or not.

Went back to attemping to install on my MacBook Pro, but rc2 is even worse - the EFI partition is read only, xen.efi is completely empty, and the second writeable partition has nothing inside the efi folder. The installer still loads and completes successfully, but the resulting system is unbootable.

Are there installer logs or other configuration information that I can provide? I'm pulling down the rEFInd image linked earlier, which will hopefully resolve at least some of the cfg file issues, but as far as a 3.1 release goes, seems like the post-install scripts might need to get fixed.

[vshakel]

Joining this thread and Aktariel as I'm facing the very same issues on my 2015 Macbook Pro. So far I managed to enter passphrase for LVM ( keyboard input is okay) and still trying. If anyone has a clear installation instruction please let me know.

[ideologysec]

vshakel, have you tried disabling radeon driver in kernel options passed via xen.cfg? Also, how did you get keyboard input to work?

I figured out my issues with editing the rc2 xen.cfg, and will test at home and report back.

[vshakel]

Actually I feel like there is no real issue with ATI or keyboard support. It was just working as expected though i can confirm some flickering for like 1 second at the installer's first page when it offers language selection but after a second it redraws all correctly.

I did actually try all the available iso files aka: http://ftp.qubes-os.org/%7Emarmarek/Qubes-20150930-x86_64-LIVE.iso http://ftp.qubes-os.org/%7Emarmarek/Qubes-20151226-x86_64-DVD-refind.iso https://mirrors.kernel.org/qubes/iso/Qubes-R3.1-alpha1.1-x86_64-LIVE.iso https://mirrors.kernel.org/qubes/iso/Qubes-R3.1-rc2-x86_64.iso

I'm going with rEFInd and creating USB images with dd like this:

diskutil list diskutil unmountDisk /dev/disk2 which is my USB drive hdiutil convert -format UDRW -o ./qubes.img ./Qubes-20151226-x86_64-DVD-refind.iso sudo dd if=./qubes.img of=/dev/disk2 bs=1m

The 20151226 iso was performing best of all and I managed to mount the LVM partitions. Still the OS failes to boot after the installation is completed.

My xen-4.6.0.cfg is empty, I'm renaming it and I trying to adapt an example from this thread:

[global] default=xen.efi

[xen.efi] options=loglvl=all kernel=xen.efi root=/dev/sda5/root vconsole.font=latarcyrheb-sun16 rd.lvm.lv=/dev/sda5 i915.preliminary_hw_support=1 rd.blacklists.drivers=nouveau nouveau.modeset=0 ramdisk=initramfs-4.1.13-7.pvops.qubes.x86_64.img

I have chosen to set up LVM partition scheme as I consider it is fairly secure. At the same time I'm not killing all the mac's hsf+ partitions as it is handy to edit xen.cfg from OS X. Also this is my first Mac and I'm still testing it all.

As I understand my root= value is not properly set and I am confused which one should I specify as there appeared many options from HardenedArray - Never gave up and it just takes time ;-)

Here are my available partitions and UIDs where sda1 sda2 sda3 are OS X partitions and all the rest are created with the installer.

Please share your working macbook configs and experience if any. I think i will try rc2 again and then tell you how it goes.

[ideologysec]

Booting RC2, disabled LVM encryption during install to bypass the USB issue - they keyboard is still not working for me to enter password correctly. Wonder what the difference in our config or hardware is.

Here's my "working" xen.cfg. You'll need to change the /dev/mapper line once you figure out what your root device is. As far as disabling the AMD graphics card, you can add i915.preliminary_hw_support=1 rd.blacklist.drivers=radeon radeon.modeset=0 and see how well that works. I'm currently stuck on a verbose boot that won't progress; troubleshooting further.

I do get an error that says " [FAILED] Failed to start Load Kernel Modules"; wondering if that's somehow related.

As far as making images, you shouldn't need to convert ISO images to be used with dd, you can just dd the image direct.

xen.cfg:

[global]
default=4.1.13-8.pvops.qubes.x86_64

[4.1.13-8.pvops.qubes.x86_64]
options=loglvl=all
kernel=vmlinuz-4.1.13-8.pvops.qubes.x86_64 root=/dev/mapper/core3-root vconsole.font=latarcyrheb-sun16 rd.lvm.lv=core3/root
ramdisk=initramfs-4.1.13-8.pvops.qubes.x86_64.img

[3.19.8-100.fc20.x86_64]
options=loglvl=all
kernel=vmlinuz-3.19.8-100.fc20.x86_64 root=/dev/mapper/core3-root vconsole.font=latarcyrheb-sun16 rd.lvm.lv=core3/root rhgb quiet
ramdisk=initramfs-3.19.8-100.fc20.x86_64.img

[vshakel]

marmarek commented on Dec 23, 2015

You should boot xen.efi from rEFInd, not vmlinuz. Not sure if it is listed automatically.

[ideologysec]

That was a reference to me choosing to boot the vmlinuz kernel directly from rEFInd, not a xen.cfg issue. (Which is bad, and won't work). In this case, the vmlinuz line is necessary to tell Xen which kernel to use for Dom0 by default. Basically, rEFInd -> Xen -> vmlinuz 4.1.13 kernel, and I was trying to do rEFInd -> vmlinuz 4.1.13.

Look at the example xen.cfg marmarek posted up for me to use earlier, maybe. That is what has been working for me 100% up until rc2- and the only thing I had to change was the 4.1.13-7 to 4.1.13-8 sections.

[vshakel]

So which release is working for you?

[ideologysec]

None of them. they all hang at "Load Kernel Modules", and finding /dev/mapper/root3, don't ever boot to the graphical loginwindow. (Fedora 23 Live works just fine, none of the same issues - it's using the Gallium 0.4 driver and a much newer kernel - 4.2 or 4.3 even). Are there newer versions of Xen or the 4-series pvops kernel that could be included in Qubes 3.1 release?

I managed to pull the system log; here's some shots.

(currently booting with the above xen.cfg - rghb quiet removed and i915.preliminary_hw_support=1 rd.blacklist.driver=radeon radeon.modeset=0).

[vshakel]

Okay. I managed to make it work. Now I can see the finishing part of installer and all the rest of configuration like whoonix and tor setup options. Unluckily it freezed during networking configuration.

I got my correct lvm volume names after reading fedora docs. I entered "lvm lvdisplay" to dracut shell and there it was. Also if you do ls -al /dev/disk/by-uuid/ and press TAB twice to see the autocomplete it will show you the volume name routing and edit your xen.cfg accordingly. This is my xen.cfg example:

[global] default=4.1.13-8.pvops.qubes.x86_64

[4.1.13-8.pvops.qubes.x86_64] options=loglvl=all kernel=vmlinuz-4.1.13-8.pvops.qubes.x86_64 root=/dev/dm-2 vconsole.font=latarcyrheb-sun16 rd.lvm.lv=qubes_dom0/root i915.preliminary_hw_support=1 rd.blacklist.drivers=radeon radeon.modeset=0 ramdisk=initramfs-4.1.13-8.pvops.qubes.x86_64.img

[3.19.8-100.fc20.x86_64] options=loglvl=all kernel=vmlinuz-3.19.8-100.fc20.x86_64 root=/dev/mapper/core3-root vconsole.font=latarcyrheb-sun16 rd.lvm.lv=core3/root rhgb quiet ramdisk=initramfs-3.19.8-100.fc20.x86_64.img

Going on. Now we are here: https://groups.google.com/forum/#!topic/qubes-users/X5Iomlu_oo4 :)

[HardenedArray]

vshakel and Aktariel,

I am not pointing any fingers here, but since both of you are running MacBookPros, when does this question get asked/answered?:

What percentage of my install, and boot, problems can be attributed to my willing use of Apple's proprietary drivers, and firmware, versus possible code errors, or hardware support issues, due to the relative immaturity of the Qubes OS?

I think trying/completing a 3.1-rc2 install on a known to-be-compatible rig might provide some clarity to that query.

I hope you get lucky, and/or find a solution, but after reading multiple, often-desperate, pleas for help/support from Apple 'PC' users involving many disparate Linux and BSD based OSes, over the years, to be honest, I will not be surprised if you do not.

All the best,

HardenedArray

[ideologysec]

I'm fully aware that the issues I'm running into might be described as a "me problem." Macs are not well supported by Qubes in general. They do use commodity hardware, though, broadcom wireless and radeon or geforce graphics, pci flash modules, etc, and getting Qubes running on as many hardware configs as possible helps broaden the user base.

I have at this point essentially given up what feels like a fool's crusade. I eagerly await the certification of additional laptops by the Qubes team; most of the well supported hardware configurations seem to be either older, or lacking in the power department, neither of which are particularly thrilling.

[HardenedArray]

Hi Aktariel,

I am sorry to read about your conclusion that your efforts may be a 'fool's crusade.'

I am not a Qubes dev, and I have no technical expertise relative to Apple or Mac hardware, so I cannot offer much other than I found these three MacBookPro items on the Qubes HCL at:

https://www.qubes-os.org/compatible-hardware/#hardware-laptops

Apple_Inc. MacBookPro10,2 i5-3210M MBP102.88Z.0106. B03.1211161133 yes yes R2B2 4.1.6.1 3.12.23-1 read more ph145h

Apple_Inc. MacBookPro11,1 i7-4558U Haswell-ULT HD Graphics MBP111.88Z.0138. B14.1501071216 yes yes R2 4.1.6.1 3.12.23-1 wifi does not work William Welton

Apple_Inc. MacBookPro6,2 i7-620M HD Graphics + GT 330M yes no R2B2 No Network, Chipset doesn't support VT-d! read more read more Alex Dubois

There was also a sole item for Apple under the Desktops section:

Apple_Inc. MacMini6,2 i7-3615QM Ivy Bridge HD Graphics MM61.88Z.0106. B03.1211161202 yes yes R2 4.1.6.1 3.12.23-1 Qubes installation does not work, but running an installed system with some tweaks does. Miłosław Smyk

I don't know whether you can glean any useful information from those items. I only wanted to to point them out in the event you were not aware of them.

I also fervently agree with your more general comment that the Qubes devs should, and need, to be doing everything in their power to broaden Qubes' hardware compatibility so we can maximize the size of the potential tester/user base.

I am aware that Apple currently produces many, fine, high-end, powerful machines, and that they also, obviously, possess a base of vocal, rabid AppleFanBoys. :-3

From my POV, Apple had staunchly, and based solely on their own proprietorial-based 'logic', adamantly refused to use Intel CPUs and chipsets for 30+ years. This critical, corporate/Steve Jobs, decision led to innumerable 'compatibility/usability' problems for Apple users, and damn near bankrupted Apple, and I do mean, APPL, on multiple occasions.

However, when Apple finally caved in, or more precisely; when death-spiraling Apple finally was forced, as always, due to a lack of cash, to acknowledge market realities, and was then further forced, as a direct result of Apple's lack of any financial alternatives, to 'welcome' Intel under precisely the same 'logic' Intel had been 'selling' Apple for 30+ years, Apple began: using Intel CPUs and chipsets.

This equally critical, yet bitter-in-the-extreme, corporate decision, both to disavow Apple's entire corporate 'culture and history,' and to begin using, Intel CPUs and chipsets, literally, saved Apple's 'PC' business, and now, as a result, there is no legitimate reason why Linux, BSD, Windows, or any major OS 'family' should not support Apple 'out of the box,' despite Apple's historically-continuous, and current, relatively small 'PC' market share.

Qubes should support new, high-end Apple systems, period.

I have no problem with Qubes not working on, or Qubes not providing support for, any low-end, or previously high-end, but now, old, machine, including Apple, or any of the other hundreds of millions of existing 'rigs' which qualify under this exclusionary view of the computing hardware universe.

That being said, I think it is completely unacceptable, and ultimately, myopic, grossly-negligent and self-defeating, for Qubes to be known/claimed as being: 'uninstall-able,' unboot-able' or unusable by any user of a high-end, recent, 'real-power-enabled-rig' produced by ANY major, global, OEM PC manufacturer. Apple clearly qualifies with several of its products in this, albeit, far smaller, computing hardware category.

In the future, I would like Qubes to disassociate itself of from its HCL 'concept' and issue a far more inclusive 'global vision,' such as proposed in this example:

++++++++++++++++++++++++++++++++++

Qubes Hardware Requirements:

1. A quad-core Intel i3 CPU, or better.
2. 8 Gb of DDR3 RAM, or better.
3. A laptop, desktop or server produced by a major, global, OEM brand within the past year.
4. A machine which natively supports UEFI booting.
5. A Qubes-dedicated, internal SSD/HDD, or an USB external HDD, 100 GiB, or better.

If your computer meets all of those five requirements, we are confident, Qubes R4+ will work on your computer. If you run into installation, or booting, issues, please contact our support forum so we can isolate, and correct, the problem.

If your computer does not meet those five requirements, do not expect Qubes to be install-able, or boot-able. You may get lucky, and find a solution, but do expect to do so, without support.

++++++++++++++++++++++++++++++++++

Note: The absence, above, of long-term, death-spiraling, AMD, was not an oversight, and there are several others in both the HDD, and DRAM 'commodity businesses' facing similar competitive and financial challenges. Those challenges are being massively amplified by the rapid shift in global consumer preference away from 'computers,' and toward smartphones.

I might also note despite this rapid shift toward smartphones, no effective 'competitive response' has yet to have been issued from the far larger 'computer' industry. The economics of 'computing via phone' is one 'thing,' but the economics of staying relevant, by establishing, multiple, clearly superior, 'use cases' is a far more important 'thing.' Hopefully, enough lecturing, for those, who theoretically, carefully, 'care.'

Now, segueing into another, closely-related, 'thing', if you want your 'thing' to succeed, wildly, it is essential to accurately, recall:

Since cavemen/humans first established trading posts, markets have only awarded out-sized profits to those who offered GREATness, and never to those who merely offered goodness.

Need I mention Alphabet (nee Google (aka GOOG)), nor a current, litany/rash of semi-skilled (excluding Zuckerberg), 20+-something-year-old, woefully-inexperienced, embarrassingly immature, multi-hundred-millionaires/billionaires to spark your imagination?

In other words, from a business POV, offering goodness, may, or may not, feed and clothe your family. Offering competitive-GREATness, however, allows you to buy a house, and maybe a few (hundred) villages. How do you want to spend your, constrained, time? ;D

I prefer supporting only hardware offered by competitively-viable, respected, corporations, with sufficient CASH on their balance statements. This POV is also in the best interest of Qubes, as a project with limited resources, and its current, and future, users.

Aktariel, even though my comments are now, hopefully, and obviously, aimed at a much larger audience, as I said earlier, I hope you can identify a working solution for your MacBookPro. The final decision is, of course, yours, alone, to make.

However, I, personally, would not, ever, give up, but, do tread carefully, as guys, like me, are often accused, especially by folks who have run out of, or have been depleted/defeated of reasonable, and plausible, ideas and logic, of 'having issues.'

On an Apple-related extremely positive, reverent, note: I recall Steve Jobs saying it was 'only' those 'having-issues' kind/types of folks who were, and are, 100% responsible, for propelling our World: forward.

Steve, was astutely, 100% correct in his assessment, and summary!

I deeply miss Steve's rare-genius, in addition to his massive Pixar business, and technical, industry-revolutionizing, success, which few even remember, or discuss.

Count em' up ladies, that would be two massive industries, not merely two companies, that one MAN revolutionized, from their foundations, UP!

I do not know Joanna, personally. With this being the case, I am going to guess/assume that she, also, 'has issues.' In fact, I would be far more shocked to learn Joanna was just 'a normal guy.' Who knows, maybe I'll learn something 'valuable' as we move forward. ;D

I only know I want to help Qubes undergo the, often painful, transition from its current, and non-viable, market position of: acceptably-good-for-some-determined-geeks, to: GREATness-for-the-Majority-of-Power-Users!

Does anyone else see, or recall, the parallels to the early years of either Apple, or Intel?

In closing Aktariel, I would not give up, especially if I could get my 'filthy hands' on a friend's MacBookPro, or to sweet-talk some super-hot chick/rep at an AppleStore (tm), employing both approaches 'for testing,' whether in-store, or not. ;D

Cheers,

HardenedArray

[vshakel]

Right now my installer freezes on its very final part where it says we are almost there and that network is being set up. PC users recommend to disable a network adapter using bios but as Mac just doesn't have one. I'm trying rd.driver.blacklist=skge boot option at the moment. Also there is a bug in the installer itself - it only works once after the installation and freezes within seconds if you restart it. Reinstall from scratch is helpful but it takes time. I will try disabling networking with rd.driver.blacklist=skge and maybe this is going to work out.

[HardenedArray]

vshakel,

I am happy, and encouraged, to see make you make it 'this far' to the 'Create VMs' setting screen.

I honestly believe, you do have compatible hardware. Screw the particular/correct networking hardware support 'for now.'

I'm hopeful that if you emailed your 'lspci -k' output from within a 'properly working, Net-connected' Linux OS terminal using the exact same hardware you are trying to install Qubes on to Joanna, and, of course, referencing this thread, she might be able to find: something, even if that requires a 'new' testing ISO.

Almost: there!

Thanks for sharing, 'progress,' vshakel.

[ideologysec]

Hrm. I am beginning to wonder if we do in fact have the same computer - vshakel, do you have the MacBook Pro with the dedicated AMD card, or just the Intel Iris Pro integrated one?

One thing that should not be different is our wireless cards, though. I suspect you're disabling skge because it's an ethernet driver that was causing issues, but retina MacBook Pros don't have ethernet hardware. Instead, I think that it's hanging up at the same place for the same reason that the Dell I was using was - the very latest Broadcom 3x3 ac wireless chip. Try blacklisting brcmfmac or brcmsmac, and then seeing if you can get past the setup & netvm creation screen.

Edit: confirmed, it looks like brcmfmac, at least from a vanilla Fedora installation. The Fedora 23 workstation install I have, fully updated, is using a 4.2.3-300 kernel. The 4.1.13 in Qubes may not support or have the latest driver, or it could be an issue with the PCI device not supporting the proper interrupts. (https://www.qubes-os.org/doc/assigning-devices/#tocAnchor-1-1-3)

Are there any plans for a newer 4-series kernel in Qubes anytime soon?

For me, it appears that without some EFI hacking or chainloading, that the intel integrated card will never be seen. I'll keep hacking on it, but I'm hung up on the fact that Qubes can't find the /dev/mapper/core3 root device, even if I do a straight ext4 or LVM no encryption install.

https://github.com/0xbb/gpu-switch https://github.com/0xbb/apple_set_os.efi https://lists.gnu.org/archive/html/grub-devel/2013-12/msg00442.html

[vshakel]

Aktariel: Seems like we have exactly the same hardware. My Mac is mid 2015 Pro 15" retina i7/16/512 with a broadcom chip. Please try to edit your xen.cfg like this: (maybe you will have to edit root=/dev/dm-2 to some /dev/dm-X that you can discover in your dracut shell, also maybe updating kernel might help)

[global] default=4.1.13-8.pvops.qubes.x86_64

[3.19.8-100.fc20.x86_64] options=loglvl=all kernel=vmlinuz-3.19.8-100.fc20.x86_64 root=/dev/dm-2 vconsole.font=latarcyrheb-sun16 rd.lvm.lv=qubes_dom0/root rd.blacklist.drivers=brcmfmac rd.blacklist.drivers=radeon radeon.modeset=0 ramdisk=initramfs-3.19.8-100.fc20.x86_64.img

[4.1.13-8.pvops.qubes.x86_64] options=loglvl=all kernel=vmlinuz-4.1.13-8.pvops.qubes.x86_64 root=/dev/dm-2 vconsole.font=latarcyrheb-sun16 rd.lvm.lv=qubes_dom0/root rd.blacklist.drivers=brcmfmac rd.blacklist.drivers=radeon radeon.modeset=0 ramdisk=initramfs-4.1.13-8.pvops.qubes.x86_64.img

HardenedArray: I believe we do have compatible hardware but we all deserve a straightforward installation.

I've tried to launch with rd.blacklist.drivers=brcmfmac but it frozen again on setting up networking. If anyone knows how to bypass Qubes network configuration please let me know. For now I had to choose Do-not-configure-anything option so the installer finished the installation. Victory, guys!

[ideologysec]

I also have it working; also had to choose to set up nothing on the final screen for it to proceed. I wonder if simply unchecking the "create system qubes" option would be enough.

Blacklisting the brcmfmac driver does not appear to be enough to prevent Qubes from trying to load it, which is likely what is causing the freezing. Had the same issue on the Dell; had to manually disable the wireless card in UEFI setup on that one, which is

Booting qubes with "spoof_osx_version=10.10" set in rEFInd causes Qubes to not boot at all; it hangs after loading USB devices. Methinks it does not like the dual graphics cards and/or the Crystalwell graphics. Shame; this'd be a great candidate for using the Intel chip for Qubes and passing the Radeon card over to a Windows HVM.

Next steps: newer kernel (I have no idea how to create a pvops-qubes kernel from the 4.3 branch though)? create sys-net vm and see if we can assign the wireless card to it, and if that will work properly?

@marmarek or @rootkovska or someone else with more hardware knowledge, what are next steps we can/should be taking?

[vshakel]

We need to setup now Qubes OS environment almost in full and make the networking work. lspci reports Broadcom Corporation BCM43602 802.11ac Wireless LAN SoC (rev 01) Also I can see there my Radeon HD 8870M / R9 M370x (rev 83) and some proper drivers would be right on point as brightness control is not reacting though the keyboard lighting surprisingly working.

It is promising that I've mounted a USB flash drive no problem so uploading new files is sorted. Right now I managed to have internet inside my Qubes using USB tethering I'd like to have some guidance from Qubes OS experts here... I need to enable repos, install firewall etc.

[ideologysec]

@vshakel I started a GoogleGroups thread since I was worried this project of ours is hijacking the original EFI boot issue. I managed to get sys-net created by physically removing the wireless card, going through installation and setup, and then reinserting it. it still causes Qubes to freeze and not boot when assigned to sys-net after the fact though.

https://groups.google.com/forum/#!topic/qubes-users/RCOsHt1rMUo

@marmarek it'd be really nice if Qubes switched to rEFInd for the bootloader, since it seems to work better on Macs out of the box, but I understand I have a niche case and that Grub2 might work better for the majority of folks. (it'd also be sweet to get a newer kernel shipping by default, but I know that most of the effort for 3.1 is focused on closing issues at this point.)

[marmarek]

I'm currently working on some newer image with rEFInd and yes - considering using rEFInd in the final 3.1 image.

[vshakel]

Thanks. Moving to GoogleGroups as we're getting out of context here.

[marmarek]

New image, with rEFInd (and most recent versions of all the things): http://ftp.qubes-os.org/~marmarek/Qubes-20160207-x86_64-refind.iso http://ftp.qubes-os.org/~marmarek/Qubes-20160207-x86_64-refind.iso.asc (still uploading, ETA 1h)

[ideologysec]

Downloaded, tested - still all the same issues, documented in the google groups thread (empty xen.cfg, rEFInd seemingly not installed, wireless still freezing the system).

But hey, sys-whonix is black and the Qubes installer has pretty boot pictures and starts up without an external rEFInd stick!

[wardwouts]

On a Lenova Yoga 2 pro I'm getting four penguins, like @ManoftheSea wrote on Nov 25, 2015. Fedora boots fine in UEFI mode.