self-contained Qubes templates including meta scripts (salt) / improve Qubes-Whonix installation usability

[adrelanos]

Usability Issue

The more lengthy the instructions, the more commands users needs to manually type into a terminal emulator, the more can go wrong usability wise.

For example in Qubes-Whonix installation instructions, users need to be told:

Upgrade Qubes dom0. sudo qubes-dom0-update

This is required to make sure

• version file /srv/formulas/base/virtual-machines-formula/qvm/whonix.jinja contains the current version number of is up to date,
• a recent version of Qubes repository definition files,
• Qubes salt,
• qubes-core-admin-addon-whonix,
• as well as qubes-mgmt-salt-dom0-virtual-machines are installed and up to date.

Many users skipped that step. That might be because Qubes dom0 updates have a chance, in past broke connectivity or Qubes dom0 boot.

Then documentation was updated to say:

Upgrade Qubes dom0. This step is mandatory.

That helped but still some users skipping that step. Therefore it was required to make sure the user really did update Qubes dom0. More documentation was added:

Verify whonix_version is 16.

If the previous sudo qubes-dom0-update was completed, it should not be necessary to verify the version number. However, this is mentioned because many users fail to update dom0 packages beforehand.

In dom0. View contents of file /srv/formulas/base/virtual-machines-formula/qvm/whonix.jinja.

sudo cat /srv/formulas/base/virtual-machines-formula/qvm/whonix.jinja

Example output:

16

If it shows something else, then Qubes dom0 is outdated. In that case, it is not possible to continue.

At this point, usability of these instructions is already really bad.

Next step in documentation is telling users:

Download both Whonix-Gateway ™ and Whonix-Workstation ™ Templates.

In dom0, run.

sudo qubes-dom0-update --enablerepo=qubes-templates-community qubes-template-whonix-gw-16 qubes-template-whonix-ws-16

That is a lengthy command to type. Users by Qubes default cannot copy from a VM to dom0 for security reasons. But then typing such a command for users who are not using a (kinda) 10 finger typing system is a challenge. One typo, and it won't work.

That command could be replaced by a shorter command sudo qubesctl state.sls qvm.anon-whonix but then that can take ages (especially when downloaded over Tor) so users reported this to be confusing as it's not known by the user if the download is still taking a long time or the entire process hanging forever. #4215

Also the parameter --enablerepo=qubes-templates-community was reported to be needed quite some times in Qubes-Whonix and Qubes forums.

After that users are told:

Configure sys-whonix and anon-whonix safely.

In dom0, run.

sudo qubesctl state.sls qvm.anon-whonix

Yet another command. Rationale for that is explained in #3447.

On top of that: #6904

It's a lot rough edges that really add up to bad usability.

Technical Issue

The template upgrade is currently scattered into many different pieces. At least:

• Qubes-Whonix template,
• qubes-core-admin-addon-whonix,
• qubes-mgmt-salt-dom0-virtual-machines

I.e. the template + meta scripts.

Suggested Solution

A Qubes template should be self-contained with everything (template + meta scripts).

For example, Whonix for VirtualBox doesn't have this issue because VirtualBox .ova files are self-contained. These contain both, the virtual machine image file as well as the metadata (virtual machine configuration).

If we had that + #4215 + #6904 then perhaps command line installation instructions could be really shrunk to sudo qubesctl state.sls qvm.anon-whonix.

[marmarek]

I'm not sure if I follow your reasoning. I see it this way: the proper instruction is:

1. Update dom0, using a single sudo qubes-dom0-update command (or GUI equivalent).
2. Install Whonix using a single sudo qubesctl state.sls qvm.anon-whonix (no GUI for this one yet, but will be at some point)

But then, for user who skipped any of those simple steps, you added way more complex steps (like installing templates via qubes-dom0-update). If they skip simple steps, why would adding complex ones could help? If anything, it makes things worse...

Now to the details:

Verify whonix_version is 16.

A simpler check would be rpm -q qubes-mgmt-salt-dom0-virtual-machine (although the package name is a bit long, to type manually). Maybe Whonix formula(s) should be in a separate package, that could have versions aligned with Whonix releases?

Also the parameter --enablerepo=qubes-templates-community

Yes, it is required because community templates repo is (intentionally) not enabled by default. This step is already covered with salt formula (as well as many other things).

A Qubes template should be self-contained with everything (template + meta scripts).

The template itself can (and do) have a bit of metadata about itself. Things like installed applications, OS inside, what services are available inside etc. But it (intentionally) cannot influence anything outside of that template, like creating new VMs, installing dom0 packages etc. Otherwise, templates would be much more sensitive, negating a lot of separation (for example, we'd need significantly higher bar for accepting community templates, which already is rather high). We have salt formula for all this, and it gets much more review etc.

[unman]

This shouldnt be difficult, but the potential of system breakage strikes me as enormous. Can you say exactly what you would want sudo qubesctl state.sls qvm.anon-whonix do? What other salt commands would you want to support?

[brendanhoar]

I'm not sure if I follow your reasoning. I see it this way: the proper instruction is:

1. Update dom0, using a single sudo qubes-dom0-update command (or GUI equivalent).

2. Install Whonix using a single sudo qubesctl state.sls qvm.anon-whonix (no GUI for this one yet, but will be at some point)

But then, for user who skipped any of those simple steps, you added way more complex steps (like installing templates via qubes-dom0-update). If they skip simple steps, why would adding complex ones could help? If anything, it makes things worse...

In addition to the dom0 updates/jinja gaps, I think there was an additional precipitating event: for some people, executing the salt formula led to a) the appearance of a system freeze (noted above) while waiting for a very long download or b) an actual system freeze/kernel panic (I ran into the latter but did not investigate, and I saw at least one other had a similar experience).

"a)" is covered in the ui/usability gap discussion above.

"b)" seemed to be remedied either by retrying later and/or manually downloading the templates first...leading to the writing more complex instructions, perhaps.

B

[adrelanos]

I'm not sure if I follow your reasoning. I see it this way: the proper instruction is:

1. Update dom0, using a single sudo qubes-dom0-update command (or GUI equivalent).
2. Install Whonix using a single sudo qubesctl state.sls qvm.anon-whonix (no GUI for this one yet, but will be at some point)

But then, for user who skipped any of those simple steps, you added way more complex steps (like installing templates via qubes-dom0-update). If they skip simple steps, why would adding complex ones could help?

By experience, this creates less questions. Well, that might also be because users give up at that point. Dunno.

A Qubes template should be self-contained with everything (template + meta scripts).

The template itself can (and do) have a bit of metadata about itself. Things like installed applications, OS inside, what services are available inside etc. But it (intentionally) cannot influence anything outside of that template, like creating new VMs, installing dom0 packages etc. Otherwise, templates would be much more sensitive, negating a lot of separation (for example, we'd need significantly higher bar for accepting community templates, which already is rather high). We have salt formula for all this, and it gets much more review etc.

I see.

Would it be sane if Whonix templates would have Qubes salt as a dependency? That way salt upgrade would be guaranteed and no way an omitted "sudo qubes-dom0-update" causing any issues?

This shouldnt be difficult, but the potential of system breakage strikes me as enormous. Can you say exactly what you would want sudo qubesctl state.sls qvm.anon-whonix do? What other salt commands would you want to support?

It's good as is - if dom0 is up to date - which is often not. "Ideally" it would run "sudo qubes-dom0-update" but that is too much too ask. It would solve part of this usability issue (assuming having salt progress indicator) but would also create bigger mess (unwanted, unexpected general Qubes dom0 update).

I'm not sure if I follow your reasoning. I see it this way: the proper instruction is:

1. Update dom0, using a single sudo qubes-dom0-update command (or GUI equivalent).
2. Install Whonix using a single sudo qubesctl state.sls qvm.anon-whonix (no GUI for this one yet, but will be at some point)

But then, for user who skipped any of those simple steps, you added way more complex steps (like installing templates via qubes-dom0-update). If they skip simple steps, why would adding complex ones could help? If anything, it makes things worse...

In addition to the dom0 updates/ninja gaps, I think there was an additional precipitating event: for some people, executing the salt formula led to a) the appearance of a system freeze (noted above) while waiting for a very long download or b) an actual system freeze/kernel panic (I ran into the latter but did not investigate, and I saw at least one other had a similar experience).

"a)" is covered in the ui/usability gap discussion above.

"b)" seemed to be remedied either by retrying later and/or manually downloading the templates first...leading to the writing more complex instructions, perhaps.

B

Right.