search

QubesOS / qubes-issues

The Qubes OS Project issue tracker
https://www.qubes-os.org/doc/issue-tracking/
541 stars 48 forks source link

dom0 UX visibility issues - system state, free space, per-vm used storage, CPU, RAM, attachments, virt_type, pool & tags #3241

Closed na-- closed 6 years ago

na-- commented 7 years ago

Qubes OS version:

R4.0 RC2

Affected TemplateVMs:

none (dom0 issue)

Expected behavior:

Have an easy way to see the accurate current system state:

Ideally, the above should be easily achievable both via the GUI (for normal users) and via the CLI (for scripting and CLI/advanced users).

Actual behavior:

Most of the things above are far more difficult than they should be. I wrote about my issues with determining the free space when using LVM thin provisioning in another issue. Other UX issues (at least for me) will be discussed below.

General notes:

I'm very slowly adjusting to the lack of qubes manager in 4.0, but it has been a huge usability drop for me. It was a single place that showed the current accurate system state and allowed me to modify it and perform all sorts of actions very easily. Right now there is simply no alternative.

qvm-ls is a very poor replacement of the old qubes manager: it's a CLI tool (unsuitable for linux newbies), it does not show current CPU and RAM usage and does not allow sorting. It also lacks some information like virt_type, kernel, storage pool and tags. RAM usage can be found via xl list, and information for disk and usb attachments is in other CLI tools (qvm-block, qvm-usb), as it should be, but that makes system information gathering cumbersome. qvm-prefs and qvm-tags show some things that are not present in qvm-ls, but lack others (storage pool for VM).

The new system tray widgets for VMs and attachments show only a fraction of the needed information, are somewhat broken (1, 2) and have somewhat poor UX, at least in my view (more on this below). Right now gathering the current system information from one source is impossible, especially via the GUI.

Current GUI shortcomings and missing features that I see:

Sorry if this looks a bit like a rant or criticism, it's not. I'm a long time Qubes user and have no intention of using something else any time soon, or even reverting to 3.2 if I can help it :) If there's some consensus, I would love to help with improvements to the GUI, even though my python skills are more than a bit rusty. I realize that the old qubes manager had to be scrapped or rewritten because of the changes in the internals, but is there a chance something like it could be resurrected or the current widgets made more informative, functional and configurable?

Related issues:

2132 seems to be the issue in which the removal of the qubes manager was decided and some of the new GUI elements were prototyped

3240 where I mentioned my storage issues in more detail

qubesuser commented 7 years ago

I agree.

My suggestion would be to write a bunch of applications with a similar UI, one for each concern.

This separation would essentially mimic the way traditional operating systems manage resources, but one could also have a single application with multiple tabs, and perhaps a way to look at a given VM in other apps/tabs (e.g. in Windows you use Explorer/WinDirStat to look at storage, Task Manager for CPU/memory/I/O usage, Network control panel for network, Device Manager for devices).

Tree views would become list views when sorted.

Running VMs Add a new "Qubes Task Manager" that would show, in a list view, running VMs, their CPU, memory, network and disk I/O usage, as percentages and graphs (only update the graphs when the window is visible!), and allows to easily launch System Monitor and Terminal in the VMs

Change to the Qubes domains tray icon to open it on double click or when clicking a new menu entry.

Storage Add a new "Qubes Storage Manager" that would show, in a tree view with pools as parent and contained VMs as children, all VMs, their disk space used, and that allows to easily launch Baobab, Nautilus and Terminal in the VMs.

Provide an XFCE panel widget that shows the current free space in the default pool and launches the "Qubes File Manager"

Devices Add a new "Qubes Device Manager" that would show, in a tree view, all devices and the VMs/USB hubs/etc. they are physically attached to in a tree, as well the name of the VM they are logically assigned to, and allows to launch some sort of Device Manager app and Terminal in the VMs.

Change the devices tray icon to open it on double click or when clicking a new menu entry.

Also fix the devices tray icon to use a different icon for attached and unattached devices.

Network Add a new "Qubes Network Manager" that would show, in a tree view, all VMs with VMs that have them as the netvm as children, as well as other networking details like IP addresses and would allow to configure the firewall, and allows to launch some sort of graphical version of "netstat" in the VM or the Terminal. This would also contain transient data like network recv/send bytes used by each VM.

Provide an XFCE panel widget that shows the current total network usage, that opens it on click.

Configuration Add a new "Qubes Configuration Manager" that would show, in a tree view with label colors as parents and VMs as children, all VMs and all their preferences and open up the preference dialog on right click, and allow to launch Settings apps and Terminal in VMs, and also allow to set Qubes preferences and edit Qubes RPC policies.

This would be launched from the system menu, replacing Qubes preferences.

marmarek commented 7 years ago

@rootkovska what do you think about all this?

SimonSelg commented 7 years ago

@na-- : You can view the CPU/Memory usage per vm using xentop, which is installed in dom0 (sudo xentop -f).

@qubesuser: I really like your idea of having xfce4 panels widgets for that purpose. At least one that shows the current system load (cpu, memory, disk i/o), having to go through xentop is somewhat annoying.

Anyways, I'd like to contribute here - my python skills aren't that bad, and I have some free time on my hand.

na-- commented 7 years ago

@SimonSelg: thanks! I feel like I should have known about xentop after several years of using Qubes... I guess the old qubes manager was so good that I never needed it, heh :)

@qubesuser: while I would personally prefer a more centralized solution that allows for an easier bird's eye view of the whole system state, what you describe would probably be an improvement over the current state. I have a few concerns with such a strategy:

The old qubes manager basically did 3 different jobs at once:

I understand how the above could be somewhat confusing, especially for new users. But it was just so darn useful and convenient :) I don't see anything there that can't be fixed or improved. At the same time I think that trying to decompose and replace all of the different functions that the qubes manager did only makes everything more confusing and difficult to use, both for beginners but especially for advanced users.

andrewdavidwong commented 7 years ago

There's also a lot of good discussion on this issue in this qubes-users thread: https://groups.google.com/d/topic/qubes-users/e5rGdU73QcU/discussion

rootkovska commented 7 years ago

We definitely don't want to create, maintain and force the user to use just The One Centralized Qubes Apps. This is simply not how modern OSes work. As @qubesuser mentioned above, we need specialized apps for different purposes. The decomposition of the Qubes Manager we did for 4.0 has been the first step into this direction. Soon we will write an app for volume/template/updates/backups management. This will be a separate app from the Qubes (Task) Manager applet. Just like on other systems there are different apps/applets/widgets for different things.

marmarek commented 7 years ago

Just like on other systems there are different apps/applets/widgets for different things.

Well, if you look into gnome, they actually integrated multiple widgets (network manager, volume control, logout, brightness, power manager) into one widget. So this is exactly the opposite direction...

3hhh commented 7 years ago

Just like on other systems there are different apps/applets/widgets for different things.

Well, if you look into gnome, they actually integrated multiple widgets (network manager, volume control, logout, brightness, power manager) into one widget. So this is exactly the opposite direction...

Decomposition of code is good, decomposition of the main configuration interface not so as users want to have one central place they need to look at to configure their OS. So a plugin or app-like model for a central management interface seems ideal. Plugins might give a more integrated look & feel (centrally managed), apps might be more easy to customize. Also backend devs don't really want to care about the GUI, so some interface to advertise options to the frontend should be helpful.

Examples that I'm aware of:

andrewdavidwong commented 7 years ago

Further discussion and feedback: https://www.reddit.com/r/Qubes/comments/790jr4/qubes_40_rc1rc2_updates/

section0x8 commented 7 years ago

This is simply not how modern OSes work.

I think all OSes have a control panel of some sort. I'm not sure why Qubes should be any different.

errin07 commented 7 years ago

For me it is not about a control panel. The vm-settings option is in 2 places, easy to find. The qubes-manager for me was in the 1st place a system-monitor. You had to add a popup dialogue "VM is starting" as a "minute fix" because there was no user feedback at all. The app-menu can't do that, in fact for now it gives out wrong information for me what is template and what is appvm and which colors they have. I needed qubes-manager for: 1) the quick overview+feedback of the whole system I had (every VM, state, templates used, color, net-vm used. mem, updateable, had errors etc.) where I could even order the VMs, the qubes-manager then had the 2) practical benefit to perform easy actions like starting/stop/pause (clone/backup/restore) or attach block devices without going to cli.

So it would be nice to address at least point 1) in a qubes specific way while point 2) gets split into different widgets for differents tasks.

qubesuser commented 7 years ago

It might make sense to just make the old Qubes Manager code work with the new admin API and then reinvent the UI as a successive step, so that Qubes 4.0 can be released in a timely manner and be at least somewhat usable by non/less-technical users, which is currently impossible due to the lack of an UI to do many things.

marmarek commented 7 years ago

Well, one of the reasons why we're ditched old Qubes Manager is the architecture of its code, especially the main window. In many cases it does things directly with underlying lowlevel API, subverting qubes-core. Having such things in Qubes 4.0 is no-go, everything should go through official (Admin) API. Rewriting main window handling code to proper API will be a lot of work, so not something we want to do if we're going to replace it anyway. But, if someone could contribute that migration, I think we may include it as an alternative VM manager.

tasket commented 7 years ago

An assessment of 'modern desktop OSes' would mean paying heed to how Windows does it and regarding the Linux options as "so bad you cannot even give them away for free". Too many devs ignore defacto standards when it comes to personal computing, and also not realizing that much of those standards can only be understood by observation bc Microsoft is not going to write the book for us.

Chapter 1, if such a book existed, would impress on feature-stability and UI stability (tellingly, these are not concepts discussed around FOSS desktops). People walk away from shaky platforms, no matter how positive they felt initially or how well-abstracted the layers underneath.

The trade-off we seem to be making at the moment is to dump critical UI so that eventual replacement(s) will readily port to whatever hypervisor+hardware in the future. That could chain ITL and contributors to R3.2 support for extended periods and also put pressure on Qubes' R4 API to rapidly expand in complexity. I would opt for a different tradeoff: Keep Qubes Manager and slowly move some functions (starting with backup/restore) to different apps.

qubesuser commented 7 years ago

@marmarek any UI design probably needs a graphical xentop / "task manager" tool that includes a sortable list view with VMs as rows and CPU, memory, disk, network usage as columns, and the difference between that and Qubes Manager is essentially a bunch of additional columns for persistent state and the right click menu/toolbar actions, so most work done on a 4.0 Qubes Manager should at least be reusable for the "task manager" tool.

lunarthegrey commented 7 years ago

Putting my 2 cents in. I think it was silly to remove the Qubes Manager in the first place, it's one thing I really loved using and somewhere I could go to get a total overview of everything. Qubes shouldn't follow the development of "modern OSes" just because "modern OSes".

n13 commented 6 years ago

Data point: As somebody who just installed qubes, and found it surprisingly usable, the absence of qubes manager is baffling. For example, I have no clue how much total disk space is available. I googled it, and found multiple references to the unicorn, qubes manager or qubes vm manager.

As Einstein said, make it as simple as possible, but not simpler.

I read all the discussion on the problems with Qubes Manager, and I can see the issues. But if it does some not so good things under the hood, surely it could be shipped and bit by bit improved to instead use the APIs that the command line tools use to do its work.

From a usability perspective it would be good to have a VM manager, to get a quick overview of VMs and a disk manager to have a quick overview of disk usage. Like a disk utility.

On a practical level, 95% of what I wanted to do in the last 2 hours was in qubes manager. I want to do X, I google for it, instructions read "Open qubes manager...." - so the removal of this tool made googling documentation pretty much useless. And so productivity or even getting familiar with the system has become super hard.

Adding this as a newbie comment.

tasket commented 6 years ago

I might as well link to the discussion in qubes-users:

https://groups.google.com/d/msgid/qubes-users/eddf5625-11ca-40d9-be53-190c60f366fe%40googlegroups.com

On a practical level, 95% of what I wanted to do in the last 2 hours was in qubes manager.

If articles and docs keep referring to a component as a starting point, that's usually a sign that its functional because people can understand the system through it. This also suggests the component really isn't "optional".

People come to Qubes because they want to focus on security domains. Qubes Manager did that admirably. In an age when 'dashboards' that focus the user on pertinent objects/info are all the rage, R4.0 seems to be going in the wrong direction. I'd support the return of QM and even adding to its functions and reach.

mannp commented 6 years ago

I guess there needs to be at least a warning to the user that they have ran out of space and the system is about to corrupt itself.

I am trying to recover from such a state and however much I love using qubes, its truly tedious.

marmarek commented 6 years ago

configuration (persistent) information: what's the VM type, label, template, netvm; some configuration information was not shown in the table by default (like kernel version, allowed storage), but it could have been something in-between: how much space is currently used by the VM, date of last backup (don't remember if that could actually be shown the table)

We've talked internally about possible solutions for this problem and concluded that, given the time constraints, we won't have new application for this before final 4.0. So, we're going to resurrect this part of Qubes Manager. The static part of the main table, without any automatic updating logic (there will be "refresh" button) or other things already covered by new widgets (like device attaching).

najamelan commented 6 years ago

I would like to suggest also thinking about debugging. Centralized place to see logs and failed systemd services in all VM's, especially qubes services.

Personally I'm in favor of having a centralized management, in a separate VM, but have one well designed gui application (not running in dom0) from where one can manage everything that is qubes specific (et. what makes qubes different from OS users are used to). That would be a great help to a new user like me. Current situation makes me blind as to the state of the system until I figure out where I have to look, which command line command, which log file and in cubes these are spread out over different domains to make things worse.

That's bad for usability, debugging and security.

Modern OS does work with integration. Systemd is an attempt at this (with poor front end for the moment), configuration tools like gnome-settings is another example. For servers you will get stuff like CPanel, ...

Overview is important!

Shane-Optima commented 6 years ago

This is simply not how modern OSes work. As @qubesuser mentioned above, we need specialized apps for different purposes.

I'm not certain how to say this without sounding unbearably snarky, but... really? I mean, has Windows Explorer been split into three apps, one to rename files, one to copy them, one to delete them? Have "start menu" type buttons been decomposed into thirty different widgets covering the screen, each one launching a different type of application? I thought that end user applications had fairly broad but intuitive goals like "launching applications", "viewing and manipulating files".... or "viewing and interacting with VMs." Behind the scenes there could be two dozen totally modular executables, but the user is faced with one application for task.

...I thought? I mean I've not used anything after Windows 7, so it's possible I've missed some particularly horrific UI "revolution"

I won't spam points I've already said elsewhere, but I want to repeat that as a power user (not to be confused with expert), the best UI development of the 21st century has far and away been the little text box in the start menu, and I know many power users who would readily agree with me. That's because that little box does EVERYTHING... lets me change any setting, launch any program, open any file. It doesn't insist that I memorize a new UI or new incantations to recite. Not everything can be simplified to that extent, but I think it is an important example of the role and limitations of modularity.

I think everyone is totally on board with the idea of the Unix philosophy on the back end, but we're less convinced that it entails the complete abandonment of elegant front-ends that are capable of launching and unifying these "specialized apps" into a compact manner that makes perfect intuitive sense to people who do understand that Qubes is made up of many different VMs and wish to interact with this one or that one. Which is not to say your criticisms of the Qubes Manager do not make sense--they do, but an instance on pure and absolute modularity and orthogonality and separation in the final user-facing interface is... well, it doesn't make sense to a lot of us.

I'll leave it at that. Thank you again for everything you've done. Though I may be only a semi-technical user, I have usually been very impressed with and have remained in total agreement with your insistence on correctness.

zander commented 6 years ago

@Shane-Optima the only way I could make Qubes 4 bearable was by running KDE on top of it with the "application dashboard" startmenu. Which is actually a full-screen thing and does search etc.

The good thing is that when the admin-api (and qrexec) become mature then people can write their own tools and have no need to use the ones that the core team creates.

marmarek commented 6 years ago

For Xfce there is alternative menu, which also contains search: whisker menu. It's installed by default, you just need to add it to the panel. Also, launcher accessible with alt-f2 has menu search option when you press down arrow.

Polygonbugs commented 6 years ago

I really like Qubes OS 4.0 (Official release) than the previous one 3.2 almost everything except UX issues. I'm sorry for developers who made this incredible OS very hard but I think UX is still incomplete and almost unusable. I would understand if users stay 3.2, that would be lack, laggy, and buggy qubes GUI management tools. You should have dealt with more carefully when implementing new UI for managing OS. You already know that visibility is also one of the most important aspects of security.

I'm not sure what is your current goal for UI but it definitely need to be changed. In my opinion, I think you have three options. First, rebuild new Qubes-Manager. This one is something you don't like to do because of increased complexity by implementing admin API from 4.0. Also against policy of qubes decomposition which implemented due to excessive bug reports of old Qubes-Manager and possible future security issues (other reasons don't make sense). Though Qubes-Manager revived on 4.0 because of users' claim, I'm evident that you formally made it. It is super laggy, has more errors (I think the number of 4.0 Qubes-Manager level errors are almost as same as total 3.2 OS level errors), has no auto refresh. Second, HTML Qubes Management Tool. It is something like router managing pages on browser via remote protocols. Of course, it could be bad for security since it has to use internet browsers which have huge amount of vulnerabilities. Also updating browser would be difficult since dom0 is outdated fedora version so users have to manually upgrade. But if you could make this, there is some chance to make the UI more elegant and more powerful. Last one is focusing on widgets, which I think is your current goal. It has clear pros and cons. The advantage of using widget is good for someone using desktop machine with a lot of virtual desktops or laptop. Yeah, someone could argue that why don't we add extra monitors to a machine. But in case of laptop it would be very difficult and inefficient. Disadvantage is general inconvenience already explained by many users. However, this could be improved. I can give you concrete advice with this. Below is everything about it.

Adding New Widgets

Add features to current Widgets and Fix bugs

marmarek commented 6 years ago

Going through initial list:

* how much free space there is in the used system drives(s)

This is now handled by disk space widget. Including a warning when you're running out of space.

* how much space each VM and template uses, preferably with the ability to sort them

Revived Qube Manager do have this.

* currently used CPU and memory for active VMs, also sortable

Memory usage is covered by domains widget, but CPU usage is not.

* other properties like virt_type, kernel, used storage pool, tags, etc. for every VM without having to muck about individually with `qvm-prefs` or `qubes.xml`

You can do that with qvm-ls --fields name,virt_mode,.... Adding it to Qube manager could be useful. 

* one-second answers for questions like "is my mic/webcam/external drive attached to a vm?"

Attached devices are now in bold and with VM name straight in devices list.

  • easy answers to questions like "What are the VMs that use the debian/archlinux/etc. template?", "Which VMs are connected to the sys-whonix gateway?", "Which VMs use kernel 4.13?", etc. (i.e. filtering/sorting/querying by certain VM properties)

From GUI: sort in Qube Manager. From cmdline - use qvm-ls | grep ... (possibly with --fields).

marmarta commented 6 years ago

Alright, I went through this discussion, created separate tickets for issues arising that are not yet fixed and sorted them into what is realistically doable soon-ish (and assigned those to myself) and far-in-the-future ones. This discussion, while very interesting, is not really easy to parse for someone who wants to fix something, so I'll close this one and hopefully go and fix the newly made templates. Soon.