Closed emkll closed 3 years ago
I'm suddenly having issues updating fedora-32-minimal both using Fedora-32 as the Template for default-mgmt-dvm AND using Debian-10 as the Template as well. Here's the error message using Debian 10 as default-mgmt-dvm:
Updating fedora-32-minimal
Error on updating fedora-32-minimal: Command '['sudo', 'qubesctl', '--skip-dom0', '--targets=fedora-32-minimal', '--show-output', 'state.sls', 'update.qubes-vm']' returned non-zero exit status 20
fedora-32-minimal:
----------
_error:
Failed to return clean data
retcode:
1
stderr:
Traceback (most recent call last):
File "/var/tmp/.root_dd8a91_salt/salt-call", line 15, in <module>
salt_call()
File "/var/tmp/.root_dd8a91_salt/py3/salt/scripts.py", line 405, in salt_call
import salt.cli.call
File "/var/tmp/.root_dd8a91_salt/py3/salt/cli/call.py", line 5, in <module>
import salt.utils.parsers
File "/var/tmp/.root_dd8a91_salt/py3/salt/utils/parsers.py", line 27, in <module>
import salt.config as config
File "/var/tmp/.root_dd8a91_salt/py3/salt/config/__init__.py", line 101, in <module>
_DFLT_IPC_WBUFFER = _gather_buffer_space() * .5
File "/var/tmp/.root_dd8a91_salt/py3/salt/config/__init__.py", line 90, in _gather_buffer_space
import salt.grains.core
File "/var/tmp/.root_dd8a91_salt/py3/salt/grains/core.py", line 32, in <module>
from platform import _supported_dists
ImportError: cannot import name '_supported_dists' from 'platform' (/usr/lib64/python3.8/platform.py)
stdout:
This did allow for fedora-32-testing to upgrade. Using fedora-32 as the Template for default-mgmt-dvm:
Updating fedora-32-minimal
Error on updating fedora-32-minimal: Command '['sudo', 'qubesctl', '--skip-dom0', '--targets=fedora-32-minimal', '--show-output', 'state.sls', 'update.qubes-vm']' returned non-zero exit status 20
fedora-32-minimal:
----------
_error:
Failed to return clean data
retcode:
1
stderr:
Traceback (most recent call last):
File "/usr/lib/qubes-vm-connector/ssh-wrapper/ssh", line 101, in <module>
sys.exit(main())
File "/usr/lib/qubes-vm-connector/ssh-wrapper/ssh", line 94, in main
return ssh(args)
File "/usr/lib/qubes-vm-connector/ssh-wrapper/ssh", line 29, in ssh
assert args[1] == '/bin/sh'
AssertionError
stdout:
Using fedora-32 as the Template for default-mgmt-dvm did allow me to update the fedora-32 templateVM yesterday.
So basically, all the Debian templateVMs updated fine, fedora-32 and fedora-32-testing updated fine, but the fedora-32-minimal templates I have did not. Let me know if you require my /var/log/xen/console/guest-disp-mgmt-fedora log as well and I'll post up later.
I ran into this problem myself, and have a patch. I will submit a PR shortly. @andrewdavidwong can you assign me to this issue?
Salt used to pass the command to run as the standard input of SSH, but has since changed to pass it in the arguments to SSH. This caused an assertion in ssh-wrapper.
Assigned. Thank you, @DemiMarie!
Wonderful, thanks for tackling it! @DemiMarie don't hesitate to ping here for testing once you have a PR up.
Updates in SaltStack 3001 on November 3 were tied to three CVEs, this one probably explains the changes to what we see:
CVE-2020-16846: This CVE affects any users running the Salt API. An unauthenticated user with network access to the Salt API can use shell injections to run code on the Salt-API using the SSH client.
Advisory: https://www.saltstack.com/blog/on-november-3-2020-saltstack-publicly-disclosed-three-new-cves/
I performed a quick test of @DemiMarie's PR by building for mgmt-salt-vm for fc32 and installing the qubes-mgmt-salt-vm-connector onto my fedora 32 template; it does resolve the assert args[1] == '/bin/sh'
error when performing a qubesctl
update and completes successfully.
Automated announcement from builder-github
The package You
has been pushed to the r4.1
testing repository for the Debian template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list
by uncommenting the line containing stretch-testing
(or appropriate equivalent for your template version), then use the standard update command:
sudo apt-get update && sudo apt-get dist-upgrade
Automated announcement from builder-github
The package mgmt-salt
has been pushed to the r4.0
testing repository for the CentOS centos7
template.
To test this update, please install it with the following command:
sudo yum update --enablerepo=qubes-vm-r4.0-current-testing
Automated announcement from builder-github
The package qubes-mgmt-salt-4.1.8-1.fc32
has been pushed to the r4.1
testing repository for dom0.
To test this update, please install it with the following command:
sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing
Automated announcement from builder-github
The package qubes-mgmt-salt-4.0.24-1.fc25
has been pushed to the r4.0
testing repository for dom0.
To test this update, please install it with the following command:
sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing
Automated announcement from builder-github
The package You
has been pushed to the r4.0
testing repository for the Debian template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list
by uncommenting the line containing stretch-testing
(or appropriate equivalent for your template version), then use the standard update command:
sudo apt-get update && sudo apt-get dist-upgrade
In case you have already salt-3001.3 installed in template for default-mgmt-dvm, the fix needs to be installed by manually calling dnf update --refresh
(with --enablerepo=qubes-vm-*-current-testing
option, until the update migrates to the current
repository in about a week) in the template - Qubes updater won't work for this.
In case you have already salt-3001.3 installed in template for default-mgmt-dvm, the fix needs to be installed by manually calling
dnf update --refresh
(with--enablerepo=qubes-vm-*-current-testing
option, until the update migrates to thecurrent
repository in about a week) in the template - Qubes updater won't work for this.
Slightly confused:
There's no builder-github announcement for Fedora templates (only dom0, CentOS, and Debian).
Not sure which exact package from current-testing needs to be updated in the template used for default-mgmt-dvm. Would like to avoid updating all available in current-testing (e.g., img converter, passwordless root). Only need what's necessary for this particular fix. Guessing it's just qubes-mgmt-salt-vm-connector-4.0.24-1.fc32
for Fedora 32 on R4.0?
Something odd is happening with the builder-github notifications for Debian templates. It says "The package You
has been pushed...."
* Guessing it's just `qubes-mgmt-salt-vm-connector-4.0.24-1.fc32` for Fedora 32 on R4.0?
Yes, this one.
As for the other two, I've just fixed it. The issue was just with comments, labels were properly added (including *-fc*-cur-test
).
My qubes-u2f stopped working around the same time this updater issue appeared. Could that be linked?
My qubes-u2f stopped working around the same time this updater issue appeared. Could that be linked?
@Coeos3 I'd recommend opening a new issue. python-u2flib-host was updated around same time, but that's still in current-testing.
So oneliner:
1- Open up templatevm fedora-31 or fedora-32
2- run sudo dnf update -y --refresh --enablerepo=qubes-vm-*-current-testing qubes-mgmt-salt-vm-connector
3- shutdown templatevm
4- relaunch QubesOS updater
Enjoy.
Automated announcement from builder-github
The package qubes-mgmt-salt_4.0.24-1+deb10u1
has been pushed to the r4.0
stable repository for the Debian template.
To install this update, please use the standard update command:
sudo apt-get update && sudo apt-get dist-upgrade
Automated announcement from builder-github
The package qubes-mgmt-salt-4.0.24-1.fc25
has been pushed to the r4.0
stable repository for dom0.
To install this update, please use the standard update command:
sudo qubes-dom0-update
Or update dom0 via Qubes Manager.
Automated announcement from builder-github
The component mgmt-salt
(including package qubes-mgmt-salt-4.0.24-1.fc32
) has been pushed to the r4.0
stable repository for the Fedora template.
To install this update, please use the standard update command:
sudo yum update
Automated announcement from builder-github
The package mgmt-salt
has been pushed to the r4.0
stable repository for the CentOS centos8
template.
To install this update, please use the standard update command:
sudo yum update
Automated announcement from builder-github
The package mgmt-salt
has been pushed to the r4.1
stable repository for the CentOS centos8
template.
To install this update, please use the standard update command:
sudo yum update
Automated announcement from builder-github
The package qubes-mgmt-salt_4.1.9-1+deb10u1
has been pushed to the r4.1
stable repository for the Debian template.
To install this update, please use the standard update command:
sudo apt-get update && sudo apt-get dist-upgrade
Automated announcement from builder-github
The component mgmt-salt
(including package qubes-mgmt-salt-4.1.9-1.fc32
) has been pushed to the r4.1
stable repository for the Fedora template.
To install this update, please use the standard update command:
sudo yum update
Automated announcement from builder-github
The package qubes-mgmt-salt-4.1.9-1.fc32
has been pushed to the r4.1
stable repository for dom0.
To install this update, please use the standard update command:
sudo qubes-dom0-update
Or update dom0 via Qubes Manager.
Qubes OS version R4.0
Affected component(s) or functionality Qubes Updater / Fedora-31 template, Fedora-32 template
Brief summary After updating the Fedora-31 and 32 templates today, the Qubes Updater GUI can no longer update Templates. This is likely due to the salt/salt-ssh updates upstream.
To Reproduce
default-mgmt-dvm should be set to fedora-31 (this is from a 4.0.3 iso install)
Expected behavior The updater should update the machine
Actual behavior The updater fails with the following error:
Screenshots
Additional context
Contents of /var/log/xen/console/guest-disp-mgmt-debian-10
[ 0.000000] Linux version 4.19.152-1.pvops.qubes.x86_64 (user@build-fedora4) (gcc version 6.4.1 20170727 (Red Hat 6.4.1-1) (GCC)) #1 SMP Sun Oct 18 12:03:19 UTC 2020 [ 0.000000] Command line: root=/dev/mapper/dmroot ro nomodeset console=hvc0 rd_NO_PLYMOUTH rd.plymouth.enable=0 plymouth.enable=0 xen_scrub_pages=0 nopat [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x008: 'MPX bounds registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x010: 'MPX CSR' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: xstate_offset[3]: 832, xstate_sizes[3]: 64 [ 0.000000] x86/fpu: xstate_offset[4]: 896, xstate_sizes[4]: 64 [ 0.000000] x86/fpu: Enabled xstate features 0x1f, context size is 960 bytes, using 'compacted' format. [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable [ 0.000000] BIOS-e820: [mem 0x00000000000a0000-0x00000000000fefff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000ff000-0x00000000000fffff] ACPI data [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000258003ff] usable [ 0.000000] BIOS-e820: [mem 0x00000000fc000000-0x00000000fc007fff] ACPI data [ 0.000000] x86/PAT: PAT support disabled. [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] DMI not present or invalid. [ 0.000000] Hypervisor detected: Xen HVM [ 0.000000] Xen version 4.8. [ 0.122477] tsc: Fast TSC calibration failed [ 0.122480] tsc: Detected 1992.123 MHz processor [ 0.122500] last_pfn = 0x25800 max_arch_pfn = 0x400000000 [ 0.122545] Disabled [ 0.122557] CPU MTRRs all blank - virtualized system. [ 0.122559] x86/PAT: Configuration [0-7]: WB WT UC- UC WB WT UC- UC [ 0.128961] Using GB pages for direct mapping [ 0.129066] RAMDISK: [mem 0x02e2d000-0x03dc3fff] [ 0.129116] ACPI: Early table checksum verification disabled [ 0.129119] ACPI: RSDP 0x00000000000FFFC0 000024 (v02 Xen ) [ 0.129121] ACPI: XSDT 0x00000000FC007F70 000034 (v01 Xen HVM 00000000 HVML 00000000) [ 0.129125] ACPI: FACP 0x00000000FC007D70 00010C (v05 Xen HVM 00000000 HVML 00000000) [ 0.129129] ACPI: DSDT 0x00000000FC001050 006C9B (v05 Xen HVM 00000000 INTL 20160831) [ 0.129131] ACPI: FACS 0x00000000FC001010 000040 [ 0.129133] ACPI: FACS 0x00000000FC001010 000040 [ 0.129134] ACPI: APIC 0x00000000FC007E80 00003C (v02 Xen HVM 00000000 HVML 00000000) [ 0.129296] No NUMA configuration found [ 0.129297] Faking a node at [mem 0x0000000000000000-0x00000000257fffff] [ 0.129305] NODE_DATA(0) allocated [mem 0x257d5000-0x257fffff] [ 0.129878] Zone ranges: [ 0.129881] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.129883] DMA32 [mem 0x0000000001000000-0x00000000257fffff] [ 0.129884] Normal empty [ 0.129884] Device empty [ 0.129885] Movable zone start for each node [ 0.129887] Early memory node ranges [ 0.129888] node 0: [mem 0x0000000000001000-0x000000000009ffff] [ 0.129889] node 0: [mem 0x0000000000100000-0x00000000257fffff] [ 0.130000] Zeroed struct page in unavailable ranges: 10337 pages [ 0.130001] Initmem setup node 0 [mem 0x0000000000001000-0x00000000257fffff] [ 0.132784] ACPI: No IOAPIC entries present [ 0.132784] Using ACPI for processor (LAPIC) configuration information [ 0.132786] TSC deadline timer available [ 0.132797] smpboot: Allowing 2 CPUs, 0 hotplug CPUs [ 0.132804] [mem 0x25800400-0xfbffffff] available for PCI devices [ 0.132804] Booting paravirtualized kernel on Xen PVH [ 0.132807] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.212567] random: get_random_bytes called from start_kernel+0xc3/0x763 with crng_init=0 [ 0.212576] setup_percpu: NR_CPUS:64 nr_cpumask_bits:64 nr_cpu_ids:2 nr_node_ids:1 [ 0.212665] percpu: Embedded 44 pages/cpu s143360 r8192 d28672 u1048576 [ 0.212689] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes) [ 0.212693] Built 1 zonelists, mobility grouping on. Total pages: 151081 [ 0.212694] Policy zone: DMA32 [ 0.212695] Kernel command line: root=/dev/mapper/dmroot ro nomodeset console=hvc0 rd_NO_PLYMOUTH rd.plymouth.enable=0 plymouth.enable=0 xen_scrub_pages=0 nopat [ 0.212720] You have booted with nomodeset. This means your GPU drivers are DISABLED [ 0.212721] Any video related functionality will be severely degraded, and you may not even be able to suspend the system properly [ 0.212721] Unless you actually understand what nomodeset does, you should reboot without enabling it [ 0.214840] Memory: 556180K/614012K available (12300K kernel code, 1493K rwdata, 3892K rodata, 2660K init, 4392K bss, 57832K reserved, 0K cma-reserved) [ 0.214908] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 [ 0.214914] Kernel/User page tables isolation: enabled [ 0.214952] ftrace: allocating 38300 entries in 150 pages [ 0.226359] rcu: Hierarchical RCU implementation. [ 0.226362] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2. [ 0.226363] Tasks RCU enabled. [ 0.226363] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2 [ 0.228083] Using NULL legacy PIC [ 0.228085] NR_IRQS: 4352, nr_irqs: 48, preallocated irqs: 0 [ 0.228100] xen:events: Using FIFO-based ABI [ 0.228106] xen:events: Xen HVM callback vector for event delivery is enabled [ 0.228112] rcu: Offload RCU callbacks from CPUs: (none). [ 0.228140] Console: colour dummy device 80x25 [ 0.229066] console [hvc0] enabled [ 0.229086] ACPI: Core revision 20180810 [ 0.229133] ACPI: setting ELCR to 0200 (from ffff) [ 0.229160] Failed to register legacy timer interrupt [ 0.229180] APIC: Switch to symmetric I/O mode setup [ 0.229192] x2apic: IRQ remapping doesn't support X2APIC mode [ 0.229351] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x396e3a4b97e, max_idle_ns: 881590739092 ns [ 0.229381] Calibrating delay loop (skipped), value calculated using timer frequency.. 3984.24 BogoMIPS (lpj=1992123) [ 0.229400] pid_max: default: 32768 minimum: 301 [ 0.229436] Security Framework initialized [ 0.229444] Yama: becoming mindful. [ 0.229455] AppArmor: AppArmor disabled by boot time parameter [ 0.229614] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes) [ 0.229713] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes) [ 0.229735] Mount-cache hash table entries: 2048 (order: 2, 16384 bytes) [ 0.229750] Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes) [ 0.230033] Last level iTLB entries: 4KB 128, 2MB 8, 4MB 8 [ 0.230044] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4 [ 0.230056] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization [ 0.230073] Spectre V2 : Mitigation: Full generic retpoline [ 0.230083] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch [ 0.230096] Spectre V2 : Enabling Restricted Speculation for firmware calls [ 0.230112] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier [ 0.230128] Spectre V2 : User space: Mitigation: STIBP via seccomp and prctl [ 0.230142] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp [ 0.230176] SRBDS: Unknown: Dependent on hypervisor status [ 0.230185] MDS: Mitigation: Clear CPU buffers [ 0.230371] Freeing SMP alternatives memory: 32K [ 0.230371] clocksource: xen: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.230371] installing Xen timer for CPU 0 [ 0.230371] smpboot: CPU0: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz (family: 0x6, model: 0x8e, stepping: 0xa) [ 0.230371] cpu 0 spinlock event irq 5 [ 0.230371] Performance Events: unsupported p6 CPU model 142 no PMU driver, software events only. [ 0.230371] rcu: Hierarchical SRCU implementation. [ 0.230371] random: crng done (trusting CPU's manufacturer) [ 0.230371] NMI watchdog: Perf NMI watchdog permanently disabled [ 0.230371] smp: Bringing up secondary CPUs ... [ 0.230371] installing Xen timer for CPU 1 [ 0.230418] x86: Booting SMP configuration: [ 0.230427] .... node #0, CPUs: #1 [ 0.230977] cpu 1 spinlock event irq 11 [ 0.230977] smp: Brought up 1 node, 2 CPUs [ 0.230977] smpboot: Max logical packages: 1 [ 0.230977] smpboot: Total of 2 processors activated (7968.49 BogoMIPS) [ 0.230977] devtmpfs: initialized [ 0.230977] x86/mm: Memory block size: 128MB [ 0.231630] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.231630] futex hash table entries: 512 (order: 3, 32768 bytes) [ 0.231630] pinctrl core: initialized pinctrl subsystem [ 0.251832] RTC time: 165:165:165, date: 165/165/65 [ 0.251999] NET: Registered protocol family 16 [ 0.252022] xen:grant_table: Grant tables using version 1 layout [ 0.252050] Grant table initialized [ 0.252111] audit: initializing netlink subsys (disabled) [ 0.252149] audit: type=2000 audit(1604675282.025:1): state=initialized audit_enabled=0 res=1 [ 0.252430] cpuidle: using governor menu [ 0.252493] ACPI: bus type PCI registered [ 0.252527] PCI: Fatal: No config space access function found [ 0.253831] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.253831] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.253831] cryptd: max_cpu_qlen set to 1000 [ 0.253831] ACPI: Added _OSI(Module Device) [ 0.253831] ACPI: Added _OSI(Processor Device) [ 0.253831] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.254405] ACPI: Added _OSI(Processor Aggregator Device) [ 0.254420] ACPI: Added _OSI(Linux-Dell-Video) [ 0.254431] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio) [ 0.256762] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.257016] ACPI: SCI (ACPI GSI 9) not registered [ 0.260684] ACPI: Interpreter enabled [ 0.260696] ACPI: (supports S0) [ 0.260705] ACPI: Using platform specific model for interrupt routing [ 0.260748] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.260895] ACPI: Enabled 1 GPEs in block 00 to 0F [ 0.260916] ACPI Error: No handler or method for GPE 00, disabling event (20180810/evgpe-841) [ 0.260942] ACPI Error: No handler or method for GPE 01, disabling event (20180810/evgpe-841) [ 0.260976] ACPI Error: No handler or method for GPE 03, disabling event (20180810/evgpe-841) [ 0.261002] ACPI Error: No handler or method for GPE 04, disabling event (20180810/evgpe-841) [ 0.261028] ACPI Error: No handler or method for GPE 05, disabling event (20180810/evgpe-841) [ 0.261053] ACPI Error: No handler or method for GPE 06, disabling event (20180810/evgpe-841) [ 0.261080] ACPI Error: No handler or method for GPE 07, disabling event (20180810/evgpe-841) [ 0.265538] xen:balloon: Initialising balloon driver [ 0.268403] vgaarb: loaded [ 0.268458] SCSI subsystem initialized [ 0.268493] ACPI: bus type USB registered [ 0.268493] usbcore: registered new interface driver usbfs [ 0.268493] usbcore: registered new interface driver hub [ 0.268493] usbcore: registered new device driver usb [ 0.268493] pps_core: LinuxPPS API ver. 1 registered [ 0.268493] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo GiomettiSolutions you've tried