search

QubesOS / qubes-issues

The Qubes OS Project issue tracker
https://www.qubes-os.org/doc/issue-tracking/
534 stars 46 forks source link

Atheros wired Ethernet fails to connect #7052

Closed rlew-metastability closed 1 year ago

rlew-metastability commented 2 years ago

How to file a helpful issue

Qubes OS release

4.1-rc1

Brief summary

Atheros E2400 wired Ethernet no longer working - does not try to connect. Hardware is known to be working

Steps to reproduce

Install Qubes-OS 4.1-rc1 on a Dell XPS 8930 and verify that the cabled Ethernet no longer functions.

Expected behavior

Wired Ethernet works

Actual behavior

Wired Ethernet does NOT work

EgosOwn commented 2 years ago

I'm having the same issue with the E2500, also verified to work and worked on 4.0, can we edit the title?

andrewdavidwong commented 2 years ago

I'm having the same issue with the E2500, also verified to work and worked on 4.0, can we edit the title?

Title updated.

GaijinSystems commented 2 years ago

I'm experiencing the same issue in 4.1.0-rc4

unman commented 2 years ago

Please remember that this is not a place for user support - consider taking such problems to the Forum or the qubes-users mailing list in the first instance.

As we are here, please provide some more basic information:

Which template are you using for sys-net? Does the interface appear in sys-net? Are there any errors in the sys-net logs?

If you change the template used by sys-net does that make any difference to the error or the log entries relating to the NIC?

GaijinSystems commented 2 years ago

Hardware works correctly in vanilla Debian 11 (without non-free firmware installed). Performed multiple clean installs of 4.1.0 (changing default template) and the problem persists. This problem did not exist in 4.0.x or 3.2.x

Ethernet does not work with Debian or Fedora as the sys-net template. The interface does appear, but doesn't seem to detect the link or negotiate an IP. (sadly, the output of "ip link" is the one thing I forgot to copy/paste/save to external file.) ens6 is present, but link state is always DOWN

00:06.0 Ethernet controller: Qualcomm Atheros QCA8171 Gigabit Ethernet (rev 10)
    Subsystem: Dell QCA8171 Gigabit Ethernet
    Physical Slot: 6
    Flags: bus master, fast devsel, latency 0, IRQ 70
    Memory at f2000000 (64-bit, non-prefetchable) [size=256K]
    I/O ports at c200 [size=128]
    Capabilities: [40] Power Management version 3
    Capabilities: [58] Express Endpoint, MSI 00
    Capabilities: [c0] MSI: Enable- Count=1/1 Maskable- 64bit+
    Kernel driver in use: alx
    Kernel modules: alx
root@sys-net:/mnt# lsmod | grep alx
alx                    57344  0
mdio                   16384  1 alx
user@sys-net:~$ sudo modinfo alx
filename:       /lib/modules/5.10.90-1.fc32.qubes.x86_64/kernel/drivers/net/ethernet/atheros/alx/alx.ko
license:        GPL
description:    Qualcomm Atheros(R) AR816x/AR817x PCI-E Ethernet Network Driver
author:         Qualcomm Corporation, <nic-devel@qualcomm.com>
author:         Johannes Berg <johannes@sipsolutions.net>
alias:          pci:v00001969d000010A0sv*sd*bc*sc*i*
alias:          pci:v00001969d000010A1sv*sd*bc*sc*i*
alias:          pci:v00001969d00001090sv*sd*bc*sc*i*
alias:          pci:v00001969d0000E0B1sv*sd*bc*sc*i*
alias:          pci:v00001969d0000E0A1sv*sd*bc*sc*i*
alias:          pci:v00001969d0000E091sv*sd*bc*sc*i*
alias:          pci:v00001969d00001091sv*sd*bc*sc*i*
depends:        mdio
retpoline:      Y
intree:         Y
name:           alx
vermagic:       5.10.90-1.fc32.qubes.x86_64 SMP mod_unload 
sig_id:         PKCS#7
signer:         Build time autogenerated kernel key
sig_key:        0C:A0:6D:08:CB:A6:17:19:CD:AF:B4:CB:C1:24:21:F4:9F:FC:52:1A
sig_hashalgo:   sha512
signature:      77:3D:C6:59:08:7C:45:1D:1A:66:E1:F9:ED:D4:D6:99:6D:91:3C:AA:
        4A:03:64:65:75:36:20:D5:E4:D9:70:9E:9C:34:37:10:FF:A9:BC:BA:
        A7:5F:43:BE:DC:19:0F:A4:74:E7:3C:B3:12:10:FA:1E:36:14:3A:7E:
        CD:18:5B:33:EE:21:CB:B5:CC:18:54:F7:77:D8:F7:69:35:9B:14:E6:
        BF:51:A3:7D:13:B6:08:AA:14:C2:6C:0B:40:56:09:BE:85:40:B0:6E:
        61:8F:35:15:80:76:0B:EE:E6:90:05:C8:10:80:A4:B0:56:4F:1A:5F:
        79:A2:7F:C3:98:27:0F:3F:02:21:89:10:15:F3:80:9A:3B:75:87:EE:
        CD:A4:4C:C0:14:FE:76:8C:88:AB:81:F0:E4:1B:BB:92:68:9E:6E:61:
        17:A4:88:75:CE:2C:A7:71:E2:17:D2:C0:ED:99:3F:B5:A8:FE:21:3C:
        C3:80:42:A5:82:BA:6D:42:8A:A4:03:CA:62:05:35:02:F9:D0:C9:96:
        F9:86:0A:38:D3:BB:40:EC:DE:DA:D2:DE:DB:E9:AF:89:49:DF:AD:2B:
        0C:13:AA:8E:A8:CB:53:30:80:72:88:34:B3:55:B3:0B:FA:27:3E:70:
        11:A5:67:89:91:E3:6D:F2:1E:D2:66:29:5B:19:DF:B3:15:61:B1:56:
        29:46:CE:8B:C1:FC:D6:4F:3E:D4:83:0C:AF:2C:60:D5:5E:69:CC:16:
        A4:81:CB:DB:07:8A:00:72:62:92:28:2A:73:B6:7E:09:8B:03:16:9A:
        1C:7A:D2:AC:54:89:C1:27:FB:A9:B9:D6:0A:7C:61:1E:86:DE:86:4E:
        D2:9D:4D:ED:BE:7E:07:8A:84:FD:91:FA:21:20:6B:AC:14:30:72:99:
        D7:76:62:DF:DA:52:82:DB:E1:90:17:4B:FE:35:02:32:E9:22:64:FC:
        01:DE:8F:A5:AD:23:43:95:4C:16:B5:E6:68:4C:9D:52:08:CC:68:45:
        E7:5E:92:60:3F:42:26:EA:43:3B:76:7C:7D:CF:07:84:B3:00:41:B1:
        C5:33:5C:8B:E4:10:16:F3:F3:94:15:36:72:1D:D7:83:EC:36:01:D0:
        81:3A:70:7E:4D:3B:73:D7:3A:47:55:8C:F9:BB:BB:B4:09:1C:2B:32:
        DC:4E:65:CE:B1:E4:52:AC:D4:36:1F:D4:F1:52:5F:D8:DA:D7:DC:D3:
        86:C4:BF:1F:37:D7:34:4A:BF:95:BC:D4:EB:5E:46:55:FC:79:7E:91:
        9B:4C:BF:76:7D:63:80:03:14:F3:BD:78:E5:0A:06:1D:67:DE:5E:67:
        A2:89:15:23:5A:39:4C:D1:25:07:A6:68
root@sys-net:/mnt# dmesg | egrep -i 'alx|eth|ens6'
[    1.988611] systemd[1]: /lib/systemd/system/qubes-db.service:11: Standard output type syslog is obsolete, automatically updating to journal. Please update your unit file, and consider removing the setting altogether.
[    2.032618] systemd[1]: /lib/systemd/system/qubes-gui-agent.service:15: Standard output type syslog is obsolete, automatically updating to journal. Please update your unit file, and consider removing the setting altogether.
[    2.778103] alx 0000:00:06.0 eth0: Qualcomm Atheros AR816x/AR817x Ethernet [f4:8e:38:XX:XX:XX]
[    3.093454] alx 0000:00:06.0 ens6: renamed from eth0
[    5.464293] alx 0000:00:06.0 ens6: Enabling MSI-X interrupts failed!

Below is every reference to the device in the logs (error or otherwise):

root@sys-net:/mnt# find /var/log/ -type f | xargs egrep -i 'alx|eth|ens6'
/var/log/messages:Feb  5 20:00:44 localhost kernel: [    2.778103] alx 0000:00:06.0 eth0: Qualcomm Atheros AR816x/AR817x Ethernet [f4:8e:38:XX:XX:XX]
/var/log/messages:Feb  5 20:00:44 localhost kernel: [    3.093454] alx 0000:00:06.0 ens6: renamed from eth0
/var/log/messages:Feb  5 20:00:45 localhost kernel: [    5.464293] alx 0000:00:06.0 ens6: Enabling MSI-X interrupts failed!
/var/log/syslog:Feb  5 15:18:49 localhost systemd-udevd[356]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
/var/log/syslog:Feb  5 20:00:44 localhost systemd-udevd[323]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
/var/log/syslog:Feb  5 20:00:44 localhost systemd-udevd[326]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
/var/log/syslog:Feb  5 20:00:44 localhost systemd-udevd[323]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
/var/log/syslog:Feb  5 20:00:44 localhost systemd-udevd[320]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
/var/log/syslog:Feb  5 20:00:44 localhost systemd-udevd[320]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
/var/log/syslog:Feb  5 20:00:44 localhost kernel: [    2.778103] alx 0000:00:06.0 eth0: Qualcomm Atheros AR816x/AR817x Ethernet [f4:8e:38:XX:XX:XX]
/var/log/syslog:Feb  5 20:00:44 localhost kernel: [    3.093454] alx 0000:00:06.0 ens6: renamed from eth0
/var/log/syslog:Feb  5 20:00:45 localhost NetworkManager[543]: <info>  [1644112845.1855] manager: (ens6): new Ethernet device (/org/freedesktop/NetworkManager/Devices/2)
/var/log/syslog:Feb  5 20:00:45 localhost NetworkManager[543]: <info>  [1644112845.1889] settings: (ens6): created default wired connection 'Wired connection 1'
/var/log/syslog:Feb  5 20:00:45 localhost NetworkManager[543]: <info>  [1644112845.1892] device (ens6): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external')
/var/log/syslog:Feb  5 20:00:45 localhost kernel: [    5.464293] alx 0000:00:06.0 ens6: Enabling MSI-X interrupts failed!
/var/log/syslog:Feb  5 20:00:48 localhost NetworkManager[543]: <info>  [1644112848.6316] manager: (vif3.0): new Ethernet device (/org/freedesktop/NetworkManager/Devices/5)
/var/log/syslog:Feb  5 20:00:49 localhost systemd-udevd[994]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
/var/log/kern.log:Feb  5 20:00:44 localhost kernel: [    2.778103] alx 0000:00:06.0 eth0: Qualcomm Atheros AR816x/AR817x Ethernet [f4:8e:38:XX:XX:XX]
/var/log/kern.log:Feb  5 20:00:44 localhost kernel: [    3.093454] alx 0000:00:06.0 ens6: renamed from eth0
/var/log/kern.log:Feb  5 20:00:45 localhost kernel: [    5.464293] alx 0000:00:06.0 ens6: Enabling MSI-X interrupts failed!
/var/log/daemon.log:Feb  5 15:18:49 localhost systemd-udevd[356]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
/var/log/daemon.log:Feb  5 20:00:44 localhost systemd-udevd[323]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
/var/log/daemon.log:Feb  5 20:00:44 localhost systemd-udevd[326]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
/var/log/daemon.log:Feb  5 20:00:44 localhost systemd-udevd[323]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
/var/log/daemon.log:Feb  5 20:00:44 localhost systemd-udevd[320]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
/var/log/daemon.log:Feb  5 20:00:44 localhost systemd-udevd[320]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
/var/log/daemon.log:Feb  5 20:00:45 localhost NetworkManager[543]: <info>  [1644112845.1855] manager: (ens6): new Ethernet device (/org/freedesktop/NetworkManager/Devices/2)
/var/log/daemon.log:Feb  5 20:00:45 localhost NetworkManager[543]: <info>  [1644112845.1889] settings: (ens6): created default wired connection 'Wired connection 1'
/var/log/daemon.log:Feb  5 20:00:45 localhost NetworkManager[543]: <info>  [1644112845.1892] device (ens6): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external')
/var/log/daemon.log:Feb  5 20:00:48 localhost NetworkManager[543]: <info>  [1644112848.6316] manager: (vif3.0): new Ethernet device (/org/freedesktop/NetworkManager/Devices/5)
/var/log/daemon.log:Feb  5 20:00:49 localhost systemd-udevd[994]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
marmarek commented 2 years ago

kernel: [ 5.464293] alx 0000:00:06.0 ens6: Enabling MSI-X interrupts failed!

MSI-X is not currently supported under Xen. It should fallback to MSI or INTx, although from the messages above it isn't clear if it happened (on the lspci output, MSI is disabled though...). I'm not sure if that's the reason for the issue, or what specifically changed that it wasn't an issue before...

Can you post also lspci -v output from dom0 of the same device?

marmarek commented 2 years ago

You can try adding pci=nomsi to the sys-net's kernel options (see qvm-prefs tool).

GaijinSystems commented 2 years ago

Thank you for the suggestion, @marmarek . Adding "pci=nomsi" to the kernel options worked.

Here's the device info from dom0:

04:00.0 Ethernet controller: Qualcomm Atheros QCA8171 Gigabit Ethernet (rev 10)
    Subsystem: Dell Device 072a
    Flags: bus master, fast devsel, latency 0, IRQ 19
    Memory at df100000 (64-bit, non-prefetchable) [size=256K]
    I/O ports at d000 [size=128]
    Capabilities: [40] Power Management version 3
    Capabilities: [58] Express Endpoint, MSI 00
    Capabilities: [c0] MSI: Enable- Count=1/16 Maskable+ 64bit+
    Capabilities: [d8] MSI-X: Enable- Count=16 Masked-
    Capabilities: [100] Advanced Error Reporting
    Capabilities: [180] Device Serial Number ff-db-02-29-f4-8e-38-ff
    Kernel driver in use: pciback
    Kernel modules: alx
EgosOwn commented 2 years ago

I can confirm that pci=nomsi worked on my e2500 as well. Thank you!

(Side note, the man page says only PV domains can use kernelopts in qvm-prefs but thats not true it seems)

RaphaelBalthazar commented 2 years ago

Thanks a lot @marmarek I can also confirm that this is working for my Qualcomm Atheros QCA8172 Ethernet!

marmarek commented 2 years ago

Capabilities: [c0] MSI: Enable- Count=1/16 Maskable+ 64bit+ Capabilities: [d8] MSI-X: Enable- Count=16 Masked-

Ok, so there is both MSI and MSI-X. Apparently the driver tries only MSI-X (which is not yet supported in Xen HVM) and doesn't fallback to MSI. The pci=nomsi disables both, and that works too (I haven't found an option to disable just MSI-X).

ghost commented 2 years ago

You can try adding pci=nomsi to the sys-net's kernel options (see qvm-prefs tool).

n00b here. What's the exact command? I tried "qvm-prefs pci=nomsi" an dit said "no such domain: 'pci=nomsi"". I'm assuming I need to tell it to apply to sys-usb somehow?

DemiMarie commented 2 years ago

@natebartram qvm-prefs sys-net kernelopts "pci=nomsi <whatever was there before>"

ghost commented 2 years ago

@natebartram qvm-prefs sys-net kernelopts "pci=nomsi <whatever was there before>"

The only part I'm unclear on is "whatever was there before." What does that mean? Is that the name of my network device? Sorry, I know that's probably a really dumb question, I just know how easy it is to mess this stuff up and I don't want to brick it.

DemiMarie commented 2 years ago

@natebartram qvm-prefs sys-net kernelopts "pci=nomsi <whatever was there before>"

The only part I'm unclear on is "whatever was there before." What does that mean? Is that the name of my network device? Sorry, I know that's probably a really dumb question, I just know how easy it is to mess this stuff up and I don't want to brick it.

I would use s=$(qvm-prefs -- sys-net kernelopts) && qvm-prefs -- sys-net kernelopts "pci=nomsi $s", but you should be sure you understand what that does before you run it.

ghost commented 2 years ago

I would use s=$(qvm-prefs -- sys-net kernelopts) && qvm-prefs -- sys-net kernelopts "pci=nomsi $s", but you should be sure you understand what that does before you run it.

I can tell some of it. I know "qvm-prefs" is about changing the options of the kernel, same with "kernelopts." I recognize "sys-net," so I can kind of get the gist of what it does. It's mainly the syntax I think I don't understand (except the "nomsi" part, I'm totally clueless on what that means). Are there any sources you'd recommend (aside from the Qubes OS documentation on their website) to learn more? I don't want to waste anybody's time by asking them to spoonfeed me explanations, but I do agree that historically when I understand the commands it becomes a thousand times easier to fix my mistakes or modify it correctly when I hit snags.

DemiMarie commented 2 years ago

@natebartram the syntax is shell command substitution syntax

kenjtoronto commented 2 years ago

Thank you for your support DemiMarie with ref to command line s=$(qvm-prefs -- sys-net kernelopts) && qvm-prefs -- sys-net kernelopts "pci=nomsi $s"

It worked for me also, for my Qubes 4.1 OS, Ethernet controller ; Qualcomm Atheros killer E220x Gigabit Ethernet Controller Since this command line is coming from someone ( yourself) that I feel is more than qualified to present this awesome command line, can I trust that this is a verified solution, or do I need to use 4.0 until kernel is updated. Understanding this command line is far beyond my capabilities, I'm a 59 year old airline mechanic who has fallen in love with Qubes , and use Qubes for everything ( at home ). Please say yes it's verified.

kenjtoronto commented 2 years ago

@natebartram the syntax is shell command substitution syntax

Thank you for your support DemiMarie with ref to command line s=$(qvm-prefs -- sys-net kernelopts) && qvm-prefs -- sys-net kernelopts "pci=nomsi $s"

It worked for me also, for my Qubes 4.1 OS, Ethernet controller ; Qualcomm Atheros killer E220x Gigabit Ethernet Controller Since this command line is coming from someone ( yourself) that I feel is more than qualified to present this awesome command line, can I trust that this is a verified solution, or do I need to use 4.0 until kernel is updated. Understanding this command line is far beyond my capabilities, I'm a 59 year old airline mechanic who has fallen in love with Qubes , and use Qubes for everything ( at home ). Please say yes it's verified.

kenjtoronto commented 2 years ago

I would use s=$(qvm-prefs -- sys-net kernelopts) && qvm-prefs -- sys-net kernelopts "pci=nomsi $s", but you should be sure you understand what that does before you run it.

I can tell some of it. I know "qvm-prefs" is about changing the options of the kernel, same with "kernelopts." I recognize "sys-net," so I can kind of get the gist of what it does. It's mainly the syntax I think I don't understand (except the "nomsi" part, I'm totally clueless on what that means). Are there any sources you'd recommend (aside from the Qubes OS documentation on their website) to learn more? I don't want to waste anybody's time by asking them to spoonfeed me explanations, but I do agree that historically when I understand the commands it becomes a thousand times easier to fix my mistakes or modify it correctly when I hit snags.

Thank you for your support DemiMarie with ref to command line s=$(qvm-prefs -- sys-net kernelopts) && qvm-prefs -- sys-net kernelopts "pci=nomsi $s"

It worked for me also, for my Qubes 4.1 OS, Ethernet controller ; Qualcomm Atheros killer E220x Gigabit Ethernet Controller Since this command line is coming from someone ( yourself) that I feel is more than qualified to present this awesome command line, can I trust that this is a verified solution, or do I need to use 4.0 until kernel is updated. Understanding this command line is far beyond my capabilities, I'm a 59 year old airline mechanic who has fallen in love with Qubes , and use Qubes for everything ( at home ). Please say yes it's verified.

ghost commented 2 years ago

Sorry, I forgot to follow up. This command did work, thank you very much.

kenjtoronto commented 2 years ago

Hey Nate, with reference to that command line. s=$(qvm-prefs -- sys-net kernelopts) && qvm-prefs -- sys-net kernelopts "pci=nomsi $s" I'm not familiar with this forum, I would believe that Demimarie is an extremely trust worthy support member, I get the impression that you also feel the same.

andrewdavidwong commented 2 years ago

@kenjtoronto: The issue tracker is not a support forum, but we have places you can go to ask for help and support.

As for whether you can trust someone who's claiming to provide you with support, that's a very good question. It's good that you're questioning whether you should blindly enter commands in dom0 given to you by a stranger over the internet. Please read this:

https://www.qubes-os.org/support/#staying-safe

unman commented 2 years ago

It is good that people question any advice given here, even if this is not a support forum. (It often looks as if it is.)

The command line here is simple: The first part s=$(qvm-prefs -- sys-net kernelopts), reads the current kernelopts for sys-net and stores it in a variable, s.

The && links two commands, only if the first completes successfully.

The second command sets the kernelopts for sys-net using qvm-prefs. The new setting is "pci=nomsi" followed by the contents of variable s, which is the current kernelopts.

The effect is to prepend pci=nomsi to the current value for kernelopts.

kenjtoronto commented 2 years ago

Thanks Andrew. I'm not very little familiarity with forums, having difficulty navigating, this post was provided to me as a wonderful solution in the Qubes forum, and blindly continued participating from that post. I obviously broke some protocol, very sorry. With a little better understanding, and a bit less blind ; ) , I will be more careful. Thank you for all your service Andrew.

kenjtoronto commented 2 years ago

It is good that people question any advice given here, even if this is not a support forum. (It often looks as if it is.) The command line here is simple: The first part s=$(qvm-prefs -- sys-net kernelopts), reads the current kernelopts for sys-net and stores it in a variable, s. The && links two commands, only if the first completes successfully. The second command sets the kernelopts for sys-net using qvm-prefs. The new setting is "pci=nomsi" followed by the contents of variable s, which is the current kernelopts. The effect is to prepend pci=nomsi to the current value for kernelopts.

Thanks for your feed back unman, this explanation is very clear thank you. I be on qubes support and maybe in the distant future I'll find my way back to Qubes issues with a more relevant post : )

ExtractedViaMev commented 1 year ago

Hi u would like to know where i place this command is it in dom0 or sys-net terminal command line and i get the message qvm-prefs command not found

andrewdavidwong commented 1 year ago

@ExtractedViaMev: Please note that the issue tracker (qubes-issues) is not intended to be a place for fielding questions. Instead, we have other venues meant for asking questions, asking for help, and having discussions. (By contrast, the issue tracker is more of a technical tool intended to support our developers in their work.) Thank you for your understanding!