morrownr / USB-WiFi

USB WiFi Adapter Information for Linux
2.6k stars 172 forks source link

ANDDEAR MT7612U004 Adapter Issue on Raspberry Pi 4B with USB 3.0 #75

Open amisix opened 2 years ago

amisix commented 2 years ago

@morrownr - I'm having a strange issue with the ANDDDEAR MT7612U004 adapter that I recently picked up where it won't initialize properly on the USB 3.0 bus of my Raspberry Pi 4B but it will work fine on the USB 2.0 bus (Kali 2022.2). I originally evaluated the adapter with a CM4 and had no issues because, USB 2.0. The issue occurs whether I have a single adapter or multiple adapters connected. If I take the adapter and swap it to USB 2.0, the issue disappears and the adapter works fine.

I think this may impact other users of this adapter if they choose to use it on a Raspberry Pi 4B with USB 3.0. We may need to modify or remove my review depending on the solution.

lsusb freezes/locks up & there is no output iw dev output does not show the adapter

EDIT: This thread got really long. Here's a recap with troubleshooting and a workaround.

morrownr commented 2 years ago

Hi @amisix

Hmmm... so Kali 2022.2 on a RasPi4B only while using a USB3 port. My first thought is to say bad words about the selection of the USB3 hardward in the RasPi4B. There are probably thousands of Pi4B users that have had issues with this.

Let's do some troubleshooting:

Do you have an extra sd card that you could do a clean installation of the RasPiOS v 2022-04-04? If so, make it happen and see if you get the same problem.

Would you mind also try the ALFA ACM with the USB3 port with Kali 2022.2 to see if you have the same problem?

Have you tested the adapter in a x86/amd64 based system with Linux installed to see if you see the problem?

Hopefully that will help us narrow things down. Not long ago a patch was applied to the RasPiOS USB3 driver due to a hardware incompatibility. The problem was not in the Linux driver but was a hardware bug. I can be more specific if depending on what we find.

Regards

amisix commented 2 years ago

My first thought is to say bad words about the selection of the USB3 hardware in the RasPi4B. There are probably thousands of Pi4B users that have had issues with this.

Fun times. Hopefully whatever we find out can be useful.. In a previous post you had mentioned bugs in the Raspberry Pi 4B USB controller, I was thinking we might end up there with this but I'm not so sure now after testing it with the Alfa ACM.

Do you have an extra sd card that you could do a clean installation of the RasPiOS v 2022-04-04? If so, make it happen and see if you get the same problem.

Sure do - I'll do that.

Would you mind also try the ALFA ACM with the USB3 port with Kali 2022.2...

Done. I do not have the same problem with the ALFA ACM in Kali 2022.2. It initializes properly without error.

Have you tested the adapter in a x86/amd64 based system with Linux installed...

Not yet but I have a live boot Kali image ready to go. I'll run it up to see what comes of it.

Thanks.

morrownr commented 2 years ago

Fun times. Hopefully whatever we find out can be useful.. In a previous post you had mentioned bugs in the Raspberry Pi 4B USB controller, I was thinking we might end up there with this but I'm not so sure now after testing it with the Alfa ACM.

That the ALFA ACM is working is just a data point. It does not rule out the Pi4B USB3 hardware being the problem imho. The 8812au driver has worked fine on the Pi4B while the 8812bu driaver has had ongoing problems for a long time. It seems the fix that went into the Pi4B has fixed the problem with the 8812bu driver but I am not aware of that fix being upstreamed.

We will just have to search and since I don't have that adapter, all I can do forward suggestions.

Regards

morrownr commented 2 years ago

Let me be clear: The 2022-04-04 version of the RasPiOS should and does appear to have the RasPi4b USB3 fix but the version of Kali you are using most likely does not as the fix would need to work its way up into the kernel Linus maintains and then downward. That does not mean the problem you are seeing is the result of the problem that the fix addressed but the fact that it is happening only on a USB3 port makes one ponder the issue.

amisix commented 2 years ago

^ Understood.

The issue does occur in RasPiOS on both USB3 ports. The adapter will not initialize on USB3 Port 1 but it will initialize on USB3 Port 2 although it will not connect to any access points. (Edited: Previously I had stated USB3 Port 2 functioned, it does not)

There is an issue with the ANDDEAR adapter in Kali on x86 hardware although it is not identical it still leads to the adapter being non-functional. I can see the adapter but it won't connect to anything then it says "network disconnected" while no longer allowing me to even attempt a connection. Is it loading the incorrect firmware - ? There's a reference to mt7662.bin (not 7612)? Line 2 and Line 5.

Errors on x86 hardware:

[ 15.069700] mt76x2u 2-3:1.0: ASIC revision: 76120044 [ 15.069796] mt76x2u 2-3:1.0: firmware: direct-loading firmware mt7662_rom_patch.bin [ 15.069807] mt76x2u 2-3:1.0: ROM patch build: 20141115060606a [ 15.126913] Bluetooth: hci1: RTL: fw version 0x0999646b [ 15.201949] mt76x2u 2-3:1.0: firmware: direct-loading firmware mt7662.bin [ 15.201958] mt76x2u 2-3:1.0: Firmware Version: 0.0.00 [ 15.201962] mt76x2u 2-3:1.0: Build: 1 [ 15.201964] mt76x2u 2-3:1.0: Build Time: 201507311614____ [ 15.928821] ieee80211 phy1: Selected rate control algorithm 'minstrel_ht' [ 15.930020] usbcore: registered new interface driver mt76x2u [ 18.503647] r8169 0000:03:00.0: firmware: direct-loading firmware rtl_nic/rtl8168h-2.fw [ 18.533693] Generic FE-GE Realtek PHY r8169-0-300:00: attached PHY driver (mii_bus:phy_addr=r8169-0-300:00, irq=MAC) [ 18.725739] r8169 0000:03:00.0 eth0: Link is Down [ 18.741103] iwlwifi 0000:02:00.0: Applying debug destination EXTERNAL_DRAM [ 18.818936] iwlwifi 0000:02:00.0: Applying debug destination EXTERNAL_DRAM [ 18.820194] iwlwifi 0000:02:00.0: FW already configured (0) - re-configuring [ 18.855815] iwlwifi 0000:02:00.0: Applying debug destination EXTERNAL_DRAM [ 18.933961] iwlwifi 0000:02:00.0: Applying debug destination EXTERNAL_DRAM [ 18.935387] iwlwifi 0000:02:00.0: FW already configured (0) - re-configuring [ 20.885724] mt76x2u 2-3:1.0: mac specific condition occurred [ 21.005747] usb 2-3: USB disconnect, device number 3 [ 21.140248] mt76x2u 2-3:1.0: mac specific condition occurred [ 21.349695] mt76x2u 2-3:1.0: mac specific condition occurred [ 21.453699] mt76x2u 2-3:1.0: mac specific condition occurred [ 21.553693] mt76x2u 2-3:1.0: timed out waiting for pending tx [ 21.885886] usb 2-3: new SuperSpeed USB device number 4 using xhci_hcd [ 21.906677] usb 2-3: New USB device found, idVendor=0e8d, idProduct=7612, bcdDevice= 1.00 [ 21.906681] usb 2-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [ 21.906682] usb 2-3: Product: 802.11ac WLAN [ 21.906683] usb 2-3: Manufacturer: MediaTek Inc. [ 21.906684] usb 2-3: SerialNumber: 000000000 [ 22.038055] usb 2-3: reset SuperSpeed USB device number 4 using xhci_hcd

morrownr commented 2 years ago

The issue does not occur whatsoever in RasPiOS - the adapter works fine on USB 3.0 - able to see and connect to access points with no errors in the logs.

Hmmm... Kali is starting to look guilty.

There is an issue with the ANDDEAR adapter in Kali on x86 hardware although it is not identical it still leads to the adapter being non-functional.

Kali is really starting to look guilty.

Are you going to be able to test a non-Kali x86 Linux distro such as Ubuntu 22.04?

I can see the adapter but it won't connect to anything then it says "network disconnected" while no longer allowing me to even attempt a connection. Is it loading the incorrect firmware - ?

It is possible that the firmware is somehow messed up in Kali.

There's a reference to mt7662.bin (not 7612)? Line 2 and Line 5.

The mt7612u chipset requires two firmware files:

mt7662u.bin mt7662u_rom_patch.bin

My recommendation is that you see my guide:

https://github.com/morrownr/USB-WiFi/blob/main/How_to_Install_Firmware_for_Mediatek_based_USB_WiFi_adapters.md

What I think you should do at this point is download both files as instructed in the above document and then compare the files to what is in Kali. The files in Kali should be identical to what you download byte per byte. There is one thing to note, the mt7612u firmware files are usually installed in two different directories and there is a history to understand.

At one time, all firmware files were just dumped into lib/firmware but as time passed and more files were showing up, it was decided that each company have its own directory. All Mediatek firmware files are in /lib/firmware/mediatek however due to old code in the mt7612u driver, it looks in the old location which is lib/firmware. What most distros do is they put the two files in both locations so you should check the files in both locations.

The adventure continues...

amisix commented 2 years ago

Kali is really starting to look guilty.

We're narrowing it down... yay

Are you going to be able to test a non-Kali x86 Linux distro such as Ubuntu 22.04?

Yeah, I'll just create another live boot usb with Rufus or balena.

My recommendation is that you see my guide...

It could be the firmware? I'll follow your instructions and do a diff/hash comparison as suggested.

We shall see. Thanks for breaking it all out.

morrownr commented 2 years ago

It could be the firmware?

It could. These files get sent all over the place. One little bit gets flipped and now the file does operate as designed. Is it likely this is the problem? No, but firmware errors are what is showing up so we need to be 100% that firmware file integrity is not the problem. I am looking forward to your report from a test on an x86/amd64 system. So far, the only time we are seeing the problem is with Kali.

amisix commented 2 years ago

Ubuntu x86: No go, adapter not recognized at all. lsusb locks up when run, no output. iw dev shows no wireless device. dmesg showed another "mac specific condition occurred" with no additional details. I stopped there as I could go down another rabbit hole..

Ubuntu ARM/Raspberry Pi No go either. Adapter recognized by lsusb but iw dev and ifconfig shows no wireless device other than onboard. dmesg & logread show nothing of value/no visible errors.

Kali firmware: 1: Firmware content in /lib/firmware does not match firmware in /lib/firmware/mediatek.

2: Firmware in /lib/firmware is named mt7662.bin & mt7662_rom_patch.bin (missing "u" after mt7662 that's in /lib/firmware/mediatek/ filename. Example: mt7662u.bin)

3: Firmware in directory /lib/firmware/mediatek matches firmware from git (https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/mediatek)

Why the discrepancy between the firmware in /lib/firmware compared to /lib/firmware/mediatek (and git)? I copied the firmware from git to /lib/firmware (& rebooted) and the issue persists. I'm going to do it again for good measure but need a bit more time.

morrownr commented 2 years ago

Well. Interesting.

Let me dive into the code in the mt7612u driver so that we know exactly which firmware files the driver is looking for and what their location without a doubt.

amisix commented 2 years ago

Let me dive into the code in the mt7612u driver so that we know exactly which firmware files the driver is looking for and what their location without a doubt.

Awesome, thanks.

amisix commented 2 years ago

@morrownr RE: Kali Firmware

I removed the firmware from /lib/firmware/ then copied the firmware from /lib/firmware/mediatek in its place. I renamed the two .bin files to match what was already in /lib/firmware (removed the "u" after mt7662, Example: mt7662u.bin = mt7662.bin & mt7662u_rom_patch.bin = mt7662_rom_patch.bin)

Weirdest thing, the adapter now works on one USB 3.0 port (top) without errors, but not the other USB 3.0 port (bottom). The error I receive on the bottom USB 3.0 port is the same as the previous error:

mt76x2u 2-1:1.0: error: mt76x02u_mcu_wait_resp failed with -108 mt76x2u 2-1:1.0: mac specific condition occurred

BUT, one port is now working with the firmware swap.. thoughts?

EDIT: After additional testing both USB 3.0 ports are now working without errors. Apparently swapping the firmware worked.. ?

morrownr commented 2 years ago

EDIT: After additional testing both USB 3.0 ports are now working without errors. Apparently swapping the firmware worked.. ?

I have busy since we worked on thiss yesterday. Give me a chance to read your reports and then I will try to sit aside the next hour to look at this.

morrownr commented 2 years ago

Let me throw out some info so that we are on the same sheet of music:

The Linux kernel firmware repository for Mediatek:

https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/mediatek

It shows the following files as available:

  91412 May  5 12:54 mt7662u.bin
  20686 May  5 13:00 mt7662u_rom_patch.bin

It does not show any files called mt7662.bin or mt7662_rom_patch.bin at all. So, we have a mystery on our hands.

Let me do the digging in the driver source that I started to do yesterday.

morrownr commented 2 years ago

While I am working, here is something for you to test:

Delete mt7662.bin and mt7662_rom_patch.bin from /lib/firmware and then reboot so as to have a clean boot and then see what happens.

morrownr commented 2 years ago

A little grepping later and this is what I have:

mt76/mt76x0/mt76x0.h:#define MT7610E_FIRMWARE       "mediatek/mt7610e.bin"
mt76/mt76x0/mt76x0.h:#define MT7650E_FIRMWARE       "mediatek/mt7650e.bin"
mt76/mt76x0/mt76x0.h:#define MT7610U_FIRMWARE   "mediatek/mt7610u.bin"
mt76/mt76x2/mt76x2.h:#define MT7662_FIRMWARE        "mt7662.bin"
mt76/mt76x2/pci.c:MODULE_FIRMWARE(MT7662_FIRMWARE);
mt76/mt76x2/pci.c:MODULE_FIRMWARE(MT7662_ROM_PATCH);
mt76/mt76x2/pci_mcu.c:  ret = request_firmware(&fw, MT7662_FIRMWARE, dev->mt76.dev);
mt76/mt76x2/usb_mcu.c:  err = request_firmware(&fw, MT7662_FIRMWARE, dev->mt76.dev);
mt76/mt76x2/usb.c:MODULE_FIRMWARE(MT7662_FIRMWARE);
mt76/mt76x2/usb.c:MODULE_FIRMWARE(MT7662_ROM_PATCH);

It appears mt7662.bin and its mate are for the PCI version of the chipset and should be in /lib/firmware whick should answer the below question you had:

Why the discrepancy between the firmware in /lib/firmware compared to /lib/firmware/mediatek (and git)?

One is for the PCI chipset and one is for the USB chipset. It is interesting how they ended up to two different places.

I am not seeing anything in the code that tells me when mt7662u.bin and its mate should be. Interesting. We will have to assume it is /lib/firmware/mediatek as I have checked multiple distros now and that it where they are.

It is interesting that you copied the files from /mediatek to /lib/firmware and removed the u and things started working. I am being to suspect a faulty part in the adapter as what you did should not have solved the problem. Is it possible that there is some dust in the ports or in the adapter?

This may be a case where putting things down for a couple of days and then start over.

amisix commented 2 years ago

I removed mt7662.bin and mate from /lib/firmware and the issue reappears. The adapter fails to initialize.

Error:

[ 7.329300 ] mt76x2u 2-2:1.0: Direct firmware load for mt7662_rom_patch.bin failed with error -2 [ 7.329382] bcmgenet fd580000.ethernet: configuring instance for external RGMII (RX delay) [ 7.337947] bcmgenet fd580000.ethernet eth0: Link is Down [ 7.446507] systemd-journald[171]: Oldest entry in /var/log/journal/71f4eaa596ae498e8f46633c136a5627/system.journal is older than the configured file retention duration (1month), suggesting rotation. [ 7.464436] systemd-journald[171]: /var/log/journal/71f4eaa596ae498e8f46633c136a5627/system.journal: Journal header limits reached or header out-of-date, rotating. [ 8.040892] mt76x2u: probe of 2-2:1.0 failed with error -2 [ 8.048231] usbcore: registered new interface driver mt76x2u

The adapters and plugs are clean. I have a second MT7612U004 adapter I've tried it with, same behavior.

EDIT: I put mt7662.bin and mate back in /lib/firmware and the adapter starts working again.

EDIT EDIT: It fails when I connect a second adapter, whether it be an ANDDEAR or Alfa MT7612U. The first adapter that initialized fine stays functioning but the second adapter immediately starts throwing disconnect/reconnect errors repetitively every few seconds. Oh well, one works great for now.

So, there's a temporary fix - but, why? This can totally wait if you want to try again another day.

morrownr commented 2 years ago

I removed mt7662.bin and mate from /lib/firmware and the issue reappears. The adapter fails to initialize.

Error:

[ 7.329300 ] mt76x2u 2-2:1.0: Direct firmware load for mt7662_rom_patch.bin failed with error -2

This makes no sense to me. Why would a USB adapter load mt7662_rom_patch.bin?

[ 8.040892] mt76x2u: probe of 2-2:1.0 failed with error -2

?

[ 8.048231] usbcore: registered new interface driver mt76x2u

Okay

The adapters and plugs are clean. I have a second MT7612U004 adapter I've tried it with, same behavior.

EDIT: I put mt7662.bin and mate back in /lib/firmware and the adapter starts working again.

Is this mt7662.bin and mate the originals or the ones that are actually mt7662u.bin and mate but were renamed?

EDIT EDIT: It fails when I connect a second adapter, whether it be an ANDDEAR or Alfa MT7612U. The first adapter that initialized fine stays functioning but the second adapter immediately starts throwing disconnect/reconnect errors repetitively every few seconds. Oh well, one works great for now.

When I put two 7612u adapters in any system here, they both work fine but I don't have a ANDDEAR MT7612U004.

This is starting to hurt my head.

This testing needs to be done in a scientific manner. We really need a sheet where we write the exact details. I'm getting lost on which distro and which hardware and how many or which adapters you are testing.

Are you using a usb hub or direct into the port?

So, there's a temporary fix - but, why? This can totally wait if you want to try again another day.

I'll try to hang in there.

amisix commented 2 years ago

This makes no sense to me. Why would a USB adapter load mt7662_rom_patch.bin?

Unknown. Agreed, why?

Is this mt7662.bin and mate the originals or the ones that are actually mt7662u.bin and mate but were renamed?

*Copied from /lib/firmware/mediatek and renamed.

This is starting to hurt my head. This testing needs to be done in a scientific manner. We really need a sheet where we write the exact details. I'm getting lost on which distro and which hardware and how many or which adapters you are testing.

Same. I'm trying my best. I can go back through the thread and compile things a bit better if need be. I only used the ALFA ACM in place of the ANDDEAR as requested in one of your tests, and once as a secondary adapter (recently) on the USB 3.0 bus - past that it's all the ANDDEAR-MT7612U004 that's throwing a fit (I have two of them on hand).

No hub, direct to the port(s). Single adapter connected unless specifically testing dual adapters to see how it responded (1 test).

Thanks..

morrownr commented 2 years ago

How about we do a short review and come up with a plan? We can contact the Mediatek kernel devs if need be but right now all we have is kind of a mess as far as something to present.

We zeroed in on checking the firmware as a possible problem. Here is how we think the firmware should be setup for mt7612u/mt7612e chipsets:

Located in /lib/firmware/mediatek: 91412 mt7662u.bin 20686 mt7662u_rom_patch.bin

Located in /lib/firmware: 81908 mt7662.bin 26350 mt7662_rom_patch.bin

I have checked the above files in multiple distros and this is what is working. Take note that the files in /lib/firmware/mediatek are different than those in /lib/firmware and the theory is that the ones without u are for the pci version of the chipset. Our look inside the code of the mt76 driver appeared to confirm that.

Assumption: If test systems have the above 4 files exactly as they are shown above, the firmware files should be the right files and in the right place. I've seen distro makers mess this up so it was worth checking.

Mt suggestion now is to take that clean sd of RasPiOS 2022-04-04 and use it in the RasPi4B to do a full checkout and save logs. You had said that the adapter was working fine with this setup so let's work this setup to confirm that it is working fine so that we can save log files that show how things should be when it is working fine.

I think it worthwhile to refresh out memories on how to clean out log files between boots so we are not confusing things.

Can I get you to run the following tests while booting in between tests? Save from log on each boot using something like...

$ dmesg | grep -i mt76

Configurations to test: (cold boot between tests)

one ANDDEAR 004 adapter in usb3 port two ANDDEAR 004 adapters in usb3 ports one ANDDEAR 004 adapter in usb2 port two ANDDEAR 004 adapters in usb2 ports two ANDDEAR 004 adapters in one of each type of port one ANDDEAR 004 adapter in usb3 port and one ALFA ACM adapters in the other usb3 port if they both fit at the same time

Also, run the follow and save results while the ANDDEAR adapter is in a usb3 port:

$ lsusb -t

You should see something like this:

Port 2: Dev 2, If 0, Class=Vendor Specific Class, Driver=mt76x2u, 5000M

That is to make sure we are in usb3 mode. It is possible to use a usb3 capable chipset but make the adapter only usb2 capable.

Once we have a full set of good results in hand, we can then boot with Kali and run similar tests to see what the difference is. My recommendation is that the Kali bootable sd be a clean installation. The reason is that I have seen many things get messed up in Kali over time due to the rolling release. A clean, updated installation should help with any problems that could build up over time.

Regards

amisix commented 2 years ago

How about we do a short review and come up with a plan? We can contact the Mediatek kernel devs if need be but right now all we have is kind of a mess as far as something to present.

Sure, Ok.

We zeroed in on checking the firmware as a possible problem. Here is how we think the firmware should be setup for mt7612u/mt7612e chipsets:...

Yes, on the same page.

Assumption: If test systems have the above 4 files exactly as they are shown above, the firmware files should be the right files and in the right place.

Ok, will run through requested tests to confirm.

Can I get you to run the following tests while booting in between tests?

Of course. It will take a bit but I will get it. All previous tests have been completed with a clean, freshly imaged copy of whichever OS was requested, I will continue to do that for these tests.

Thanks much for all your efforts regarding this.

morrownr commented 2 years ago

Take your time. I enjoy solving a good mystery. It often takes time.

You may want to start a text document to put the results of your testing and you can zip it and post it here instead of having a very long report. It is also easier to add to a text document than to edit and keep up with results here. Below is an example like I might use:

RasPi4B 
RasPiOS 2022-04-04
one ANDDEAR 004 adapter in usb3 port
$ dmesg | grep -i mt76
[   11.289928] mt76x2u 2-2:1.0: ASIC revision: 76120044
[   11.332857] mt76x2u 2-2:1.0: ROM patch build: 20141115060606a
[   11.497666] mt76x2u 2-2:1.0: Firmware Version: 0.0.00
[   11.497688] mt76x2u 2-2:1.0: Build: 1
[   11.497694] mt76x2u 2-2:1.0: Build Time: 201507311614____
[   12.319504] usbcore: registered new interface driver mt76x2u
[   12.363881] mt76x2u 2-2:1.0 wlx0013ef5f0c7c: renamed from wlan0

The multiple tests with the RasPiOS hopefully will give us a baseline of tests that are successful, then the same tests with Kali on the RasPi4B can be compared.

Regards

morrownr commented 2 years ago

Can I get you to add one test to the previously posted list?

one ALFA ACM adapter in usb3 port

amisix commented 2 years ago

OK, 50th time is the charm. Github sure doesn't make it easy to simply upload a text file without trying to tie it to some code or a branch or some such sh*t. Here's a .zip file with the RasPiOS tests

morrownr commented 2 years ago

Thanks for giving it 50 times. I think I can show you an easier way to upload the compressed file. What OS are your trying to upload it with?

I read through the file. I saw consistent results. I took the primary error line and did some searching. Try this:

Open a terminal (Ctrl + Alt + t)

sudo nano /etc/modprobe.d/mt76_usb.conf

add:

options mt76_usb disable_usb_sg=1

Save the file: Ctrl + Alt + o, Enter, Ctrl + Alt + x

Reboot

Redo one of the tests that failed.

Results?

amisix commented 2 years ago

Thanks for giving it 50 times. I think I can show you an easier way to upload the compressed file. What OS are your trying to upload it with?

Please do. Windows 10 (chrome) - I thought it should have been easier, don't know what I was missing. Can run basic tests in CLI, can't upload .txt file. Hopeless.

I read through the file. I saw consistent results. I took the primary error line and did some searching. Try this (disable SG):

Consistent results, good.

I disabled USB Scatter Gather, rebooted, then confirmed it was off with the command below. I tested with a single ANDDEAR adapter on USB3 port. Unfortunately the issue persists with no change in error messages.

cat /sys/module/mt76_usb/parameters/disable_usb_sg

Thanks for searching the error further. I also completed some searching for similar error messages but almost all referred to the error "mt76x02u_mcu_wait_resp failed with -110", not the "mt76x02u_mcu_wait_resp failed with -108" I've been receiving. Although I don't know how relevant that difference is at this time.

Thanks.

amisix commented 2 years ago

Quick test (re-run a dozen times): Kali 32bit 2022.1 Single ANDDEAR adapter, USB3 Port 1 (top) - FAIL (same errors as in RasPiOS) Single ANDDEAR adapter, USB3 Port 2 (bottom) - SUCCESS

Why would the adapter work on one USB3 port and not the other?

I also tested USB3 Port 1 with an Alfa ACM and the Alfa adapter works fine on that port, no errors. Then I ran the test on a different Raspberry Pi 4 and USB3 Port 1 with the ANDDEAR adapter failed just like previous tests.

I also found another error - something related to the xhci controller itself?

xhci_hcd 0000:01:00.0: WARN Set TR Deq Ptr cmd failed due to incorrect slot or ep state. mt76x2u 2-2:1.0: mac specific condition occurred

USB bug?

TurfJakkals commented 2 years ago

Greetings! I've read through the thread of progress made regarding this issue. Now the thing is, I'm using the MT7612UN and having no luck whatsoever, either. Not sure what makes the "UN" different from the "U" or "BU" versions?

morrownr commented 2 years ago

Please do. Windows 10 (chrome) - I thought it should have been easier, don't know what I was missing. Can run basic tests in CLI, can't upload .txt file. Hopeless.

Zip the text file and the drag the zip and drop it here (actually in the window where you are replying)

I disabled USB Scatter Gather, rebooted, then confirmed it was off with the command below. I tested with a single ANDDEAR adapter on USB3 port. Unfortunately the issue persists with no change in error messages.

Well, okay. While searching I saw similar reports. I had previously only seen issues requiring that parameter in AP mode.

morrownr commented 2 years ago

Quick test (re-run a dozen times): Kali 32bit 2022.1 Single ANDDEAR adapter, USB3 Port 1 (top) - FAIL (same errors as in RasPiOS) Single ANDDEAR adapter, USB3 Port 2 (bottom) - SUCCESS

Why would the adapter work on one USB3 port and not the other?

Very good question. I don't have an answer yet.

Which revision of the RasPi4B do you have?

At this point, I'd like to see the results of testing on x86/amd64 hardware. Yes, again. The expanded version of testing like you have been doing and documenting. Please continue documenting your testing as it is the best we have right now to see something to help and if we figure out what the problem is, it will be something to send in with a report.

I have 4 different adapters based on the mt7612u chipset and I have never run into such as this. I have a RasPi4B. It is a v1.2.

I also tested USB3 Port 1 with an Alfa ACM and the Alfa adapter works fine on that port, no errors. Then I ran the test on a different Raspberry Pi 4 and USB3 Port 1 with the ANDDEAR adapter failed just like previous tests.

Are both Pi's the same version?

I also found another error - something related to the xhci controller itself?

xhci_hcd 0000:01:00.0: WARN Set TR Deq Ptr cmd failed due to incorrect slot or ep state. mt76x2u 2-2:1.0: mac specific condition occurred

USB bug?

Need to do some searching.

morrownr commented 2 years ago

Greetings! I've read through the thread of progress made regarding this issue. Now the thing is, I'm using the MT7612UN and having no luck whatsoever, either. Not sure what makes the "UN" different from the "U" or "BU" versions?

Hi @TurfJakkals

I don't have an answer to your question at this time.

Can you be specific exactly which adapter that you have? And the hardware that you are plugging the adapter into?

Regards

morrownr commented 2 years ago

Hi @amisix

I have moved the listing and review of the ANDDEAR 04 until such time as we resolve this issue. If I am not comfortable with a listing, I can't keep it up. Have you seen any issues with the ANDDEAR 03?

amisix commented 2 years ago

Well, okay. While searching I saw similar reports. I had previously only seen issues requiring that parameter in AP mode.

It's a variable we needed to account for, now it's done.

Which revision of the RasPi4B do you have?

Both boards are revision 1.5 (2GB version).

At this point, I'd like to see the results of testing on x86/amd64 hardware. Yes, again.

I can do that. I'll be able to complete it within the next couple of days. I'll include completed ARM Kali tests as well.

I have 4 different adapters based on the mt7612u chipset and I have never run into such as this.

Odd, I know, but if the issue was going to affect anybody.. it'd be me. My opinion of Mediatek is not impacted by this issue. Apparently this is currently impacting other users as well.. I was stoked to learn it's quite possibly an actual issue as opposed to raw incompetence on my part. I almost said F* it, how many people will have this hardware combination (very few)? Who needs to do all this effort troubleshooting an issue with an off label USB wifi adapter? I mean, there is a "fix" - don't put the adapter in that port. But it's a lazy solution that just passes the buck to the next poor SOB that happens to stumble upon this or something similar. And I want an answer.

I have moved the listing and review of the ANDDEAR 04 until such time as we resolve this issue. If I am not comfortable with a listing, I can't keep it up. Have you seen any issues with the ANDDEAR 03?

Thank you - I feel the same way. I legit had anxiety that someone was going to purchase one of these based on my review only to find out it is the complete opposite of "it just works" (for now). I haven't had any issues with the ANDDEAR 03. I've swapped them in/out of USB2 & USB3 ports without errors many times. They've been working great so far although I may test them more rigorously due to this experience.

amisix commented 2 years ago

@TurfJakkals From what I've read the U stands for USB (we knew that) and the N stands for Nano. I could be wrong. Is it a nano adapter? Thanks for letting us know this impacts others, hopefully that will help us to determine a solution. The Mediatek chipset in the adapter is great.. Are you also on a Raspberry Pi or another platform? What distro are you using?

TurfJakkals commented 2 years ago

Can you be specific exactly which adapter that you have? And the hardware that you are plugging the adapter into?

I got it working on my Win11 system by assigning it a driver manually. Driver: MediaTek, Inc. 802.11ac Wireless LAN Card Version:5.1.22.0 (updated to 5.1.25.0 after installation) Driver node strong name: oem15.---:efa2156a6b29dd80:OS61_RTWLANR_7650.ndi:5.1.25.0:usb\vid_0e8d&pid_7650 Plugged into USB3.0. Works perfectly on my Win11 system after assigning it that driver.

What distro are you using?

Tried to get it working on 3 different distros over on my other system. Latest updates on all of them. Plugged into USB3.0, oh and no, it isn't a Nano adapter.

Kali: Doesn't pick up, not showing when lsusb. Parrot: Shows when lsusb, nothing further. Zorin: Shows when lsusb, nothing further.

morrownr commented 2 years ago

Hi @TurfJakkals

I got it working on my Win11 system by assigning it a driver manually.

You are talking about an ANDDEAR MT7612U004 adapter?

Driver: MediaTek, Inc. 802.11ac Wireless LAN Card Version:5.1.22.0 (updated to 5.1.25.0 after installation) Driver node strong name: oem15.---:efa2156a6b29dd80:OS61_RTWLANR_7650.ndi:5.1.25.0:usb\vid_0e8d&pid_7650 Plugged into USB3.0. Works perfectly on my Win11 system after assigning it that driver.

Interesting. I have a laptop that has Win11 on it these days. I don't use the Win11 but it is there. I booted it and plugged my ALFA ACM into a USB3 port. The ALFA ACM (mt7612u chipset) was detected and driver v5.1.22 was autoinstalled. I turned off the internal wifi and logged into my AP. It worked fine. It is not clear to me why it would have been necessary to install a driver.

What distro are you using?

Tried to get it working on 3 different distros over on my other system. Latest updates on all of them. Plugged into USB3.0, oh and no, it isn't a Nano adapter.

I am not aware of any nano adapters ever having been built with a mt7612u chipset. It may not be suitable for use in a nano adaoter.

Kali: Doesn't pick up, not showing when lsusb. Parrot: Shows when lsusb, nothing further. Zorin: Shows when lsusb, nothing further.

Request that both of you post the output of $ lsusb . Thanks.

@amisix @TurfJakkals

A recap of where my thinking is at this point: amisix has done a good job of testing and creating a document with results that can help us identify or at least narrow down where the problem is.

So far, the results on a RasPi4B with the RasPiOS and Kali for RasPi4B show the ANDDEAR MT7612U004 Adapter autodetected and working fine if plugged into a USB2 port and not working at all if plugged into a USB3 port. A comparison with the ALFA ACM shows the ALFA ACM autodeteched and working fine on both USB2 and USB3 ports.. I see what appears to be some testing earlier in this thread that appears to be with Linux distros on x86/amd64 bit is not clear as it i is not a clear as it could be.

Both boards are revision 1.5 (2GB version).

Details like this need to documented in the master document that amisix is creating IMHO.

Let add something to the conversation that you may not be aware of:

In addition to the USB-WiFi repo, I also have 5 repos specifically for modern Realtek based WiFi adapters. One of them is:

https://github.com/morrownr/88x2bu-20210702

As your read the README.md for the 88x2bu driver, you will see:

[1] Overall this driver does a good job with AP mode. During testing and work prior to making this driver available, the team working on this driver noticed some problems in AP mode if used with a Raspberry Pi 4B. We were unable to discover or fix the exact cause of the problem but the workaround is to keep the driver in USB2 mode. This workaround only applies to AP mode with Raspberry Pi 4B. No problems were noted with systems that use x86 or amd64 processors.

The USB3 problem specifically with the RasPi4B is so severe that we had to publish a specific workaround for users. Interestingly enough, the following driver does not suffer the same problem, or at least not to the same degree:

https://github.com/morrownr/8812au-20210629

Both are Realtek AC1200 chipsets for USB WiFi adapters and are widely available. Those repos are VERY heavily used. What is the difference between the 2 Realtek chipsets? The two drivers are very different and the large coded firmware files make it a challenge to understand many details but testing has shown the the 88x2bu is a faster chipset than the 8812au. However, the 8812au is an overall better chipset based over compatibility with AP mode and monitor mode.

My point being that USB3 problems are certainly not unique to the subject Mediatek based adapter. The problem on the 88x2bu based adapter shows in 5 MHz AP mode and the symptoms range from reduced performance to total connection shutdown and is 100% reproducible. The problem is ONLY found with the Raspi4B, well, probably with the RasPi 400 as well but no testing has been completed due to lack of hardware.

In my mind, we are at the point where it we get solid evidence that the ANDDEAR MT7612U004 Adapter does not work in a USB3 x86/amd64 port, then we likely are looking at a ANDDEAR MT7612U004 Adapter bug. We need more testing to confirm or disprove that theory. I can't help due to lack of having that specific adapter but I will stay engaged to help get the problem.

Regards

amisix commented 2 years ago

ANDDEAR-MT7612U004 x86 laptop / Asus USB Chipset: Intel 100/C230 Series

Ubuntu 22.04 LTS Linux ubuntu 5.15.0-25-generic x86_64 x86_64 x86_64 GNU/Linux

Full test results: ANDDEAR-MT7612U004 UBUNTU 22.04 LTS.txt

Test A, USB3 Port 1: FAILED Test B, USB3 Port 1: SUCCESS, adapter functions (disconnect/reconnect without restart)

Test C, USB3 Port 2: SUCCESS, adapter functions Test D, USB3 Port 2: SUCCESS, adapter functions (disconnect/reconnect without restart)

Test E, USB2 Port 1: SUCCESS, adapter functions. Test F, USB2 Port 2: SUCCESS, adapter functions.

Interestingly one USB3 port worked just fine with the ANDDEAR adapter while the other did not. Just like the Raspberry Pi 4B. Disconnecting/reconnecting the adapter after Test A failed made the adapter begin to function. Disconnecting/reconnecting a functioning adapter didn't cause any issues. USB3 port 1 was problematic, other ports (including USB2) were not.

morrownr commented 2 years ago

This is starting to get interesting.

Question: When you say disconnect/reconnect without restart, do you mean that you pulling the adapter out of the port and then put it back in? Can you clarify what you are doing?

Another question: Are you checking with lsusb -t to see if the individual tests are resulting in USB3 (5000) or USB2 (480)?

It might be a good idea to add USB3 chipset to the header on each test. The info can be found with sudo lshw. Here is the chipset from my main dev system:

7 Series/C216 Chipset Family USB

The USB3 chipset on the RasPi4B is:

VL805

I have not reviewed the document with the full results yet but will as I have time today to see if I can pick up on anything else. This last series of tests is very interesting. I think I am going to setup to run detailed testing with the four 7612u based adapters that I have. I have a small lab with 4 systems that I use for testing and said systems span a wide range of capabilities , ages and have various distros installed. Maybe I can find information that can be of use.

Is this an issue with the adapter? Is this an issue with the 7612u driver? Is this an issue with the USB stack in Linux? Is this issue specific to certain chipsets? Many questions. The problem with difficult issues like this is knowing where to report the problem. We do not have the data yet to know where to go with a report.

Something you may want to do is contact the folks behind the below article and ask them what they are seeing:

https://wlan-pi.github.io/wlanpi-documentation/admin/cf912_issues/

The contact info is shown at the bottom of the article.

Regards

amisix commented 2 years ago

When you say disconnect/reconnect without restart, do you mean that you pulling the adapter out of the port and then put it back in?

Pulling the adapter out of the port and plugging it back in while the machine is on.

Are you checking with lsusb -t to see if the individual tests are resulting in USB3 (5000) or USB2 (480)?

Yes. USB3 connectivity displayed 5000Mbps and USB2 displayed 480Mbps as expected.

It might be a good idea to add USB3 chipset to the header

I'll do that. The x86 laptop has an Intel 100 Series/C230 Series Chipset. I also have an older machine with Intel 7/C210 and Intel 7/C216 Series chipsets (as you do). It also has an ASMedia ASM1042 USB3 chipset... hmmmm.

I think I am going to setup to run detailed testing with the four 7612u based adapters that I have. I have a small lab with 4 systems that I use for testing and said systems span a wide range of capabilities

Well, now it's a party. I bet your lab is fun (more details?), I'm very much looking forward to the results.

Is this an issue with the adapter? Is this an issue with the 7612u driver? Is this an issue with the USB stack in Linux? Is this issue specific to certain chipsets?

Well, we're ruling out pebkac so I'm happy.

Something you may want to do is contact the folks behind the below article and ask them what they are seeing:

Oh, that's neat. I'll reach out to them. Thanks.

amisix commented 2 years ago

Quick testing with Intel 7/210 Series and ASMedia 1042 USB3 chipsets.

ANDDEAR-MT7612U004 x86 desktop / Asus P8Z77 v-pro Thunderbolt motherboard USB Controller(s): Intel 7/210 Series & ASMedia ASM1042

Ubuntu 22.04

Test A: FAILED ASMedia 1042, USB3 Port 1

Test B: SUCCESS ASMedia 1042, USB3 Port 2

Test C: FAILED Intel 7/C210 Series, USB3 Port 1

Test D: SUCCESS Intel 7/C210 Series, USB3 Port 2

On all 4 USB 3.0 chipsets we've tested the adapter fails on USB3 Port 1 but it works on USB3 Port 2 (Kali & Ubuntu)

EDIT: In the aircrack-ng FAQ I found something that may be useful? At the bottom of the FAQ it describes a bug and I received a nearly identical error in Kali on the Raspberry Pi - "xhci_hcd 0000:01:00.0: WARN Set TR Deq Ptr cmd failed due to incorrect slot or ep state.". Something about it being a USB subsystem bug that affects USB 3 and kernels 4.2 and above?

amisix commented 2 years ago

TL;DR

Adapter: ANDDEAR-MT7612U004 Adapter Chipset: Mediatek mt7612u Platform: Raspberry Pi 4B (Revision 1.5)

Issue: ANDDEAR-MT7612U004 wifi adapter doesn't initialize on hot or cold boot when plugged into a USB3 port. Workaround: Try USB3 Port 2 (counting from top to bottom) or use a USB2 port.

Findings: Other adapters based on the Mediatek mt7612u chipset do not have this issue (Alfa ACM, Netgear A6210, etc.). 32bit & 64bit distros based on Debian & Arch are affected. The adapter doesn't function in any USB3 or USB2 ports in Ubuntu 22.04*.

USB3 Port 1: The adapter doesn't function in USB3 Port 1 in any of the Linux distros tested (Kali, RasPiOS, Ubuntu, Manjaro) USB3 Port 2: The adapter functions intermittently in Manjaro, Kali, & RasPiOS. Remove/reinsert the adapter then delete & recreate the network connection.

USB2 Port 1: The adapter functions in Kali, RasPiOS, and Manjaro. USB2 Port 2: The adapter functions in Kali, RasPiOS, and Manjaro.

*Neither the ANDDEAR-MT7612U004 or an Alfa AWUS036ACM worked with the Ubuntu 22.04 system image for Raspberry Pi The ANDDEAR-MT7612U004 adapter functions in all USB3 & USB2 ports in Windows 10 (x86).

Linux distros tested:
RasPiOS 2022-04-04 (ARM 32bit, kernel 5.15.32-v7l+) | Partial Success Ubuntu 22.04 (ARM 64bit, kernel 5.15.0-1005-raspi) | Failed Kali 22.01 (ARM 32bit, kernel 5.4.83-Re4son-v7l+) | Partial Success Kali 22.01 (ARM 64bit, kernel 5.4.83-Re4son-v81) | Partial Success Kali 22.02 (ARM 32bit, kernel 5.10.103-Re4son-v7l+) | Partial Success Manjaro 22.03 (ARM 64bit, 5.15.32-2) | Partial Success

Additional testing info: USB3 Scatter Gather was disabled unless otherwise noted. A fresh system image was used for all tests. The adapter was directly connected to the USB ports with no extension cable. An Alfa AWUS036ACM adapter was tested on all USB3 & USB2 ports and it functioned without issue (except for Ubuntu 22.04) Some tests were run again with a different ANDDEAR-MT7612U004 adapter and the results were the same. Some tests were run again with a different Raspberry Pi 4B (Revision 1.5) and the results were the same.

USB chipsets tested: VL805 (Raspberry Pi 4B) Intel 100/C230 Series (x86 Windows 10)

Additional Adapter Info: ID 0e8d:7612 MediaTek Inc. MT7612U 802.11a/b/g/n/ac Wireless Adapter mt76x2u 1-1.3:1.0: ASIC revision: 76120044 mt76x2u 1-1.3:1.0: ROM patch build: 20141115060606a mt76x2u 1-1.3:1.0: Firmware Version: 0.0.00 mt76x2u 1-1.3:1.0: Build: 1 mt76x2u 1-1.3:1.0: Build Time: 201507311614 mt76x2u 1-1.4:1.0: ASIC revision: 76120044 mt76x2u 1-1.4:1.0: ROM patch build: 20141115060606a mt76x2u 1-1.4:1.0: Firmware Version: 0.0.00 mt76x2u 1-1.4:1.0: Build: 1 mt76x2u 1-1.4:1.0: Build Time: 201507311614 usbcore: registered new interface driver mt76x2u

Errors encountered:

mt76x2u 2-3:1.0: MAC RX failed to stop xhci_hcd 0000:00:14.0: WARN Set TR Deq Ptr cmd failed due to incorrect slot or ep state. mt76x2u: probe of 2-3:1.0 failed with error -5

usb 2-1: USB disconnect, device number 2 mt76x2u 2-1:1.0: error: mt76x02u_mcu_wait_resp failed with -108 mt76x2u 2-1:1.0: mac specific condition occurred usb 2-1: Device not responding to setup address.

error: mt76x02u_mcu_wait_resp failed with -19 mt76x2u 2-1:1.0: mac specific condition occurred

usb 2-1: Set SEL for device-initiated U1 failed. usb 2-1: Set SEL for device-initiated U2 failed. usb 2-1: reset SuperSpeed Gen 1 USB device number 3 using xhci_hcd mt76x2u 2-1:1.0: ASIC revision: 76120044 mt76x2u 2-1:1.0: vendor request req:06 off:0080 failed:-110 mt76x2u 2-1:1.0: vendor request req:07 off:0080 failed:-110

[ 15.069700] mt76x2u 2-3:1.0: ASIC revision: 76120044 [ 15.069796] mt76x2u 2-3:1.0: firmware: direct-loading firmware mt7662_rom_patch.bin [ 15.069807] mt76x2u 2-3:1.0: ROM patch build: 20141115060606a [ 15.126913] Bluetooth: hci1: RTL: fw version 0x0999646b [ 15.201949] mt76x2u 2-3:1.0: firmware: direct-loading firmware mt7662.bin [ 15.201958] mt76x2u 2-3:1.0: Firmware Version: 0.0.00 [ 15.201962] mt76x2u 2-3:1.0: Build: 1 [ 15.201964] mt76x2u 2-3:1.0: Build Time: 201507311614____ [ 15.928821] ieee80211 phy1: Selected rate control algorithm 'minstrel_ht' [ 15.930020] usbcore: registered new interface driver mt76x2u [ 18.503647] r8169 0000:03:00.0: firmware: direct-loading firmware rtl_nic/rtl8168h-2.fw [ 18.533693] Generic FE-GE Realtek PHY r8169-0-300:00: attached PHY driver (mii_bus:phy_addr=r8169-0-300:00, irq=MAC) [ 18.725739] r8169 0000:03:00.0 eth0: Link is Down [ 18.741103] iwlwifi 0000:02:00.0: Applying debug destination EXTERNAL_DRAM [ 18.818936] iwlwifi 0000:02:00.0: Applying debug destination EXTERNAL_DRAM [ 18.820194] iwlwifi 0000:02:00.0: FW already configured (0) - re-configuring [ 18.855815] iwlwifi 0000:02:00.0: Applying debug destination EXTERNAL_DRAM [ 18.933961] iwlwifi 0000:02:00.0: Applying debug destination EXTERNAL_DRAM [ 18.935387] iwlwifi 0000:02:00.0: FW already configured (0) - re-configuring [ 20.885724] mt76x2u 2-3:1.0: mac specific condition occurred [ 21.005747] usb 2-3: USB disconnect, device number 3 [ 21.140248] mt76x2u 2-3:1.0: mac specific condition occurred [ 21.349695] mt76x2u 2-3:1.0: mac specific condition occurred [ 21.453699] mt76x2u 2-3:1.0: mac specific condition occurred [ 21.553693] mt76x2u 2-3:1.0: timed out waiting for pending tx [ 21.885886] usb 2-3: new SuperSpeed USB device number 4 using xhci_hcd [ 21.906677] usb 2-3: New USB device found, idVendor=0e8d, idProduct=7612, bcdDevice= 1.00 [ 21.906681] usb 2-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [ 21.906682] usb 2-3: Product: 802.11ac WLAN [ 21.906683] usb 2-3: Manufacturer: MediaTek Inc. [ 21.906684] usb 2-3: SerialNumber: 000000000 [ 22.038055] usb 2-3: reset SuperSpeed USB device number 4 using xhci_hcd

mt76x2u 2-1:1.0: mac specific condition occurred [ 268.341160] mt76x2u 2-1:1.0: vendor request req:07 off:1004 failed:-71 [ 269.135832] xhci_hcd 0000:01:00.0: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 0 comp_code 4 [ 269.135860] xhci_hcd 0000:01:00.0: Looking for event-dma 000000041ad45520 trb-start 000000041ad45540 trb-end 000000041ad45590 seg-start 000000041ad45000 seg-end 000000041ad45ff0 [ 269.338837] usb 2-1: USB disconnect, device number 22 [ 269.388858] wlan1: deauthenticating from 00:c0:ca:b0:20:7d by local choice (Reason: 3=DEAUTH_LEAVING) [ 269.559587] mt76x2u 2-1:1.0: mac specific condition occurred [ 269.779417] mt76x2u 2-1:1.0: mac specific condition occurred [ 269.888844] mt76x2u 2-1:1.0: mac specific condition occurred [ 269.988872] mt76x2u 2-1:1.0: timed out waiting for pending tx [ 270.439110] usb 2-1: new SuperSpeed USB device number 23 using xhci_hcd [ 270.470157] usb 2-1: device descriptor read/all, error -71 [ 270.699097] usb 2-1: new SuperSpeed USB device number 24 using xhci_hcd [ 270.730189] usb 2-1: device descriptor read/all, error -71 [ 270.738569] usb usb2-port1: attempt power cycle [ 271.519263] usb 2-1: new SuperSpeed USB device number 25 using xhci_hcd [ 271.550620] usb 2-1: device descriptor read/all, error -71 [ 271.789261] usb 2-1: new SuperSpeed USB device number 26 using xhci_hcd [ 271.820527] usb 2-1: device descriptor read/all, error -71 [ 271.829283] usb usb2-port1: unable to enumerate USB device

Logs (additional logs will be posted once complete): ANDDEAR-MT7612U004 Testing - RasPiOS 2022-04-04 ARM 32Bit.txt ANDDEAR-MT7612U004 Testing - Kali 22.02 ARM 32bit.txt ANDDEAR-MT7612U004 Testing - Manjaro 22.05 ARM 64bit.txt

morrownr commented 2 years ago

Good report. It could use a little editing before we get to the point that we submit it. After a few years of taking bug reports from users of the Realtek drivers I have up, I have a good feel for what should be included and what not. I can do a little editing as an example if you do not mind?

Status from my perspective:

Problem: USB WiFi adapter will not successfully come up on hot or cold boot if plugged into USB3, port 1.

Operates normally if plugged into USB3 ports 2+ or if plugged into any USB2 port.

Hardware: ANDDEAR MT7612U004 (URL: ? )

Adapter chipset: MT7612u

USB3 chipsets tested:

VL805 (RasPi4B) Intel 7/210 Series Intel 100/230 Series ASMedia 1042 (powered USB3 external hub)

Linux distros tested:

Ubuntu 22.04 (kernel 5.15) (x86_64) Kali 22.1 (kernel ?) (32 bit ARM) RasPiOS 2022-04-04 (kernel 5.15) (32 bit ARM)

Log: to be added.


Thoughts:

I took a look at the bug you pointed me to regarding xhci. Wow. That thread is longer than this one. I need time to digest.

In looking at the distros and hardware you have tested with, two things in common stick out: All distros are Debian based and all use systemd. I'm pondering if testing with Manjaro (Arch based) or Fedora would show anything. I'm also wondering if finding a distro that does not use systemd would be of any help.

I started setting up yesterday to test my adapters to see what I see in the logs. I have four USB3, mt7612u based adapters and various test systems so I will take a look as I have time. Hopefully today.

What do I think the problem is at this point?

We are closer than we were. Why would this only show up on USB3 port 1? Maybe something during the boot process is happening in the wrong order initially? Maybe it is a timing issue and something is not ready for port 1? The xhci usb3 driver could be the guilty party. Time for more research and reading. I'll report back as able.

Regards

morrownr commented 2 years ago

Oh, nearly forgot this minor detail: When you talk about the ports in the RasPi4b, you say top and bottom ports. Well, I run my Pi's up side down. My opinion is that many things work better that way. We can discuss at some point if you want but my point is just say port 1 and port2 because my top and bottom are different than yours. Cheers.

amisix commented 2 years ago

I will edit as you've suggested and change the verbiage regarding port numbers. I need a short break tho, too. much. data.

I took a look at the bug you pointed me to regarding xhci. Wow. That thread is longer than this one. I need time to digest.

Yeah, I dunno - it's not an exact fit but it's close. I found a couple others but need to read up more on them.

In looking at the distros and hardware you have tested with, two things in common stick out: All distros are Debian based and all use systemd. I'm pondering if testing with Manjaro (Arch based) or Fedora would show anything.

I'll test with fedora.

I started setting up yesterday to test my adapters to see what I see in the logs. I have four USB3, mt7612u based adapters and various test systems so I will take a look as I have time.

Thanks, I know it can be tedious, I appreciate the effort. I don't think you're going to have a single issue. Can you get your hands on one of these ANDDEAR adapters? I'll buy you a few pints of beer to cover things.

I'm glad we're closer, maybe? So much data to parse that its become convoluted and difficult to follow, making me feel we're farther away. The damn thing keeps throwing different, obscure errors with behavior differing from port to port, what a mess.

Thanks.

amisix commented 2 years ago

Fedora does not work either (Workstation 36, x86 64bit). Same exact errors as on the other distros. Also, while troubleshooting one time (in Ubuntu, x86) the adapter was somehow set to USB2 protocol while plugged into a USB3 port. And it worked fine as if it was in a USB2 port.

I don't think I can recommend this adapter. lol

morrownr commented 2 years ago

Can you get your hands on one of these ANDDEAR adapters?

I have investigated this. I can't get one currently without international shipping being involved. I have tried international shipped twice in the last two years and it did not turn out well either time so I would like to avoid that which leaves me without a good option.

Fedora does not work either (Workstation 36, x86 64bit). Same exact errors as on the other distros.

Understand.

I need a short break tho, too. much. data.

I very much understand this. I need a break also. When troubleshooting, it can be good to take a few days off and then come back to the master document to start fresh. I've had to do this hundreds of times over the last years as I try to support those that use the Realtek drivers. I've grown to very much dislike the company Realtek in that their drivers are missing many features you see in the Mediatek drivers and because there is no way to work issues with Realtek. I am very much looking forward to the new generation of Mediatek USB WiFi adapters based on the mt7921u driver.

I don't think I can recommend this adapter. lol

I've already pulled the listing and review. Before writing this replay, I did a quick review of this thread and while we are making progress that could eventually lead to code modifications that could help with this issue and others that are similar, it is beginning to look like there is something specific to this adapter that is problematic.

https://wlan-pi.github.io/wlanpi-documentation/admin/cf912_issues/

Have you had time to contact the folks from the above article? There are several folks there that have this adapter and can test to see if they have any luck with USB3 port 1.

I know it can be tedious,

I started testing my adapters with mt7612u chipsets yesterday. I started testing on a very modern laptop that has three USB 3.1 (3.0) ports and one USB 3.2 port. The system is running Ubuntu 22.04 (not my favorite distro but it will do until the next release of Mint is available). Anyway, I test my ALFA ACM, Netgear A6210 and TEROW ROW02CD. I also have a COMFAST CF-WU785AC and I need to include it as well. I tested each on all 4 ports. I have a little converter for the USB 3,2 USB-C port. I decided to start with WPA3, DFS channel on my router and anything else I could think of to make it hard. Result: No operational problems at all. I tested and pushed them with iperf3 and all I could see was smooth sailing. I had never tested USB-C 3.2 before so a good test was a good thing to see. I did capture mt76 and usb log entries to files and I will look at them today to see what came up.

At this point, I am beginning to suspect a manufacturing abnormality in the ANDDEAR adapter could be the cause. Have you tested with Windows or OSX?

I think we probably have enough data at this point and maybe we should slow down some and look at doing more research.

Regards

amisix commented 2 years ago

I have tried international shipping twice in the last two years and it did not turn out well either time so I would like to avoid that...

Yeah, that's no good. I have an "extra" one that I can mail to you if you'd be amenable to that.

I very much understand this. I need a break also. When troubleshooting, it can be good to take a few days off and then come back...

Agreed. Start seeing double and dyslexia kicks in, not too good for things like mt7612u & mt7621u etc.

I've grown to very much dislike the company Realtek in that their drivers are missing many features you see in the Mediatek drivers and because there is no way to work issues with Realtek.

Realtek sounds like monsters, flooding the market with sh*t chipsets & code. But... consumers would never know because: Windows. Seems like their primary market is naive users who never look under the hood because they wouldn't know how to fix it if they tried. Then a bunch of linux aficionados come along and are like... hey... that's a nice chipset you have there, it would be a shame if we made it do things you didn't design it to do... and it just spirals from there.

I too am looking very much forward to the upcoming release of the mt7621u chipset despite not having an AX capable rout... oh, yeah, "If you build it, wifi will come".

I've already pulled the listing and review. (of the ANDDEAR adapter)

Thank you. This thread is more like an anti-review, unfortunately.

Before writing this reply, I did a quick review of this thread and while we are making progress that could eventually lead to code modifications that could help with this issue and others that are similar, it is beginning to look like there is something specific to this adapter that is problematic.

Awesome, you work fast. Having some eyes on it that know what they're doing is very much appreciated. Looking forward to hopefully determining a resolution that doesn't involve percussive maintenance. and a shot glass.

Have you had time to contact the folks from the above article?

Not yet as their contact link goes to an error 404 page. But I found their repository and am still trying to determine the best way to reach out (It's not really geared towards Q&A, more like feedback). Their input may be invaluable though imo. Interestingly... they have a system image that runs on Raspberry Pi 4 that supposedly uses this adapter - is a functioning driver baked into their image or...? We shall see when I download and try it out.

Result: No operational problems at all. I tested and pushed them with iperf3 and all I could see was smooth sailing...

Yeah, there we go, a bunch of testing to show the ANDDEAR adapter does not function in the same environment as other mt7612u adapters. I mean.. it's a USB adapter based on an exceptional chipset - is there a hardware difference compared to other mt7612u adapters? Is there some kind of memory addressing issue or is it a power management issue that resets the adapter? Or is it a multi-layered issue that's compounding the problem? Just postulating a bit - I'll wait for more info from you while I do a bit of searching.

Have you tested with Windows or OSX

Yup, Windows 10, It works - I've actually been using one of them as my 2nd wifi adapter part of the time.

Thanks.

morrownr commented 2 years ago

I too am looking very much forward to the upcoming release of the mt7621u chipset despite not having an AX capable rout... oh, yeah, "If you build it, wifi will come".

Ah, you don't need an AX capable router. You just need 2 mt7921u wifi adapters. My router is an AC2600 device with a USB3 port and a USB2 port. I have tested my ALFA ACM in the USB3 port as a second 5 GHz radio and it works very well. I use OpenWRT. You can run OpenWRT on many routers AND on a RasPi4B. I can show you how with one of your Pi's if interested. You could learn with the ALFA ACM and then you are ready to go when we actually have adapters that we can buy.

Yup, Windows 10, It works...

Well, it seems this problem is specific to Linux but not specific to Debian which makes it look like something between power on and interface creation is fouled up. Since we are looking at many other mt7612u adapters that are not having the problem, it very well could be something in this specific adapter. Adapter makers buy the chipset from the chipset maker and then add antennas, amps and the various other things that make up an adapter. This really does not seem to be a chipset problem but rather something ANDDEAR has added may have messed up the timing.

Regarding the WLAN Pi...

I think the default hardware they use only has one USB2 port so they may not be aware of the USB3 problem. But, of course, they should have adapters that they can test in USB3 porrts so if you can get in touch with of them and ask them to test, that could be very useful and it is something they probably need to know and warn people about.

I have an "extra" one that I can mail to you if you'd be amenable to that.

I'm fine with that. Now if we had a good way to exchange private email addresses so that we aren't broadcasting them to the world. I'm sure there is a way, I just haven't run into this situation before.

Regards

amisix commented 2 years ago

Ah, you don't need an AX capable router. You just need 2 mt7921u wifi adapters...

Exactly where I was going with that. Just an excuse to build another WiPi hotspot! I built one that's been running nicely for a couple months now as my only router. It's a RPi 3B with OpenWrt and an Alfa ACM along with two Realtek 8153 USB ethernet adapters. Learning OpenWrt while at the same time figuring out the Pi atmosphere has been challenging and fun.

I can show you how with one of your Pi's if interested. You could learn with the ALFA ACM and then you are ready to go when we actually have adapters that we can buy.

lol, similar thinking, I like it. If it's not wasting your time I'll take you up on that offer. I'd like to do antenna and system optimization (DFS channels? why irqbalance? who cares about jitter?) with what I have now but it's my edge router so let's build something with Openwrt that I can break.

I think the default hardware they use only has one USB2 port so they may not be aware of the USB3 problem. But, of course, they should have adapters that they can test in USB3 ports

Yeah, USB 2.0 on the CM4 version.. I downloaded the wlanpi system image for the Raspberry Pi 4B and the ANDDEAR adapter initialized fine with no errors (dmesg is clear, ifconfig shows adapter). So I put another one in the other USB3 port and it initialized fine with no errors. Then I checked it with iwlist scanning and both adapters function. lsusb shows both adapters at 5000Mbps on the 3.0 bus. Rebooted a few times and repeated. Ran airodump-ng scans and they work..

Also, I reached out to ANDDEAR and they sent me these drivers that should come on a CD with the adapter. They're 6 years old. The documentation makes reference to the Ralink/Mediatek RT2870 chipset? But there are .bin files related to mt7612 and mt7662. This looks like a nightmare to implement given their "instructions" - what to do?

Are these drivers included in the wlanpi system image or is it something else? dmesg | grep -i mt76 info displayed is identical to other builds we've tested (because it's polling the adapter, not the driver?).

mt76x2u 1-1.3:1.0: ASIC revision: 76120044 mt76x2u 1-1.3:1.0: ROM patch build: 20141115060606a mt76x2u 1-1.3:1.0: Firmware Version: 0.0.00 mt76x2u 1-1.3:1.0: Build: 1 mt76x2u 1-1.3:1.0: Build Time: 201507311614 mt76x2u 1-1.4:1.0: ASIC revision: 76120044 mt76x2u 1-1.4:1.0: ROM patch build: 20141115060606a mt76x2u 1-1.4:1.0: Firmware Version: 0.0.00 mt76x2u 1-1.4:1.0: Build: 1 mt76x2u 1-1.4:1.0: Build Time: 201507311614 usbcore: registered new interface driver mt76x2u

amisix commented 2 years ago

@morrownr Got a new error message this morning. ERROR Transfer event..?

xhci_hcd 0000:01:00.0: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 0 comp_code 4 xhci_hcd 0000:01:00.0: Looking for event-dma 000000041ad45520 trb-start 000000041ad45540 trb-end 000000041ad45590 seg-start 000000041ad45000 seg-end 000000041ad45ff0 usb 2-1: USB disconnect, device number 22

It happened shortly after I received this error:

xhci_hcd 0000:01:00.0: WARN Set TR Deq Ptr cmd failed due to incorrect slot or ep state.