morrownr / 88x2bu-20210702

Linux Driver for USB WiFi Adapters that are based on the RTL8812BU and RTL8822BU Chipsets - v5.13.1
Other
923 stars 171 forks source link

(Info) Clients are dropped under heavy load in 5 GHz AP mode while USB3 is active #2

Open morrownr opened 2 years ago

morrownr commented 2 years ago

I am posting this issue as a place for everyone to add the information they have in an effort to fix the issue.

Bug: This appears to only happen when the driver is under heavy load in 5 GHz AP mode while USB3 is active on a RasPi4B. The AP will simply drop the client and the only way to get things going again is to reboot the AP. Testing has shown that if USB2 is active, there is no problem so this is a very specific issue.

Note: This issue is specific to Raspberry Pi 4B hardware as far as we can tell.

spcharc commented 2 years ago

Also, don't forget that you may need to put some heavy stress on the adapter to get failure. The stress needs to be sustained. I use iperf3 with a time duration of at least 60 seconds (-t 60). I usually see failure between 10 to 30 seconds into the test.

I don't have iperf because I am using a windows machine. Generally the result is acceptable.

I have to paste this again in case you forgot my network setup:

MyWindowsComputer >>> 88x2bu1(AP)---88x2bu2(client) >>> router---AT&T

For this test, I copied an 8GB file from my AP computer to my windows computer. It took around 2min to finish this copy work. I did a screenshot every 20% of the content copied.

Capture1 Capture2 Capture3 Capture4 Capture5 Capture6

I also copied this file back to the AP computer, it is also around 60MB/s. I don't want to make this post too long so I did not do screenshots, but if you want I can do that again and post them here.

morrownr commented 2 years ago

Hi @spcharc

I don't have iperf because I am using a windows machine.

Not a problem. I just threw out iperf3 in case you or someone was pondering something to use to stress the connection. Anything that stresses the connection should work.

Good report. If I am reading your report correctly, it seems that you did not see any anomalies. I am assuming, since you did not say, that your test was accomplished with USB2.

I am currently setting up to do 5 GHz AP mode testing on this driver. I'm going to start with the following options:

options 88x2bu rtw_drv_log_level=0 rtw_led_ctrl=1 rtw_vht_enable=2 rtw_power_mgnt=1 rtw_beamform_cap=1 rtw_dfs_region_domain=1 rtw_switch_usb_mode=2

Time to see if DFS channels will work. I'm going to start with channel 100.

Another issue I am going to take a look at is txpower and range.

I'll report results when able.

Nick

spcharc commented 2 years ago

I am assuming, since you did not say, that your test was accomplished with USB2.

Can you really reach 60MB/s data speed using usb2?

Yes 480Mbps = 60MB/s, but don't forget usb protocol overhead.

Generally, you cannot get anywhere near 60MB/s if you use usb2. Actually you will hugely surprise me if you can even reach 40MB/s wifi data transfer speed with usb2.

And don't forget 480Mbps is the speed for all usb2 devices on the same usb2 bus combined!

Time to see if DFS channels will work

My AP is running on channel 100(106) so I have already done the test using DFS channels

henkv1 commented 2 years ago

I experienced a connection drop after about 30 seconds when using a 5GHz AP using USB 3 and a channel width of 40 or 80 MHz. I was unable to reproduce this with one the following configuratons:

morrownr commented 2 years ago

I experienced a connection drop after about 30 seconds when using a 5GHz AP using USB 3 and a channel width of 40 or 80 MHz. I was unable to reproduce this with one the following configuratons:

  • 2.4 GHz AP
  • 5GHz AP and USB2
  • 5GHz AP, USB3 and a channel width of 20MHz

That is what I am seeing as well. 5GHz AP mode using USB3 will drop offline while under load. The only way to get the AP going again is to reboot. I am currently assessing various settings.

Would you mind sharing your hostapd.conf and 88x2bu.conf with us?

I took a fair amount of time this morning to see if USB3 has any detrimental effects on managed mode and I have found nothing. Managed mode just seems to work as it should.

henkv1 commented 2 years ago

I use the hostapd.conf from the Bridged_Wireless_Access_Point.md guide in this repository.

morrownr commented 2 years ago

HEADS UP...

I need everyone running and testing AP mode to try to confirm good results from the following settings:

hostapd.conf

vht_capab=[MAX-MPDU-11454][SHORT-GI-80][TX-STBC-2BY1][RX-STBC-1][SU-BEAMFORMER][SOUNDING-DIMENSION-2][HTC-VHT][MAX-A-MPDU-LEN-EXP7]

88x2bu,conf

options 88x2bu rtw_drv_log_level=0 rtw_led_ctrl=0 rtw_vht_enable=2 rtw_power_mgnt=1 rtw_beamform_cap=1 rtw_dfs_region_domain=1 rtw_switch_usb_mode=1

Setup:

RasPi4B RasPiOS 32 bit, kernel 5.10 8812bu chipset based adapter

Status: The main thing that I changed just before seeing solid results was the vht_capab= line in hostapd.conf. I have been pounding this AP with multiple clients and with iperf3 set to -t 60 so that it pounds for 60 seconds. I cannot get the AP to go down. To make sure this is a clean test, request you ./remove-driver.sh, delete the directory and start the installation instructions over.

Please advise results.

Nick

morrownr commented 2 years ago

Continuation of above message...

I just finished a 10 minute long POUNDING with iperf3. No drops out at all. Solid connection. Only 2 retries over 10 minutes at full speed.

Just to make sure I had a USB3 set: $ lsusb -t

Port 2: Dev 2, If 0, Class=Vendor Specific Class, Driver=rtl88x2bu, 5000M

This is very interesting.

morrownr commented 2 years ago

I use the hostapd.conf from the Bridged_Wireless_Access_Point.md guide in this repository.

@henkv1

It looks like that is going to have to be changed!

spcharc commented 2 years ago

My AP is quite stable under USB3, that is why I don't understand why for many people it is not stable. And now it seems we have found the reason.

vht_capab=[MAX-MPDU-11454][SHORT-GI-80][TX-STBC-2BY1][RX-STBC-1][SU-BEAMFORMER][SOUNDING-DIMENSION-2][HTC-VHT][MAX-A-MPDU-LEN-EXP7]

However [MAX-A-MPDU-LEN-EXP7] is obviously not supported based on the report from iw list. I am really surprised your hostapd can start without complaining. It does not make sense.

BTW, my setting is:

ht_capab=[LDPC][HT20][HT40+][SHORT-GI-20][SHORT-GI-40][RX-STBC1][MAX-AMSDU-7935]
vht_capab=[RX-STBC-1][MAX-A-MPDU-LEN-EXP3][MAX-MPDU-11454][RXLDPC][SHORT-GI-80][TX-STBC-2BY1][HTC-VHT][SU-BEAMFORMER][SU-BEAMFORMEE][SOUNDING-DIMENSION-2][BF-ANTENNA-2][MU-BEAMFORMEE]

These settings [SU-BEAMFORMER][SU-BEAMFORMEE][SOUNDING-DIMENSION-2][BF-ANTENNA-2][MU-BEAMFORMEE] might not be useful since I don't know if single user beamformer helps. I just put them in anyway.


Oh forgot to mention I set rtw_beamform_cap=11

henkv1 commented 2 years ago

I tried all combinations of settings, but I was unable to get a stable setup. Sometimes it appears to be stable, but after a reboot (or two) things are messed up again.

My system is: Raspberry Pi 4b 2GB Arch Linux Arm Kernel 5.10.75-1-raspberrypi4-ARCH Realtek 8812bu USB3.0 802.11ac 1200M Adapter

morrownr commented 2 years ago

An example of what I see in an iperf3 log regarding this issue:

[ 5] 28.00-29.00 sec 26.2 MBytes 220 Mbits/sec 0 1.96 MBytes
[ 5] 29.00-30.00 sec 31.2 MBytes 262 Mbits/sec 0 1.96 MBytes
[ 5] 30.00-31.00 sec 37.5 MBytes 315 Mbits/sec 0 1.96 MBytes
[ 5] 31.00-32.00 sec 12.5 MBytes 105 Mbits/sec 0 1.96 MBytes
[ 5] 32.00-33.00 sec 2.50 MBytes 21.0 Mbits/sec 0 1.96 MBytes
[ 5] 33.00-34.00 sec 1.25 MBytes 10.5 Mbits/sec 1 1.41 KBytes
[ 5] 34.00-35.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 35.00-36.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 36.00-37.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 37.00-38.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 38.00-39.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 39.00-40.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 40.00-41.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 41.00-42.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 42.00-43.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 43.00-44.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes

Notice that things are fine until around the 31 second point but things die at the 34 second point.

I don't see anything like this with the 8812au driver. The 8812au is the older generation AC1200 chipset from Realtek. I've never seen anything like this with mt7612u and mt7610u chipsets. I wonder if this is a firmware problem.

I am going to continue testing today. I am asking each of you test and report as able. When you submit a test report, please include your hardware platform, os and the 88x2bu,conf and hostapd.conf that you are using. If some are seeing this and others are not, we need to narrow down when we are and when we are not seeing this problem.

Thanks,

Nick

morrownr commented 2 years ago

I tried all combinations of settings, but I was unable to get a stable setup. Sometimes it appears to be stable, but after a reboot (or two) things are messed up again.

Raspberry Pi 4b 2GB

My AP test system is a RasPi4b 4 GB.

We need the others to tell us if they are using RasPis.

morrownr commented 2 years ago

Good morning @spcharc

My AP is quite stable under USB3, that is why I don't understand why for many people it is not stable. And now it seems we have found the reason.

After more testing last night, I am not so sure we have found the reason. I am going to continue testing.

What hardware and os are you using?

vht_capab=[MAX-MPDU-11454][SHORT-GI-80][TX-STBC-2BY1][RX-STBC-1][SU-BEAMFORMER][SOUNDING-DIMENSION-2][HTC-VHT][MAX-A-MPDU-LEN-EXP7]

However [MAX-A-MPDU-LEN-EXP7] is obviously not supported based on the report from iw list. I am really surprised your hostapd can start without complaining. It does not make sense.

I would not use the word "obviously" when working with Realtek drivers. They often feed bad info to iw. I used the flag you are questioning after determining it was supported in hw by other means. I can do more research on this topic.

BTW, my setting is:

ht_capab=[LDPC][HT20][HT40+][SHORT-GI-20][SHORT-GI-40][RX-STBC1][MAX-AMSDU-7935]
vht_capab=[RX-STBC-1][MAX-A-MPDU-LEN-EXP3][MAX-MPDU-11454][RXLDPC][SHORT-GI-80][TX-STBC-2BY1][HTC-VHT][SU-BEAMFORMER][SU-BEAMFORMEE][SOUNDING-DIMENSION-2][BF-ANTENNA-2][MU-BEAMFORMEE]

These settings [SU-BEAMFORMER][SU-BEAMFORMEE][SOUNDING-DIMENSION-2][BF-ANTENNA-2][MU-BEAMFORMEE] might not be useful since I don't know if single user beamformer helps. I just put them in anyway.

Oh forgot to mention I set rtw_beamform_cap=11

Thanks for this info.

henkv1 commented 2 years ago

I see the same behavior. The connection drops after 30 seconds, but sometimes earlier. Here is the output of some tests:

[  5]  18.00-19.00  sec  53.8 MBytes   451 Mbits/sec    0   2.63 MBytes       
[  5]  19.00-20.00  sec  52.5 MBytes   440 Mbits/sec    0   2.63 MBytes       
[  5]  20.00-21.00  sec  55.0 MBytes   461 Mbits/sec    0   2.63 MBytes       
[  5]  21.00-22.00  sec  52.5 MBytes   440 Mbits/sec    0   2.63 MBytes       
[  5]  22.00-23.00  sec  52.5 MBytes   440 Mbits/sec    0   2.63 MBytes       
[  5]  23.00-24.00  sec  53.8 MBytes   451 Mbits/sec    0   2.63 MBytes       
[  5]  24.00-25.00  sec  53.8 MBytes   451 Mbits/sec    0   2.63 MBytes       
[  5]  25.00-26.00  sec  53.8 MBytes   451 Mbits/sec    0   2.63 MBytes       
[  5]  26.00-27.00  sec  17.5 MBytes   147 Mbits/sec    1   1.41 KBytes       
[  5]  27.00-28.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  28.00-29.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  29.00-30.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  30.00-31.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  31.00-32.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes  

[  5]  26.00-27.00  sec  36.2 MBytes   304 Mbits/sec    0   2.59 MBytes       
[  5]  27.00-28.00  sec  33.8 MBytes   283 Mbits/sec    0   2.59 MBytes       
[  5]  28.00-29.00  sec  21.2 MBytes   178 Mbits/sec    1   1.41 KBytes       
[  5]  29.00-30.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  30.00-31.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  31.00-32.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  32.00-33.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  33.00-34.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  34.00-35.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes 

Connecting to host 192.168.1.100, port 5201
[  5] local 192.168.1.61 port 41488 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  35.8 MBytes   300 Mbits/sec    0   1.44 MBytes       
[  5]   1.00-2.00   sec  31.2 MBytes   262 Mbits/sec    0   2.59 MBytes       
[  5]   2.00-3.00   sec  38.8 MBytes   325 Mbits/sec    0   2.59 MBytes       
[  5]   3.00-4.00   sec  36.2 MBytes   304 Mbits/sec    0   2.59 MBytes       
[  5]   4.00-5.00   sec  38.8 MBytes   325 Mbits/sec    0   2.59 MBytes       
[  5]   5.00-6.00   sec  47.5 MBytes   398 Mbits/sec    0   2.59 MBytes       
[  5]   6.00-7.00   sec  38.8 MBytes   325 Mbits/sec    0   2.59 MBytes       
[  5]   7.00-8.00   sec  41.2 MBytes   346 Mbits/sec    0   2.59 MBytes       
[  5]   8.00-9.00   sec  40.0 MBytes   336 Mbits/sec    0   2.59 MBytes       
[  5]   9.00-10.00  sec  3.75 MBytes  31.4 Mbits/sec    1   1.41 KBytes       
[  5]  10.00-11.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  11.00-12.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  12.00-13.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  13.00-14.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  14.00-15.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  15.00-16.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  16.00-17.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  17.00-18.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes 
morrownr commented 2 years ago

I see the same behavior. The connection drops after 30 seconds, but sometimes earlier. Here is the output of some tests:


[  5]  18.00-19.00  sec  53.8 MBytes   451 Mbits/sec    0   2.63 MBytes       
[  5]  19.00-20.00  sec  52.5 MBytes   440 Mbits/sec    0   2.63 MBytes       
[  5]  20.00-21.00  sec  55.0 MBytes   461 Mbits/sec    0   2.63 MBytes       
[  5]  21.00-22.00  sec  52.5 MBytes   440 Mbits/sec    0   2.63 MBytes       
[  5]  22.00-23.00  sec  52.5 MBytes   440 Mbits/sec    0   2.63 MBytes       
[  5]  23.00-24.00  sec  53.8 MBytes   451 Mbits/sec    0   2.63 MBytes       
[  5]  24.00-25.00  sec  53.8 MBytes   451 Mbits/sec    0   2.63 MBytes       
[  5]  25.00-26.00  sec  53.8 MBytes   451 Mbits/sec    0   2.63 MBytes       
[  5]  26.00-27.00  sec  17.5 MBytes   147 Mbits/sec    1   1.41 KBytes       
[  5]  27.00-28.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  28.00-29.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       

Been there, done that.

We are going to beat this!

Try this:

VHT Capabilities Info: 0x3c001b2 (hw vht capab) (with beamforming turned off)

2- [MAX-MPDU-11454]

b- [RXLDPC] b- [SHORT-GI-80] b- [TX-STBC-2BY1]

1- [RX-STBC-1]

c- [HTC-VHT]

3c -[MAX-A-MPDU-LEN-EXP7]

hostapd.conf

ht_capab=[LDPC][HT40+][HT40-][SHORT-GI-20][SHORT-GI-40][RX-STBC1][MAX-AMSDU-7935]

vht_capab=[MAX-MPDU-11454][RXLDPC][SHORT-GI-80][TX-STBC-2BY1][RX-STBC-1][HTC-VHT][MAX-A-MPDU-LEN-EXP7]

88x2bu.conf

options 88x2bu rtw_drv_log_level=0 rtw_led_ctrl=2 rtw_vht_enable=2 rtw_power_mgnt=1 rtw_beamform_cap=0 rtw_dfs_region_domain=1 rtw_switch_usb_mode=1

spcharc commented 2 years ago

What hardware and os are you using?

I am using pc with intel cpu. OS is linux mint.

I will try [MAX-A-MPDU-LEN-EXP7] and see if it works better

morrownr commented 2 years ago

@spcharc

I'll assume 64 bit Intel CPU and Linux Mint is what you meant. (Sounds like my main dev box.) It is interesting that you are not seeing the problem. I have no hard data on this issue but I have only seen it with ARM CPUs. I have limited resources in my little lab and currently do not have a x86/and64 system allocated to testing AP mode. If necessary, I can make that happen as I really want to get this fixed.

I did some work in the code this morning. I tried some things but nothing made any difference and I can't find the difference in this driver and the ones for the 8812au and 8821au, which do not show this problem.

I don't know if [MAX-A-MPDU-LEN-EXP7] will work any better but it would add to our knowledge if you can test and report. I know what iw says but I don't trust iw with these Realtek drivers. With beamforming turned off, hostapd reports that hw is saying there vht_capab of the 8812bu adapter that I have is 0x3c001b2. The settings I posted above put us right on that capab. Let me see if I can post a vht_capab example matrix so you and the others can check my work:

VHT_capab

Cheers

spcharc commented 2 years ago

I'll assume 64 bit Intel CPU and Linux Mint is what you meant.

It is correct

It is interesting that you are not seeing the problem. I have no hard data on this issue but I have only seen it with ARM CPUs.

I don't think it has something to do with architecture. I was just answering your question What hardware and os are you using?

My guess is that the chip / driver in the client device may make a difference. I do have an old apple ipod that somehow drops wifi connection. It does not drop wifi connection if connected to my router. So there might be a compatible problem here?

However for other devices, the ap seems stable. I can even play online games and get a stable ping without dropping. I am confused.

spcharc commented 2 years ago

Both of my 88x2bu adapters return 0x03d179b2 with rtw_beamform_cap=11

I interpreted the bits manually and the result is here: (and please help check my work, I may make mistakes)

0000 0011 1101 0001 0111 1001 1011 0010
00 = reserved
00 = Rx / Tx antenna pattern consistency not supported
00 = vht link adaptation not supported
111 = max A-MPDU length: exponent 7 (WHAT??? iw list shows 0x003)
1 = +HTC-VHT supported
0 = vht txop ps not supported
1 = mu beamformee supported
0 = mu beamformer not supported
001 = number of sounding dimensions is 1+1=2
011 = number of beamformer attenna supported is 1+3=4 (looks like we can use [BF-ANTENNA-4])
1 = su beamformee supported
1 = su beamformer supported
001 = Rx STBC 1 spatial stream
1 = Tx STBC supported
0 = Short GI for 160 / 80+80 not supported
1 = Short GI for 80 supported
1 = Rx LDPC supported
00 = 160 / 80+80 not supported
10 = max MPDU length [3895, 7991, 11454][2] = 11454

So ... is iw list wrong?

spcharc commented 2 years ago

It seems I should change the config file to this:

ht_capab=[LDPC][HT20][HT40+][SHORT-GI-20][SHORT-GI-40][RX-STBC1][MAX-AMSDU-7935]
vht_capab=[RX-STBC-1][MAX-A-MPDU-LEN-EXP7][MAX-MPDU-11454][RXLDPC][SHORT-GI-80][TX-STBC-2BY1][HTC-VHT][SU-BEAMFORMER][SU-BEAMFORMEE][SOUNDING-DIMENSION-2][BF-ANTENNA-4][MU-BEAMFORMEE]

I am testing this config now.


Hostapd started without complaining. Amazing. Will test if it is stable.

spcharc commented 2 years ago

I also interpreted 0x3c001b2 reported by @morrownr:

0000 0011 1100 0000 0000 0001 1011 0010
00 = reserved
00 = Rx / Tx antenna pattern consistency not supported
00 = vht link adaptation not supported
111 = max A-MPDU length: exponent 7
1 = +HTC-VHT supported
0 = vht txop ps not supported
0 = mu beamformee not supported
0 = mu beamformer not supported
000 = number of sounding dimensions is 1+0=1
000 = number of beamformer attenna supported is 1+0=1
0 = su beamformee not supported
0 = su beamformer not supported
001 = Rx STBC 1 spatial stream
1 = Tx STBC supported
0 = Short GI for 160 / 80+80 not supported
1 = Short GI for 80 supported
1 = Rx LDPC supported
00 = 160 / 80+80 not supported
10 = max MPDU length [3895, 7991, 11454][2] = 11454
morrownr commented 2 years ago

I don't think it has something to do with architecture.

Is that just a gut feeling?

What hardware and os are you using?

i7 Intel CPU with Linux Mint Cinnamon (20.2). This is my main work and dev box. I have several systems of various ages. The two oldest systems that are used for testing are 32 bit. Even older systems are in use but don't do wifi. My oldest operational system is a 486 IBM desktop from 1993.

My guess is that the chip / driver in the client device may make a difference.

I have been pondering this as well but have been testing with clients that have various wifi drivers and chipsets and so far I have seen no evidence to suggest this is the case.

I do have an old apple ipod that somehow drops wifi connection. It does not drop wifi connection if connected to my router. So there might be a compatible problem here?

Probably. Issues like this can be a real pain to track down.

However for other devices, the ap seems stable. I can even play online games and get a stable ping without dropping. I am confused.

Wifi is confusing. I hurts my brain everytime I take deep dive into the code. I learned coding on an IBM mainframe using FORTRAN. I've learned numerous languages and have even helped with a device driver or two over the years but working on this wifi stuff is like the Twilight Zone.

morrownr commented 2 years ago

I also interpreted 0x3c001b2 reported by @morrownr:

0000 0011 1100 0000 0000 0001 1011 0010
00 = reserved
00 = Rx / Tx antenna pattern consistency not supported
00 = vht link adaptation not supported
111 = max A-MPDU length: exponent 7
1 = +HTC-VHT supported
0 = vht txop ps not supported
0 = mu beamformee not supported
0 = mu beamformer not supported
000 = number of sounding dimensions is 1+0=1
000 = number of beamformer attenna supported is 1+0=1
0 = su beamformee not supported
0 = su beamformer not supported
001 = Rx STBC 1 spatial stream
1 = Tx STBC supported
0 = Short GI for 160 / 80+80 not supported
1 = Short GI for 80 supported
1 = Rx LDPC supported
00 = 160 / 80+80 not supported
10 = max MPDU length [3895, 7991, 11454][2] = 11454

You got the same thing that I did, The difference in vht capab we are seeing in simply me using rtw_beamform_cap=0 and you using rtw_beamform_cap=11.

So, this is worth pondering. For some of the capabilities we are telling the driver to tell iw to tell us what we just told the driver to tell us.

I think it was you earlier in this thread that mentioned that beamforming is of marginal benefit. Everything I have read and my own testing seems to indicate you are very correct. While documenting beamforming and adding an option to 88x2bu.conf gives folks an easier ability to play with it if they want, I am beginning to question whether the AP mode documentation should include beamforming. Thoughts?

morrownr commented 2 years ago

Both of my 88x2bu adapters return 0x03d179b2 with rtw_beamform_cap=11

I interpreted the bits manually and the result is here: (and please help check my work, I may make mistakes)


0000 0011 1101 0001 0111 1001 1011 0010
00 = reserved
00 = Rx / Tx antenna pattern consistency not supported
00 = vht link adaptation not supported
111 = max A-MPDU length: exponent 7 (WHAT??? iw list shows 0x003)

$ iw list is wrong.

1 = +HTC-VHT supported
0 = vht txop ps not supported
1 = mu beamformee supported
0 = mu beamformer not supported
001 = number of sounding dimensions is 1+1=2
011 = number of beamformer attenna supported is 1+3=4 (looks like we can use [BF-ANTENNA-4])
1 = su beamformee supported
1 = su beamformer supported
001 = Rx STBC 1 spatial stream
1 = Tx STBC supported
0 = Short GI for 160 / 80+80 not supported
1 = Short GI for 80 supported
1 = Rx LDPC supported
00 = 160 / 80+80 not supported
10 = max MPDU length [3895, 7991, 11454][2] = 11454

Looks good to me.

So ... is iw list wrong?

Yes.

I don't trust iw with Realtek drivers. It isn't iw's fault, it is Realtek's fault. Realtek usb wifi driver dev works in mysterious ways. We don't get to report bugs. We get little to no documentation. We do not get public releases... try to find where I got this code. We don't get to test prerelease versions. It beats the hell out of me how Realtek gets these drivers to do anything right.

To top it all off, the drivers Realtek is making are the wrong technology in the wrong location. These drivers should be mac80211 technology and should be in the kernel. If I needed to make a list of things showing how not to do driver development, I wouldn't have to go far, I would just list how Realtek does it.

spcharc commented 2 years ago

I don't trust iw with Realtek drivers. It isn't iw's fault, it is Realtek's fault. Realtek usb wifi driver dev works in mysterious ways.

ah, now I finally see why you recommended 7612u wifi adapters ... perhaps I should purchase one

try to find where I got this code.

must be on some wifi adapter manufacturers' website?


Update:

I purchased a 7612u adapter. I know it does not fit in this issue, but it seems to me 7612u is not a good choice as an AP?

It has only one driver parameter which is not wifi related. You cannot setup a dfs channel wifi.

But most importantly, its speed is slower than realtek adapters. (however, the problem can be that I purchased a cheap adapter)

Capture7

I was at the same location during this test, just replaced the 88x2bu adapter to this mt76 one. It seems my 88x2bu has 50% better performance in terms of wifi speed.

I am wondering, are you guys getting around 500Mbps during your iperf tests with mt76 driver? (again, the problem may be my adapter, so I need input from you guys)

I can get to 500Mbps using 88x2bu adapter (see my screenshots in previous post https://github.com/morrownr/88x2bu-20210702/issues/2#issuecomment-955101147 , it shows 64MB/s = 512Mbps)

henkv1 commented 2 years ago

Try this:

VHT Capabilities Info: 0x3c001b2 (hw vht capab) (with beamforming turned off)

2- [MAX-MPDU-11454]

b- [RXLDPC] b- [SHORT-GI-80] b- [TX-STBC-2BY1]

1- [RX-STBC-1]

c- [HTC-VHT]

3c -[MAX-A-MPDU-LEN-EXP7]

hostapd.conf

ht_capab=[LDPC][HT40+][HT40-][SHORT-GI-20][SHORT-GI-40][RX-STBC1][MAX-AMSDU-7935]

No beamforming

vht_capab=[MAX-MPDU-11454][RXLDPC][SHORT-GI-80][TX-STBC-2BY1][RX-STBC-1][HTC-VHT][MAX-A-MPDU-LEN-EXP7]

88x2bu.conf

No beamforming

options 88x2bu rtw_drv_log_level=0 rtw_led_ctrl=2 rtw_vht_enable=2 rtw_power_mgnt=1 rtw_beamform_cap=0 rtw_dfs_region_domain=1 rtw_switch_usb_mode=1

Unfortunately, this did not work:

[  5]  44.00-45.00  sec  42.5 MBytes   356 Mbits/sec    0   2.56 MBytes       
[  5]  45.00-46.00  sec  46.2 MBytes   388 Mbits/sec    0   2.56 MBytes       
[  5]  46.00-47.00  sec  20.0 MBytes   168 Mbits/sec    1   1.41 KBytes       
[  5]  47.00-48.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  48.00-49.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  49.00-50.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  50.00-51.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  5]  51.00-52.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes 

I'll try to set up a x86_64 test system to test a different architecture.

henkv1 commented 2 years ago

I did some testing with a x86_64 system and I was not able to reproduce this issue. So it looks like an ARM/Raspberry Pi specific issue (or a 32 bit issue, but I do not have a 32 bit x86 or a 64 bit Raspberry Pi installation).

morrownr commented 2 years ago

I did some testing with a x86_64 system and I was not able to reproduce this issue. So it looks like an ARM/Raspberry Pi specific issue (or a 32 bit issue, but I do not have a 32 bit x86 or a 64 bit Raspberry Pi installation).

@henkv1

Hmmm... more evidence that this is not a x86_64 issue. I'm going to burn an sd with the 64 bit version of RasPiOS today and see if the problem shows up there.

spcharc commented 2 years ago

I did some testing with a x86_64 system and I was not able to reproduce this issue.

Wow. The reason is cpu architecture?

Why architecture has something to do with this issue?

morrownr commented 2 years ago

ah, now I finally see why you recommended 7612u wifi adapters ...

I have and use 7612u and 7610u adapters as well as Realtek based adapters.

try to find where I got this code.

must be on some wifi adapter manufacturers' website?

I'll show you one of these days.

I purchased a 7612u adapter. I know it does not fit in this issue, but it seems to me 7612u is not a good choice as an AP?

It has only one driver parameter which is not wifi related.

That does seem strange at first. When you start looking at the issues, such as USB2 vs USB3, you will find that the 761x drivers check and if everything is good, it goes to USB3 mode. The way I think of the difference in the Realtek vs. Mediatek drivers is that Realtek's drivers are like a standard transmission whereas Mediatek's drivers are like an automatic transmission.

My experience is that the 7612u and 7610u chipsets and drivers make very good APs. If solid long term uptime is high on your list, they meet this need. I can go into detail later as it is getting late and there are some other things I want to talk about.

You cannot setup a dfs channel wifi.

This is an issue. At least we can talk to the devs about it. Let me see where that issue is.

But most importantly, its speed is slower than realtek adapters. (however, the problem can be that I purchased a cheap adapter)

My Alfa ACM (7612u) can sustain a little over 400 Mb/s and my Alfa ACHM (7610u) can sustain around 210 Mb/s. Can specific Realtek adapters in the right conditions do better, sure. There is a lot of variation in quality and capability with adapters with all chipsets.

I was at the same location during this test, just replaced the 88x2bu adapter to this mt76 one. It seems my 88x2bu has 50% better performance in terms of wifi speed.

This is within expectation. I have conducted performance tests using 8812bu and 7612u based adapters. If I take the right two adapters, I can show you results where Mediatek wins and vice versa. My opinion is that if everything is equal for a test, the 8812bu would beat the 7612u but I can't say exactly how much but my estimate is that the difference would be around 10%. Here is a link to a test I did earlier this year:

https://github.com/morrownr/USB-WiFi/blob/main/Performance_Comparison.md

I am wondering, are you guys getting around 500Mbps during your iperf tests with mt76 driver? (again, the problem may be my adapter, so I need input from you guys)

I have 3 adapters based on the 7612u that I can and do use for AP mode. I have another that is small and portable with limited range so I would not use it as an AP. The 3 I do use as APs can all sustain over 400 Mb/s and it is a very stable, consistent speed. I've never seen 500 Mb/s with any 7612u based adapter.

I can get to 500Mbps using 88x2bu adapter

I have seen as high as 550 Mb/s from a 8812bu based adapter in managed mode. I've never seem that in AP mode. Interesting observation from this end is that Realtek adapters on the whole seem to do better speed wise in manged mode whereas Mediatek adapters seem to do better in AP mode.

It seems to me that the best way to figure this out is what you are doing.

Here is my opinion concerning some categories that can be looked at:

Power usage:

7612u - usually around 380 mA while maxed out 8812bu - usually a little over 500 mA while maxed out

This can be important is some use cases. RasPi's only offer a total of 1200 mA for the entire USB subsystem. This is an issue on a lot of this little boards that are in use these days and less power usage is better.

Operating temp:

Mediatek tends to do much better overall in this category but that can be explain with the lower power usage.

Monitor mode operation:

Mediatek is by-the-book solid whereas Realtek is problematic. Having iw lie to you when in managed or AP mode is interesting, in monitor mode, it really sucks.

Support: We can report issues to the Mediatek devs and we can complain about no AP mode DFS support. Good luck trying to contact anyone at Realtek usb wifi.

Long term problem free operation: Well, this category is not even close. Mediatek's drivers are in the kernel where they are maintained and upgraded. Major update to you Linux distro, no sweat. Not even a consideration. On the other hand, major upgrade to your distro with Realtek drivers... oh shit. I forgot to uninstall the old version of the driver so now I have a mess. Now where is a version of the driver I need for kernel 5.x.x.

Speed: Mediatek will get beat here.

For almost all of my use cases, adequate speed along with stable, dependable, problem-free operation is what is important. I do like to mess with the Realtek drivers because there always seems to be something to work on and it can be fun.

morrownr commented 2 years ago

I did some testing with a x86_64 system and I was not able to reproduce this issue.

Wow. The reason is cpu architecture?

Why architecture has something to do with this issue?

Yes sir, Different architectures require different instructions. Long story...maybe we can address it in the future?

I installed the 64 bit version of the RasPiOS today to test and the short version report is that the problem is still in the 64 bit version so it is looking like it is not a 32 vs. 64 bit thing. It is looking more and more like a ARM/ARM64 thing and based on additional testing this evening, I think it is limited to 2 boards: RasPi4B and RasPi400.

Is anyone that reads this thread seeing this problem on anything other than the 2 boards I just mentioned?

I have tested every setting and even messed with the code and I am only seeing this problem on a RasPi4B with USB3 support turned on. I have researched reports on the internet. I have pounded the adapters (4) and have studied the logs.

My working theory right now is:

The results I am seeing are more consistent with an EMI problem as opposed to a computer coding problem. I can't fully explain what I am seeing with computer code. The RasPi4B usb subsystem is the first one to support USB3 and it has a long track record of hard to explain problems. It very well could be the specific problem is this exact chipset and the specific usbsystem in the Pi4. For us to explore further would require us to put on our EE hats and find a lab with the appropriate equipment... and even if we find and spell out the exact problem, so what? There may not be a software fix.

I am okay documenting this issue stating that it is specific to certain RasPi hardware and it only shows in USB3 mode. Performance in USB2 mode is still good enough for almost all use cases and it is very stable.

I don't see this with the 8812au chipset and driver but then that is a different chipset that runs at different frequencies. We could have a unique situation on our hands but it is an issue because there are millions of RasPi4b's out there.

Unless anyone sees something wrong with what I said, I think documenting the issue and pressing on with any additional issues is the right thing to do.

spcharc commented 2 years ago

Different architectures require different instructions. Long story...

When cpu talking to usb devices, I don't think different cpu architectures matter ... the usb protocol is the same for different cpus.

Just like accessing "www.google.com" on a rpi / pc, they goes through exactly the same process to establish connection using tcp protocol and retrieve the webpage. Internal cpu instructions don't matter.

That is why this issue is really weird when it can only be reproduced on a rpi.

Can someone reading this please test this setup: I think we can let the Makefile untouched and see what will happen. Please install this driver on a pi WITHOUT running raspi32.sh first. And then please do the iperf test to see if the connection drops after 30s.

spcharc commented 2 years ago

For almost all of my use cases, adequate speed along with stable, dependable, problem-free operation is what is important.

Thanks for your long reply. I completely agree with this statement.

Today I tested 7612u on windows and the driver do support usb mode switch on windows.

Realtek should seriously consider improving their drivers. Currently their 8812bu windows driver do not support usb mode switch, and no matter what I did it just stayed in usb2 mode. This made the adapter quite slow under windows and only useful under linux.

spcharc commented 2 years ago

So ... is iw list wrong?

Yes.

I don't trust iw with Realtek drivers. It isn't iw's fault, it is Realtek's fault. Realtek usb wifi driver dev works in mysterious ways

Perhaps, it really is iw's fault

I am looking at my intel 3165 card. It got this report from iw list

image

I tried to parse 0x33807120 manually, and got this result:

0011 0011 1000 0000 0111 0001 0010 0000

00 = reserved
11 = Rx / Tx antenna pattern consistency supported
00 = vht link adaptation not supported
111 = max A-MPDU length: exponent 7
0 = +HTC-VHT not supported
0 = vht txop ps not supported
0 = mu beamformee not supported
0 = mu beamformer not supported
000 = number of sounding dimensions is 1+0=1
011 = number of beamformer antenna supported is 1+3=4
1 = su beamformee supported
0 = su beamformer not supported
001 = Rx STBC 1 spatial stream
0 = Tx STBC not supported
0 = Short GI for 160 / 80+80 not supported
1 = Short GI for 80 supported
0 = Rx LDPC not supported
00 = 160 / 80+80 not supported
00 = max MPDU length [3895, 7991, 11454][0] = 3895

So it seems max A-MPDU should again be 0x007, not 0x003 reported by iw list


Update: it is not iw's fault. Please read the post below.

spcharc commented 2 years ago

Here are two images I have found online. The ht_capab matrix and A-MPDU matrix. I think it can be helpful to paste it here.

image

image


So I finally found the reason why iw list failed to show the correct result:

The A-MPDU length field under HT has only 2 bits, so the maximum exponent is 3. iw list always shows the value in this field.

For A-MPDU length under VHT, you need to use vht_capab matrix to find the answer.

morrownr commented 2 years ago

Glad you dug this up. iw is not as clear about where VHT stuff begins as it could be. Maybe I can add some of this knowledge to the driver docs as I get time.

As you know, I went ahead and went public this morning after cleaning up the docs. Let's see what happens. We'll know within a few days if there are any major basic problems because the old version repo gets about 1,000 clones and 3,000 hits per week and I put a message in the README channeling users over here. Yes, the traffic load is that heavy for this driver. The low amount of issues that are reported indicates how well the old driver is working and I think this new driver does offer some good improvements.

morrownr commented 2 years ago

@spcharc

Question: Look at the VHT capab below and tell me what you think?

0x318001b0

vht_capab=[RXLDPC][SHORT-GI-80][TX-STBC-2BY1][RX-STBC-1][MAX-A-MPDU-LEN-EXP3][RX-ANTENNA-PATTERN][TX-ANTENNA-PATTERN]

The issue at hand is [MAX-A-MPDU-LEN-EXP3]. It is not clear to me what it should be. I have some good documentation around here somewhere but can't find it.

morrownr commented 2 years ago

@spcharc

Found the answer in the hostapd source:

define VHT_CAP_MAX_A_MPDU_LENGTH_EXPONENT_1 ((u32) BIT(23))

define VHT_CAP_MAX_A_MPDU_LENGTH_EXPONENT_2 ((u32) BIT(24))

define VHT_CAP_MAX_A_MPDU_LENGTH_EXPONENT_3 ((u32) BIT(23) | BIT(24))

define VHT_CAP_MAX_A_MPDU_LENGTH_EXPONENT_4 ((u32) BIT(25))

define VHT_CAP_MAX_A_MPDU_LENGTH_EXPONENT_5 ((u32) BIT(23) | BIT(25))

define VHT_CAP_MAX_A_MPDU_LENGTH_EXPONENT_6 ((u32) BIT(24) | BIT(25))

define VHT_CAP_MAX_A_MPDU_LENGTH_EXPONENT_MAX ((u32) BIT(23) | BIT(24) | BIT(25))

Since BIT(23) an BIT(24) are set, the answer is 3.

spcharc commented 2 years ago

Question: Look at the VHT capab below and tell me what you think? 0x318001b0

0011 0001 1000 0000 0000 0001 1011 0000

00 = reserved
11 = Rx / Tx antenna pattern consistency supported
00 = vht link adaptation not supported
011 = max A-MPDU length: exponent 3
0 = +HTC-VHT not supported
0 = vht txop ps not supported
0 = mu beamformee not supported
0 = mu beamformer not supported
000 = number of sounding dimensions is 1+0=1
000 = number of beamformer antenna supported is 1+0=1
0 = su beamformee not supported
0 = su beamformer not supported
001 = Rx STBC 1 spatial stream
1 = Tx STBC supported
0 = Short GI for 160 / 80+80 not supported
1 = Short GI for 80 supported
1 = Rx LDPC supported
00 = 160 / 80+80 not supported
00 = max MPDU length [3895, 7991, 11454][0] = 3895
morrownr commented 2 years ago

Hi @spcharc @henkv1 @exzombie @ajitwarrier

I have been continuing to work the problem behind the scenes. I've been in contact with the Mediatek kernel devs as they have a module parameter for this issue. I had basically narrowed the search to the USB3 hub on the RasPi4B but somebody else recently submitted the following patch that is now flowing into the RasPiOS when you update:

https://github.com/raspberrypi/linux/commit/a538fd26f82b101cb6fb963042f3242768e628d4

I have been testing. I can run this driver wide open in 5GHz with a 80 MHz wide channel now with no problems noted. Same with the mt7612u driver. No module parameter required.

It was a bug in the VL805 chipset based USB3 hub that the RasPi4B uses. There are a lot of other systems out there that use this part.

I'll try to write this up soon and change the documentation. Now if I could get to the bottom of why we can't run two 8812bu adapters in the same system... anyone want to help with that?

FYI: A driver for the my7921u chipset is now flowing into the kernel. That means new WiFi 6/6e USB WiFi adapters soon and the driver will be in the Linux before the products ship. Discussions over at my USB-WiFi repo.

Regards

JasonVarney commented 2 years ago

Hi @morrownr

I compiled and installed a new kernel from this branch, https://github.com/raspberrypi/linux/commits/rpi-5.10.y, with the [usb: xhci: add a quirk for Superspeed bulk OUT transfers on VL805] patch, was hoping to get stable USB 3.0 performance, was seeing around 450Mbps with cheapo RTL88x2bu [AC1200 Techkey] wifi dongle but it's still unstable and locks-up when using rtw_switch_usb_mode=1 [1 = Switch from usb 2.0 to usb 3.0]. Is there an extra module parameter I need to add, I took a look at /sys/module/88x2bu/parameters/ but nothing seems relevant, I'm guessing this is a new parameter you mentioned, yet to be added to your wonderful 88x2bu driver.

Raspberry Pi 4 4GB running 4.10.y 64bit (with [usb: xhci: add a quirk for Superspeed bulk OUT transfers on VL805] patch) Used your AP_Mode-Bridged_Wireless_Access_Point guide, works great! Still get around 250-280Mbps in USB 2.0 mode

88x2bu.conf

options 88x2bu rtw_drv_log_level=0 rtw_led_ctrl=1 rtw_vht_enable=2 rtw_power_mgnt=0 rtw_beamform_cap=0 rtw_switch_usb_mode=1 rtw_country_code=US rtw_wireless_mode=84 rtw_dfs_region_domain=0 

hostapd-5g.conf

ssid=******
wpa_passphrase=*******

country_code=US

interface=wlxe0e1a9193073
driver=nl80211

ieee80211n=1
ieee80211ac=1
ieee80211d=1

#Enables support for 5GHz DFS channels
ieee80211h=1

bridge=br0

macaddr_acl=0
auth_algs=1
ignore_broadcast_ssid=0
wpa=2

wpa_key_mgmt=WPA-PSK
wpa_pairwise=TKIP
rsn_pairwise=CCMP

hw_mode=a
channel=36
wmm_enabled=1

country_code=US

#require_ht=1
#require_vht=1

#vht_capab=[LDPC][HT40+][HT40-][SHORT-GI-20][SHORT-GI-40][MAX-AMSDU-7935]
#ht_capab=[HT40-][HT40+][SHORT-GI-40][MAX-AMSDU-7935][DSSS_CCK-40]
ht_capab=[HT40-][HT40+][SHORT-GI-40][MAX-AMSDU-7935]

#vht_capab=[RX-LDPC][MAX-MPDU-11454][SHORT-GI-80][TX-STBC][SU-BEAMFORMEE]
vht_capab=[MAX-MPDU-11454][SHORT-GI-80][TX-STBC-2BY1][RX-STBC-1][HTC-VHT][MAX-A-MPDU-LEN-EXP7]

vht_oper_chwidth=1
vht_oper_centr_freq_seg0_idx=42
#vht_oper_centr_freq_seg1_idx=157
morrownr commented 2 years ago

Hi @JasonVarney

running 4.10.y 64bit

Hmmm... I was working on this issue a few weeks ago. I was collecting information. I decided to start with the new 64 bit release. I ran onto several networking issues immediately and did not have time to investigate or report anything. Geez... it was bad. So burned the 32 bit version over it and it is solid. I'm pretty sure the xhci patch is already in the flow. I have tested and cannot find the old problem now.

If you have an extra sd card, you might want to burn the 32 bit version and give it a go.

Regarding your module parameters, you might try this:

options 88x2bu rtw_drv_log_level=0 rtw_led_ctrl=1 rtw_vht_enable=2 rtw_power_mgnt=1 rtw_switch_usb_mode=1

The 64 bit version was in beta for what seemed like most of the last decade and now... I'm not impressed. They even shipped it with the old usb_modeswitch data file for crying out loud!

JasonVarney commented 2 years ago

Hi @morrownr

Just an update, I did try a 32bit generic Raspberry Pi OS Lite install, updated to 5.10.103, yup, pretty sure that XHCI commit has been merged. Still got a hang using USB 3.0, tried a few different combos, with power saving, without, with config.txt boost off etc, even tried running the RPi4 off a high current USB-C battery bank, it did seem to do better but subsequent tests died almost immediately so not sure it's a USB brownout issued which I assumed to be the case.

For now I have three 4GB RPi4s(64bit RPi OS) with RTL88x2bu dongles running in USB 2.0 as bridged dumb APs to extend my 5Ghz network, Gb wired backhaul. Seems to be working great, many thanks for your contribution, I've been struggling with finding the correct hardware at my current location and have had to make do.

Seems this USB issue is more complicated, could be specific to my particular dongles, slight incompatibility perhaps.

One small point, might want to add an entry to the AP_Mode-Bridged_Wireless_Access_Point guide about setting specific MAC addresses to the bridge interface (br0) in cases where one wants to clone to multiple APs, for instance...

$sudo nano /etc/systemd/network/10-create-bridge-br0.netdev

Raspberry Pi 4 One

[NetDev]
Name=br0
MACAddress=2e:7c:bf:aa:bb:11
Kind=bridge

Raspberry Pi 4 Two

[NetDev]
Name=br0
MACAddress=2e:7c:bf:aa:bb:22
Kind=bridge

Raspberry Pi 4 Three

[NetDev]
Name=br0
MACAddress=2e:7c:bf:aa:bb:33
Kind=bridge

Then set static DHCP leases for those MAC addresses on the main router/DHCP server.

Also the the WLAN name changes with every adapter/dongle so one would have to update the hostapd-5g.conf with the new interface name.

$sudo nano  /etc/hostapd/hostapd-5g.conf 
interface=[newly generated WLAN interface name here]

An example of one of my APs

$ iwconfig
lo        no wireless extensions.

eth0      no wireless extensions.

br0       no wireless extensions.

wlxe0e1a9193019  IEEE 802.11AC  ESSID:"omni"  Nickname:"WIFI@RTL88X2BU"
          Mode:Master  Frequency:5.745 GHz  Access Point: E0:E1:A9:19:30:19   
          Bit Rate:867 Mb/s   Sensitivity:0/0  
          Retry:off   RTS thr:off   Fragment thr:off
          Power Management:off
          Link Quality=55/100  Signal level=43/100  Noise level=0/100
          Rx invalid nwid:0  Rx invalid crypt:0  Rx invalid frag:0
          Tx excessive retries:0  Invalid misc:0   Missed beacon:0

So in this one particular case my /etc/hostapd/hostapd-5g.conf, "interface" needed to be changed as follows:

$sudo nano /etc/hostapd/hostapd-5g.conf
interface=wlxe0e1a9193019

A small issue many may run into, not really the scope of your guide but I had an issue booting when copying partitions with GParted, cloning the initial install, the PARTUUID in the /boot/cmdline.txt and the /etc/fstab had to be edited to match newly generated PARTUUIDs.

Find the new PARTUUIDs:

$ blkid
/dev/mmcblk0p1: LABEL_FATBOOT="boot" LABEL="boot" UUID="F914-FF4D" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="747e086f-01"
/dev/mmcblk0p2: LABEL="rootfs" UUID="1943b829-a99b-45b8-9fe5-7136dbea4c4a" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="747e086f-02"

config.txt on boot partition:

$sudo mount -o remount,rw /boot
$sudo nano /boot/cmdline.txt
console=serial0,115200 console=tty1 root=PARTUUID=747e086f-02 rootfstype=ext4 fsck.repair=yes rootwait

fstab on rootfs partition:

$sudo nano /etc/fstab
proc            /proc           proc    defaults          0       0
PARTUUID=747e086f-01  /boot           vfat    defaults,ro,flush    0       2
PARTUUID=747e086f-02  /               ext4    defaults,noatime  0       1

On the new RPi clones/copies the PARTUUID have to match the newly generated ones.

Thank you!