won't connect to hotspots generated by linux using hostapd

AndySchroder commented 1 month ago

bitaxe won't connect to a wifi hotspot generated by linux using hostapd. I've tested this on a raspi4 and a purism librem 14 laptop, both with ubuntu 22.04

Here's my hostapd config file:

root@rd:~# cat testhostap 
interface=wlan0
driver=nl80211
ssid=aa
hw_mode=b
auth_algs=1
wpa=2
wpa_passphrase=password
wpa_key_mgmt=WPA-PSK
rsn_pairwise=CCMP
channel=10
root@rd:~#

root@rd:~# 
root@rd:~# hostapd testhostap 
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED

and no connections are made. However, if trying to connect with a laptop, it works fine and some output comes in the terminal after wlan0: AP-ENABLED.

I've also tested with nmcli (https://ubuntu.com/core/docs/networkmanager/configure-wifi-access-points), and I have the same problems, but hostapd seems to allow more raw configuration and outputs.

skot commented 1 month ago

Are you sure this is a 2.4GHz WiFi network?

AndySchroder commented 1 month ago

Yes, checked that many ways. Also, hw_mode=b above forces it.

AndySchroder commented 1 month ago

More config options here explained if there is something else special that needs to be forced: https://wiki.gentoo.org/wiki/Hostapd .

skot commented 1 month ago

Does the Bitaxe log (over USB) show anything interesting when it's trying to join this network?

AndySchroder commented 1 month ago

Does USB give me a serial port? If so, what baud rate?

skot commented 1 month ago

115200 baud. The WiFi connection is one of the first things that happens, so you maybe to reset to catch it

bboerst commented 1 month ago

For reference, here's the serial output while connected to an access point created with nmcli while we troubleshoot this:

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0xc (RTC_SW_CPU_RST),boot:0x28 (SPI_FAST_FLASH_BOOT)
Saved PC:0x40375ac8
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3810,len:0x178c
load:0x403c9700,len:0x4
load:0x403c9704,len:0xcbc
load:0x403cc700,len:0x2d9c
entry 0x403c9914
I (26) boot: ESP-IDF v5.2.2-dirty 2nd stage bootloader
I (26) boot: compile time Aug 28 2024 17:27:50
I (27) boot: Multicore bootloader
I (30) boot: chip revision: v0.2
I (34) boot.esp32s3: Boot SPI Speed : 80MHz
I (38) boot.esp32s3: SPI Mode       : DIO
I (43) boot.esp32s3: SPI Flash Size : 16MB
I (48) boot: Enabling RNG early entropy source...
I (53) boot: Partition Table:
I (57) boot: ## Label            Usage          Type ST Offset   Length
I (64) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (72) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (79) boot:  2 factory          factory app      00 00 00010000 00400000
I (87) boot:  3 www              Unknown data     01 82 00410000 00300000
I (94) boot:  4 ota_0            OTA app          00 10 00710000 00400000
I (102) boot:  5 ota_1            OTA app          00 11 00b10000 00400000
I (109) boot:  6 otadata          OTA data         01 00 00f10000 00002000
I (117) boot:  7 coredump         Unknown data     01 03 00f12000 00010000
I (124) boot: End of partition table
I (129) boot: Defaulting to factory image
I (133) esp_image: segment 0: paddr=00010020 vaddr=3c0b0020 size=2c47ch (181372) map
I (174) esp_image: segment 1: paddr=0003c4a4 vaddr=3fc99e00 size=03b74h ( 15220) load
I (178) esp_image: segment 2: paddr=00040020 vaddr=42000020 size=adec4h (712388) map
I (308) esp_image: segment 3: paddr=000edeec vaddr=3fc9d974 size=01284h (  4740) load
I (309) esp_image: segment 4: paddr=000ef178 vaddr=40374000 size=15d58h ( 89432) load
I (342) boot: Loaded app from partition at offset 0x10000
I (342) boot: Disabling RNG early entropy source...
I (354) cpu_start: Multicore app
I (363) cpu_start: Pro cpu start user code
I (363) cpu_start: cpu freq: 160000000 Hz
I (364) cpu_start: Application information:
I (366) cpu_start: Project name:     esp-miner
I (371) cpu_start: App version:      v2.1.10
I (376) cpu_start: Compile time:     Aug 28 2024 17:27:46
I (383) cpu_start: ELF file SHA256:  6b3c94264...
I (388) cpu_start: ESP-IDF:          v5.2.2-dirty
I (393) cpu_start: Min chip rev:     v0.0
I (398) cpu_start: Max chip rev:     v0.99
I (403) cpu_start: Chip rev:         v0.2
I (408) heap_init: Initializing. RAM available for dynamic allocation:
I (415) heap_init: At 3FCA38F0 len 00045E20 (279 KiB): RAM
I (421) heap_init: At 3FCE9710 len 00005724 (21 KiB): RAM
I (427) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAM
I (433) heap_init: At 600FE010 len 00001FD8 (7 KiB): RTCRAM
I (440) spi_flash: detected chip: gd
I (444) spi_flash: flash io: dio
W (448) i2c: This driver is an old driver, please migrate your application code to adapt `driver/i2c_master.h`
W (459) ADC: legacy driver is deprecated, please migrate to `esp_adc/adc_oneshot.h`
I (467) sleep: Configure to isolate all GPIO pins in sleep state
I (474) sleep: Enable automatic switching of GPIO sleep configuration
I (481) main_task: Started on CPU0
I (491) main_task: Calling app_main()
I (491) bitaxe: Welcome to the bitaxe - hack the planet!
I (541) bitaxe: NVS_CONFIG_ASIC_FREQ 485.000000
I (541) bitaxe: DEVICE: Ultra
I (541) bitaxe: Found Device Model: ultra
I (541) bitaxe: Found Board Version: 204
I (551) bitaxe: ASIC: 1x BM1366 (112 cores)
I (561) SystemModule: Initial overheat_mode value: 0
I (561) SystemModule: I2C initialized successfully
I (571) vcore.c: Set ASIC voltage = 1.200V [0xC6]
I (571) gpio: GPIO[12]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
I (581) pp: pp rom version: e7ae62f
I (591) net80211: net80211 rom version: e7ae62f
I (601) wifi:wifi driver task: 3fcb1130, prio:23, stack:6656, core=0
I (621) wifi:wifi firmware version: 3e0076f
I (621) wifi:wifi certification version: v7.0
I (621) wifi:config NVS flash: enabled
I (621) wifi:config nano formating: disabled
I (621) wifi:Init data frame dynamic rx buffer num: 32
I (621) wifi:Init static rx mgmt buffer num: 5
I (631) wifi:Init management short buffer num: 32
I (631) wifi:Init dynamic tx buffer num: 32
I (641) wifi:Init static tx FG buffer num: 2
I (641) wifi:Init static rx buffer size: 1600
I (651) wifi:Init static rx buffer num: 10
I (651) wifi:Init dynamic rx buffer num: 32
I (651) wifi_init: rx ba win: 6
I (661) wifi_init: tcpip mbox: 32
I (661) wifi_init: udp mbox: 6
I (661) wifi_init: tcp mbox: 6
I (671) wifi_init: tcp tx win: 5760
I (671) wifi_init: tcp rx win: 5760
I (681) wifi_init: tcp mss: 1440
I (681) wifi_init: WiFi IRAM OP enabled
I (691) wifi_init: WiFi RX IRAM OP enabled
I (691) wifi station: ESP_WIFI Access Point On
W (701) wifi:Affected by the ESP-NOW encrypt num, set the max connection num to 10
I (701) wifi station: ESP_WIFI_MODE_STA
I (711) wifi station: wifi_init_sta finished.
I (711) phy_init: phy_version 670,b7bc9b9,Apr 30 2024,10:54:13
I (761) wifi:mode : sta (dc:da:0c:74:c8:e0) + softAP (dc:da:0c:74:c8:e1)
I (761) wifi:enable tsf
I (761) wifi:Total power save buffer number: 16
I (761) wifi:Init max length of beacon: 752/752
I (761) wifi:Init max length of beacon: 752/752
I (771) esp_netif_lwip: DHCP server started on interface WIFI_AP_DEF with IP: 192.168.4.1
I (781) wifi:new:<1,1>, old:<1,1>, ap:<1,1>, sta:<1,0>, prof:1
I (781) wifi:state: init -> auth (b0)
I (791) wifi station: ESP_WIFI setting hostname to: bitaxe
I (791) wifi:state: auth -> assoc (0)
I (801) wifi station: wifi_init_sta finished.
I (801) wifi:state: assoc -> run (10)
I (1081) SystemModule: Existing overheat_mode value: 0
I (1081) SystemModule: OLED init success!
I (1101) http_server: Partition size: total: 2884241, used: 666405
I (1101) http_server: Starting HTTP Server
I (1101) example_dns_redirect_server: Socket created
I (1111) example_dns_redirect_server: Socket bound, port 53
I (1111) example_dns_redirect_server: Waiting for data
I (1131) SystemModule: SYSTEM_task started
I (3811) wifi:state: run -> init (f00)
I (3811) wifi:new:<1,0>, old:<1,1>, ap:<1,1>, sta:<1,0>, prof:1
I (6311) wifi station: Retrying WiFi connection...
I (11641) wifi station: Retrying WiFi connection...
I (11641) wifi:new:<1,1>, old:<1,0>, ap:<1,1>, sta:<1,0>, prof:1
I (11641) wifi:state: init -> auth (b0)
I (11651) wifi:state: auth -> assoc (0)
I (11651) wifi:state: assoc -> run (10)
I (14651) wifi:state: run -> init (f00)
I (14661) wifi:new:<1,0>, old:<1,1>, ap:<1,1>, sta:<1,0>, prof:1
I (17161) wifi station: Retrying WiFi connection...
I (22491) wifi station: Retrying WiFi connection...
I (22491) wifi:new:<1,1>, old:<1,0>, ap:<1,1>, sta:<1,0>, prof:1
I (22491) wifi:state: init -> auth (b0)
I (22491) wifi:state: auth -> assoc (0)
I (22501) wifi:state: assoc -> run (10)
I (25511) wifi:state: run -> init (f00)
I (25511) wifi:new:<1,0>, old:<1,1>, ap:<1,1>, sta:<1,0>, prof:1
I (28011) wifi station: Retrying WiFi connection...
I (33341) wifi station: Retrying WiFi connection...
I (33341) wifi:new:<1,1>, old:<1,0>, ap:<1,1>, sta:<1,0>, prof:1
I (33341) wifi:state: init -> auth (b0)
I (33361) wifi:state: auth -> assoc (0)
I (33371) wifi:state: assoc -> run (10)
I (36371) wifi:state: run -> init (f00)
I (36371) wifi:new:<1,0>, old:<1,1>, ap:<1,1>, sta:<1,0>, prof:1

skot commented 1 month ago

I have a feeling there is a change needed to the hostapd config. I can't quite figure out what though. This unsolved problem seems like a clue. https://esp32.com/viewtopic.php?t=4690

AndySchroder commented 1 month ago

I've also tested with ubuntu 20.04 and I get the same result.

AndySchroder commented 1 month ago

Also, if you want to use nmcli to create a hotspot instead of hostapd, you can use the following steps. nmcli has the advantage that it also sets up IP addresses, DNS, and routing. However, it is harder to get link level (which is where the problem is here) debugging information and control with nmcli compared to hostapd. You likely want to use this nmcli approach after you think you have it working with hostapd.

Create the Hotspot connection:
- sudo nmcli d wifi hotspot ifname wlan0 ssid YourSSIDName password YourPassword
If you have UFW (Uncomplicated FireWall) installed, you will need to allow some things through the firewall in order for it to work.
- Allow DHCP leases
  - sudo ufw allow in on wlan0 to any port 67 proto udp
- Allow DNS
  - sudo ufw allow in on wlan0 from 10.42.0.0/24 to any port 53
- Allow traffic to the internet
  - sudo ufw route allow in on wlan0 from 10.42.0.0/24 out on xxxxxxxxxx
    - xxxxxxxxxx is the name of the network interface that you access the internet through.
- Note: These rules will also allow traffic when the hotspot connection is off and and a WiFi client happens to be connected to a network with address 10.42.0.0/24. However, the routed traffic likely won't get very far because network address translation will be turned off when the hotspot connection is turned off and because 10.42.0.0/24 is not a publicly routable subnet. Also, the DHCP and DNS servers should be turned off if the hotspot connection is turned off. So, this does not seem to be a practical concern. Older versions of ubuntu seemed to setup these firewall rules automatically so this used to not even be something to consider, but in ubuntu 22.04, they seem to need to be explicitly defined.
Now that you have setup the connection, you can use it
- Bring the Hotspot up
  - sudo c u Hotspot
    - Needs to be done on each boot unless you run sudo nmcli c modify Hotspot connection.autoconnect yes.
- Bring the Hotspot down
  - sudo c d Hotspot

AndySchroder commented 1 month ago

Here are some more links related to hostapd usage:

skot commented 1 month ago

I have gotten to the point where the Bitaxe continually prints this debug output;

 (59106312) wifi:new:<11,2>, old:<11,0>, ap:<11,2>, sta:<11,0>, prof:1, snd_ch_cfg:0x0
I (59106312) wifi:state: init -> auth (0xb0)
I (59106322) wifi:state: auth -> assoc (0x0)
I (59106332) wifi:state: assoc -> run (0x10)
I (59109342) wifi:state: run -> init (0xf00)
I (59109342) wifi:new:<11,0>, old:<11,2>, ap:<11,2>, sta:<11,0>, prof:1, snd_ch_cfg:0x0
I (59109342) wifi station: Could not connect to 'test_ap' [rssi -42]: reason 15
I (59111852) wifi station: Retrying WiFi connection...
I (59114682) wifi station: Could not connect to 'test_ap' [rssi -128]: reason 205
I (59117182) wifi station: Retrying WiFi connection...
I (59117192) wifi:new:<11,2>, old:<11,0>, ap:<11,2>, sta:<11,0>, prof:1, snd_ch_cfg:0x0
I (59117192) wifi:state: init -> auth (0xb0)
I (59117192) wifi:state: auth -> assoc (0x0)
I (59117202) wifi:state: assoc -> run (0x10)
I (59120222) wifi:state: run -> init (0xf00)
I (59120222) wifi:new:<11,0>, old:<11,2>, ap:<11,2>, sta:<11,0>, prof:1, snd_ch_cfg:0x0
I (59120222) wifi station: Could not connect to 'test_ap' [rssi -40]: reason 15
I (59122732) wifi station: Retrying WiFi connection...
I (59125562) wifi station: Could not connect to 'test_ap' [rssi -128]: reason 205
I (59128062) wifi station: Retrying WiFi connection...

skot commented 1 month ago

here are the reason codes 15 and 205;

AndySchroder commented 1 month ago

Here is the BitAxe variant I am using by the way:

Model:  BM1368
Uptime: 16 hours
WiFi Status:    Connected!
Free Heap Memory:   153236
Version:    v2.1.8
Board Version:  401

WiFi is connected here to an EnGenius access point (which is not what I want, but it works for now).

AndySchroder commented 1 month ago

One thing to note is linux can connect to the BitAxe as a client of the BitAxe access point during initial setup. However, this issue is about things working the other way around.

Is there any security at all for the BitAxe access point?

I've tried doing no security on the linux access point. I don't think the BitAxe allows that though. Can you make the BitAxe temporarily try to connect to an unsecure linux access point just to see if that works?

skot commented 1 month ago

One thing to note is linux can connect to the BitAxe as a client of the BitAxe access point during initial setup. However, this issue is about things working the other way around.

Is there any security at all for the BitAxe access point?

I've tried doing no security on the linux access point. I don't think the BitAxe allows that though. Can you make the BitAxe temporarily try to connect to an unsecure linux access point just to see if that works?

The Bitaxe setup wifi is open... that is good to know that the RPi STA can connect to the bitaxe AP. Sounds more and more like a security mismatch. I'll try the open RPi STA and see if that works..

AndySchroder commented 1 month ago

Actually, I've only connected from a laptop, not raspi to the BitAxe AP, but this issue happens on rasapi or a laptop, so probably it will work for a raspi to connect to the BitAxe AP.

skot commented 1 month ago

With some modification to the Bitaxe FW to allow it, I can connect to a RPi open AP! (configured with network manager)

here is my Network Manager profile;

[connection]
id=Hotspot
uuid=d57f576d-efc9-4960-b4c9-fbb4deab6372
type=wifi
autoconnect=false
interface-name=wlan0
timestamp=1727196949

[wifi]
mode=ap
ssid=test_ap3

[ipv4]
method=shared

[ipv6]
addr-gen-mode=default
method=ignore

[proxy]

further support of the security mismatch hypothesis

AndySchroder commented 1 month ago

Wondering if the 1$ wifi chip in the BitAxe only supports a security profile that linux has depreciated?

eandersson commented 1 month ago

This is what it looks on the hostap side

wlp2s0: AP-ENABLED
wlp2s0: STA 64:e8:33:47:4c:31 IEEE 802.11: authenticated
wlp2s0: STA 64:e8:33:47:4c:31 IEEE 802.11: associated (aid 1)
wlp2s0: AP-STA-POSSIBLE-PSK-MISMATCH 64:e8:33:47:4c:31
wlp2s0: AP-STA-POSSIBLE-PSK-MISMATCH 64:e8:33:47:4c:31
wlp2s0: AP-STA-POSSIBLE-PSK-MISMATCH 64:e8:33:47:4c:31

skot commented 1 month ago

Wondering if the 1$ wifi chip in the BitAxe only supports a security profile that linux has depreciated?

Surely Linux can handle whatever security -- it does control every other AP that the ESP32 works with.

AndySchroder commented 1 month ago

This is what it looks on the hostap side

wlp2s0: AP-ENABLED
wlp2s0: STA 64:e8:33:47:4c:31 IEEE 802.11: authenticated
wlp2s0: STA 64:e8:33:47:4c:31 IEEE 802.11: associated (aid 1)
wlp2s0: AP-STA-POSSIBLE-PSK-MISMATCH 64:e8:33:47:4c:31
wlp2s0: AP-STA-POSSIBLE-PSK-MISMATCH 64:e8:33:47:4c:31
wlp2s0: AP-STA-POSSIBLE-PSK-MISMATCH 64:e8:33:47:4c:31

Yes, I got that a few times last week when testing with skot, but it was not repeatable. I tried adding the -dd flag to hostapd to get more debug information, but then we never got the above messages to come back up and I had to quit for the day.

skot commented 1 month ago

@AndySchroder can you try setting your AP password to password21. That fixed it for me 😳

AndySchroder commented 1 month ago

You think there is a min password length?

AndySchroder commented 1 month ago

I had a more random password originally, it was autogenerated by nmcli if you skip the password declaration on the command line, but still short.

skot commented 1 month ago

minimum password length for WPA2 is 8.. so password should work. but maybe esp-idf has an off-by-one error?

AndySchroder commented 1 month ago

I was not aware of that constraint in WPA2. The autogenerated password from nmcli did have a length of 8. Will try a 9 and 10 character password later today.

eandersson commented 1 month ago

I found one bug that is very likely related. You cannot change the password from something else to password using the Web UI.

I opened a PR here to fix that limitation. https://github.com/skot/ESP-Miner/pull/347

With this I am able to successfully connect using the above hostapd config.

AndySchroder commented 1 month ago

nmcli d wifi hotspot ifname wlan0 ssid aa password 012345678 works.

AndySchroder commented 1 month ago

nmcli d wifi hotspot ifname wlan0 ssid aa password 01234567 also works.

AndySchroder commented 1 month ago

nmcli d wifi hotspot ifname wlan0 ssid aa password 0123456 gives Error: Invalid 'password': '0123456' is not valid WPA PSK., which matches what skot suggested for WPA2.

AndySchroder commented 1 month ago

I think likely https://github.com/skot/ESP-Miner/pull/347 is the fix. I have re-tested my original password, and it works.

Last week when I first got started, I did not have the ufw rules in place. Those are a new requirement of ubuntu 22.04. They were not required in ubuntu 20.04. I'm not sure if it is a regression or an increase in security. Either way, at some point during troubleshooting I switched to the simple password of "password" and after I figured out the ufw issue, I never thought to try another password again with the BitAxe.

AndySchroder commented 2 weeks ago

Off topic, but I wanted to post here because there is a lot of discussion about connecting to a raspi with a BitAxe in this issue:

The raspi only supports a few (4-5) clients in access point mode before the wifi adapter locks up. This issue suggests you should get a bit more than what I have before locking up, but it is still very limited: https://github.com/raspberrypi/linux/issues/3010 .

So, the raspi is not a useful solution to connect a lot of BitAxe to the internet, only good for testing with a few.

Using another linux machine with a better wifi adapter is not an issue though.

skot / ESP-Miner

won't connect to hotspots generated by linux using hostapd #339