search

espressif / esp-idf

Espressif IoT Development Framework. Official development framework for Espressif SoCs.
Apache License 2.0
13.63k stars 7.28k forks source link

Some packets cannot be captured in sniffer mode (IDFGH-3410) #5375

Closed wgcod closed 3 years ago

wgcod commented 4 years ago

Now I'm using the sniffer mode, and it works very well, but I find that some packages can't be obtained, which affects the use. WiFi module is AP6212A, MIMO is not supported. I want to know why, if there is an unsupported protocol, and how I can configure it to capture it. Thank you very much.

zhangyanjiaoesp commented 4 years ago

@wgcod which kind of packages do you can't obtain ? You can set the filter according to https://github.com/espressif/esp-idf/blob/7d75213674b4572f90c68162ad6fe9b16dae65ad/components/esp_wifi/include/esp_wifi_types.h#L376

wgcod commented 4 years ago

What can't be captured is the data type package (type 2:8). I haven't set any filtering, and the data type package of other devices can be captured, just the AP6212A in my test. But beacon frames and so on are OK. At the same time, I am sure that there are datatype packets because I can capture them with other sniffer tools. Although no valid packets are captured, the callback function also returns some information, as shown in the following figure image WIFI_ PKT_MISC type, which may be MIMO or other, but I found that ap6212a does not support MIMO, so can you tell what other types of packages esp32 does not support? Thank you very much.

zhangyanjiaoesp commented 4 years ago

@wgcod

  1. WIFI_ PKT_MISC type is MIMO
  2. The ESP32 can capture all kind of data type packets.
  3. The ESP32 won't capture CTRL packets by default if you don't set any filter. If you want to capture CTRL packets, you need to call esp_wifi_set_promiscuous_ctrl_filter() set the CTRL filter. For CTRL packets, currently we support the following types: image
wgcod commented 4 years ago

But it's true that AP6212A's data type packets(type 2:8) can't be captured, none of them can be captured, but at the same time, another chip (MT7612) can capture all of them. Previously, it was thought to be a MIMO problem, but now it is definitely not, nor is it a 20M / 40M problem. In description WiFi_PKT_MISC is other type such as MIMO etc. does 'etc' mean there are other unsupported types?

zhangyanjiaoesp commented 4 years ago

Can you provide a snapshot about the AP6212A's data type ?

wgcod commented 4 years ago

image This is a package captured by OmniPeek and mt7612. Esp32 cannot get encrypted data in the figure image

liuzfesp commented 4 years ago

HI @wgcod could you provide the brand type of the AP6212A? Maybe you can provide a snapshot of it.

wgcod commented 4 years ago

I know that the BCM43438A1 chip is used, which does not support HT40 and MIMO,is a relatively low-end chip,so I can't guess why it can't capture its data type packets.

liuzfesp commented 4 years ago

Hi @wgcod, that's really strange, could you double check that ESP device and the AP is in the same channel?

Alvin1Zhang commented 4 years ago

@wgcod Thanks for reporting, would you please help share if any updates for this issue? Thanks.

wgcod commented 4 years ago

Yes, I am very sure that the channels are the same. After all, it has been looking for this problem for so long that it will not be some low-level errors. Now we have some professional teams, but we still can't find the problem. You are more professional. Can you give some advice?

zhangyanjiaoesp commented 3 years ago

@wgcod

  1. Does ESP32 can capture the beacon packet of AP6212A?
  2. Does ESP32 can capture the unencrypted data packets? Can ESP32 get the data len when capture the encryped data packets?

I want to reproduce your problem locally. Do you use the AP6212A directly? Or do you know any product that use AP6212A?

wgcod commented 3 years ago

1 ESP32 can capture the beacon packet of AP6212A! 2 Can! Can! I used a product with AP6212A, which can be bought on https://detail.tmall.com/item.htm?id=604908576541&skuId=4412993839006 . Select B. Or I can express one directly and return it after use.

zhangyanjiaoesp commented 3 years ago

@wgcod Sorry for late reply.

I have bought the product with AP6212A you provided. And I use the simple_sniffers example to capture the encryped data packets, I can capture it. Here is my reproduce info:

  1. IDF commit: master (b0150615)
  2. Router: TPLINK Archer C2 ( mac address AC:84:C6:7E:31:2B )
  3. Phone: Honor 4x ( mac address 94:FE:22:49:E6:F9 )
  4. Goov (mac address AC:64:CF:EC:D3:7B )
  5. 2428_pkt_1 is the packets captured by Omnipeek
  6. sniffer-example is the packets captured by ESP32 simple_sniffers example

2428_pkt_1.zip sniffer-example.zip

wgcod commented 3 years ago

Our test is to use AP mode, goov as AP, and connect this AP with a relatively new mobile phone, such as mate30 or iPhone 6 or above. This probably can't be captured. Indeed, some old mobile phones or routers, or the low-speed protocol (802.11b) used when the distance is too far, can be caught. So can you use goov as an AP and try again with a newer mobile phone?

zhangyanjiaoesp commented 3 years ago

Ok, I will try to test again.

zhangyanjiaoesp commented 3 years ago

@wgcod

I have used the Goov as Ap, and iPone6 to test, I can capture the packets also.

sniffer-1217.zip

wgcod commented 3 years ago

Is there an update on this issue in the recently updated IDF? I'll test it again. At the same time, can you test with multiple different phones. And from different times, such as morning, afternoon, evening. Because it's really affected by the phone model and the environment. But it's very easy to reproduce here. We also use some of the market's packet capture tools, also can not capture data packets. So it can't be unreasonable or a simple mistake. Could you please try again

wgcod commented 3 years ago

Did you reproduce the question? Some progress?

zhangyanjiaoesp commented 3 years ago

I didn't reproduce this question. Have you tested using the latest IDF?

wgcod commented 3 years ago

Sorry for the late reply, I downloaded the latest idf and tested it. Still have this problem. I use iphone6 to connect to Goov, and I cannot capture data type packets, only broadcast packets (Type:0 Subtype:8) But if I connect with another mobile phone (nubia NX529J), the data type packet (Type: 2 Subtype: 8) can be captured. It is still very easy to reproduce here. There is another phenomenon, if I use Iphone6, the data type packet cannot be captured. But it can be clearly seen that the number of WIFI_PKT_MISC types of packets has increased. It can be determined that the data is related to the WIFI_PKT_MISC type packet. I can’t guess why you can’t reproduce it there. It’s GOOV-C6?

zhangyanjiaoesp commented 3 years ago

Of course, it's GOOV-C6, I bought the product with AP6212A you provided. By the way, do you use the simple_sniffer example to test? Or you sniffer using your own code, maybe there are some different between your code and the example code.

wgcod commented 3 years ago

I did use the simple_sniffer example, but did not write to the SD card, only output the captured camera data packet in sniffer_task. At the same time, I can also see the data packet more intuitively. We also found some paid professional companies that do wifi packet capture. They reproduced the problem, but did not find the cause. Can you try a few more times?

zhangyanjiaoesp commented 3 years ago

We also found some paid professional companies that do wifi packet capture. They reproduced the problem, but did not find the cause.

You mean that there is other packet capture software can't capture the data packets from AP6212A also? If so, there is something wrong with the packet that the AP6212A chip is sending out.

wgcod commented 3 years ago

Yes, MT7688 can't capture, but MT7612 can at the same time. There are screenshots captured by MT7612 above. See that each package is a complete package, and the video on the mobile phone is very smooth. Is it possible that there is something wrong in the data packet?

zhangyanjiaoesp commented 3 years ago

@wgcod Please provide the package you captured by OmniPeek and mt7612 to us, we will try to analysis the data packets.

And please provide more info about your test, I will try to reproduce it locally again.

  1. IDF commit you used
  2. sniffer example you used, since you said you didn't write to SD card, you may have changed the example code, so send it to me to test.
  3. My iphone 6's version is 12.1.4, which version do you use ?
  4. which ESP32 board you used? I use the ESP-WROVER-KIT
  5. Have you tried to use another ESP32 device to test ? If not, please have a try.
  6. Do you want to use ESP32 as a capture card ? Or you just want to know why the data packets can't be captured ?
wgcod commented 3 years ago

data.zip This is a data packet captured by OmniPeek and mt7612, C0:84:XX is Goov, and C8:85:XX is a mobile phone. At this time, the data type packet cannot be captured, but others can. Here is the answer to the question: 1 Tested the latest Idf and many previous 2 "cmd_sniffer.c" is the code I used, 4 changes have been made, can compare it 3 my iphone6 version is 12.5.1 4 I use ESP32-WROVER-B now, and I have tested other 5 Yes, I have used some ESP32, but they are all ESP32. Haven't used esp32s2 or other 6 We want to make a product and use this solution Thanks

zhangyanjiaoesp commented 3 years ago

Tested the latest Idf and many previous

This need to be a specific branch, such as 73db142

We want to make a product and use this solution

If you want to use ESP32 as a capture card, it's not recommended. Because there are some packets that ESP32 can't receive or parse.

zhangyanjiaoesp commented 3 years ago

hi @wgcod , I have analyse the date packets you provided, and I found that the data have an aggregated flag set, but my data packets didn't have this flag set. data_aggregated

So I added some logs when receiving AMPDU, you can add the following code in the sniffer example, then when sniffer --stop is called, the logs will show. code_change . To test with the log version, the IDF commit is 8b75cbf99f02a4b2e9a3bc498c20941c50c7c337, the wifi lib change is here: wifi_lib.zip, and the wifi firmware version: d5bc652. Note: suggest you test in the shielding box, then the packets is only related to the test device. Since you only care the data packets, you can set filter when run sniffer sniffer -f sniffer-example -F data|misc -i wlan -c 6

The log is like this: log info

And I have tested with iphone12 and Mate20, ESP32 still can receive the data from Mate20 and iphone12.

wgcod commented 3 years ago

OK, I'll test it Soon

Alvin1Zhang commented 3 years ago

Thanks for reporting, will close due to short of feedback, feel free to reopen with more updates, thanks.

wgcod commented 3 years ago

Sorry, I have tried many times to build the environment you provide, but there are all kinds of mistakes. Now I use the latest IDF and the .a file you provide. It can be compiled, but it still doesn't work. The problem still exists in IDF version 4.3.1. I use honor 30s as AP and iphone12 as STA, which can be simply reproduced. The beacon package can capture, but the data type package cannot. The data type package captured by another tool still has the aggregated flag. As shown in the screenshot above. So I may also complete the operation you provided. Is there a simpler method, such as using IDF version 4.3.1