Subcarriers Problem !!! Abnormally High Energy Values in CSI Analysis Across Multiple Bandwidths

[kuku000]

Hello everyone,

I’m currently analyzing Channel State Information (CSI) for 20 MHz, 40 MHz, and 80 MHz bandwidths. I have removed the NULL and PILOT subcarriers from my analysis, but I still observe abnormally high energy values. These values remain unchanged across multiple packets, which is puzzling.

What could be the reason for these consistently high energy values? Should I consider removing these values from my analysis, or is there a better approach to handle them? I appreciate any insights or suggestions you might have! Thank you!

NULL and Pilot

20MHz

Before del NULL and Pilot:

After del NULL and Pilot:

40MHz

Before

After

[jlinktu]

As I don't know what systems you are using nor how you process CSI values I can only guess. So here is my guess: You are missing an FFT-shift and therefore remove the wrong subcarriers.

[kuku000]

Thank you very much, I'm using raspberry pi 3b+,and I just only use CSIKIT (python module) to extract CSI data.

[kuku000]

As I don't know what systems you are using nor how you process CSI values I can only guess. So here is my guess: You are missing an FFT-shift and therefore remove the wrong subcarriers.

csi processing

from CSIKit.reader import get_reader
from CSIKit.util import csitools
import numpy as np

class Csi_Reader():
    def __init__(self):
        pass

    def read(self, path):
        try:
            my_reader = get_reader(path)
            csi_data = my_reader.read_file(path)
            print(len(csi_data.frames))

        except Exception as err:
            print(f"Unexpected {err=}, {type(err)=}")
            raise

        return csi_data

    def get_csi_matrix(self, path, csi_type = "amplitude"):
        csi_data = self.read(path)
        if csi_type == "original":
            csi_matrix, no_frames, no_subcarriers = csitools.get_CSI(csi_data, "original")
        elif csi_type == "amplitude":
            csi_matrix, no_frames, no_subcarriers = csitools.get_CSI(csi_data)
        elif csi_type == "phase":
            csi_matrix, no_frames, no_subcarriers = csitools.get_CSI(csi_data, "phase")

        return csi_matrix, no_frames, no_subcarriers

def csi_plot(csi_matrix, no_frames, no_subcarriers):
    csi_matrix = csi_matrix.reshape((no_frames, no_subcarriers))
    csi_matrix_real = abs(csi_matrix.real)
    #csi_matrix_real = csi_energy_in_db(csi_matrix)
    print(csi_matrix[0])
    print(csi_matrix.shape)
    plt.figure()
    for i in range(no_frames):
        plt.plot(csi_matrix_real[i])
        plt.show()

def csi_csv(csi_matrix, no_frames, no_subcarriers, path):
    csi_matrix = csi_matrix.reshape((no_frames, no_subcarriers))
    print(csi_matrix[0])
    df = pd.DataFrame(csi_matrix)
    df.to_excel(path, index=False, header=False, float_format="%.10f")
    print(f"CSI data saved to {path}")

def remove_null_and_pilot(csi_matrix, no_frames, no_subcarriers):
    if no_subcarriers == 64:
        bandwidth = "20MHz"
    elif no_subcarriers == 128:
        bandwidth = "40MHz"
    elif no_subcarriers == 256:
        bandwidth = "80MHz"

    null, pilot = get_subcarrier_exclusions(bandwidth)
    excluded_subcarriers = set(null + pilot)
    vaild_csi = []
    valid_subcarriers = [j for j in range(no_subcarriers) if j not in excluded_subcarriers]
    for i in range(no_frames):
         csi_vaild = csi_matrix[i][valid_subcarriers]
         vaild_csi.append(csi_vaild)

    return np.array(vaild_csi)

def get_subcarrier_exclusions(bandwidth):
    subcarrier_exclusions = {
        '20MHz': {
            'null': [0, 1, 2, 3, 32, 61, 62, 63],
            'pilot': [11, 25, 39, 53],
            'count': 52
        },
        '40MHz': {
            'null': [x+64 for x in [-64, -63, -62, -61, -60, -59, -1, 63,  62,  61,  60,  59,  1,  0]],
            'pilot': [75, 89, 117, 53, 39, 11],
            'count': 108
        },
        '80MHz': {
            'null': [0, 1, 2, 3, 4, 5, 127, 128, 129, 251, 252, 253, 254, 255],
            'pilot': [25, 53, 89, 117, 139, 167, 203, 231],
            'count':234
        }
    }
    if bandwidth in subcarrier_exclusions:
        return subcarrier_exclusions[bandwidth]['null'], subcarrier_exclusions[bandwidth]['pilot']
    else:
        raise ValueError("Unsupported bandwidth. Choose from '20MHz', '40MHz', or '80MHz'.")

def csi_energy_in_db(csi_matrix):
    csi_energy = np.abs(csi_matrix)**2
    energy_db = 10 * np.log10(csi_energy)
    return energy_db

csi_matrix, no_frames, no_subcarriers = Csi_Reader().get_csi_matrix(r"C:\Users\keng-tse\Desktop\nexmon_csi-master\utils\matlab\test10hz.pcap", "original")
csi_matrix = np.fft.fftshift(csi_matrix, axes=1)
print(csi_matrix.shape)
print(no_frames)

vaild_csi = remove_null_and_pilot(csi_matrix, no_frames, no_subcarriers)
csi_plot(vaild_csi, no_frames, 52)

Before I add the FFT-shift 20MHz (it has already been removed null_and_pilot):

After i add the FFT-shift:

It seem also have an abnormal high values.

40MHz

It's look normal after FFT:

[jlinktu]

CSIKIT is not made by us, can't give you any support with that. But you can pass me the two CSI-UDP pcap files used in your last post and I can have a look.

[kuku000]

pcap.zip 1023pc_0 is 20MHz and I just plot the first pcaket for these two pcap Thank you !!!

[jlinktu]

The 20MHz capture probably contains CSI of IEEE 802.11a/g frames (legacy frames), which have a valid subcarrier range of –26 to –1, +1 to +26. The subcarrier range you are currently using is –28 to –1, +1 to +28, which is only valid for IEEE 802.11n/802.11ac 20 MHz frames. Therefore you might want to change

'null': [0, 1, 2, 3, 32, 61, 62, 63],

to

'null': [0, 1, 2, 3, 4, 5, 32, 59, 60, 61, 62, 63],

Regarding 40MHz, for some reason, subcarrier 2 seems to carry an invalid value, therefore you might want to use:

'null': [0, 1, 2, 3, 4, 5, 63, 64, 65, 66, 123, 124, 125, 126, 127],

However, be aware that in your 40MHz capture you might encounter CSI of 20MHz frames in the sub-bands of the 40MHz bandwidth as well.

[kuku000]

The 20MHz capture probably contains CSI of IEEE 802.11a/g frames (legacy frames), which have a valid subcarrier range of –26 to –1, +1 to +26. The subcarrier range you are currently using is –28 to –1, +1 to +28, which is only valid for IEEE 802.11n/802.11ac 20 MHz frames. Therefore you might want to change

'null': [0, 1, 2, 3, 32, 61, 62, 63],

to

'null': [0, 1, 2, 3, 4, 5, 32, 59, 60, 61, 62, 63],

Regarding 40MHz, for some reason, subcarrier 2 seems to carry an invalid value, therefore you might want to use:

'null': [0, 1, 2, 3, 4, 5, 63, 64, 65, 66, 123, 124, 125, 126, 127],

However, be aware that in your 40MHz capture you might encounter CSI of 20MHz frames in the sub-bands of the 40MHz bandwidth as well.

Thank you for your response and assistance!!! And, I still have some question: (1) The reason that the 20MHz (2.4GHz) packets include 802.11a/g frames is because they are management frames, and I am extracting packets from the smartphone.

(2) If I set the AP to a specific band, channel, and bandwidth, would I still encounter the issue of 40 MHz captures possibly containing 20 MHz information or 80 MHz captures possibly containing 40 or 20 MHz information?

[jlinktu]

(1) The reason that the 20MHz (2.4GHz) packets include 802.11a/g frames is because they are management frames, and I am extracting packets from the smartphone.

Yes, that is a possibility.

(2) If I set the AP to a specific band, channel, and bandwidth, would I still encounter the issue of 40 MHz captures possibly containing 20 MHz information or 80 MHz captures possibly containing 40 or 20 MHz information?

This can still happen, especially as control frames are always sent on one of the 20MHz subbands (also called control channel) of the 40 or 80MHz main channels.

[kuku000]

(1) The reason that the 20MHz (2.4GHz) packets include 802.11a/g frames is because they are management frames, and I am extracting packets from the smartphone.

Yes, that is a possibility.

(2) If I set the AP to a specific band, channel, and bandwidth, would I still encounter the issue of 40 MHz captures possibly containing 20 MHz information or 80 MHz captures possibly containing 40 or 20 MHz information?

This can still happen, especially as control frames are always sent on one of the 20MHz subbands (also called control channel) of the 40 or 80MHz main channels.

(2)-> Can I set makecsiparams -b 0x00, 0x80 to avoid control frames? or other setting

[kuku000]

Sorry, I have one more question: @jlinktu Are these packets control frames?

Excuse me for interrupting!!!

[jlinktu]

Possibly. But I can't tell a frame's type from a magnitude plot of CSI as the frame type is not encoded there.

[kuku000]

Thx