How to interpret Atheros data

[parthmishra1996]

Hi,

I have collected data using the Atheros-CSI-Tool, I am having issues interpreting it.

from CSIKit.reader import get_reader 
from CSIKit.util import csitools
from CSIKit.util.filters import lowpass, hampel, running_mean
from CSIKit.reader.readers import read_atheros
from CSIKit.csi.frames import ATHCSIFrame 

my_reader = get_reader('data_session2.dat')
csi_data = my_reader.read_file('data_session2.dat')

csi_matrix, no_frames, no_subcarriers = csitools.get_CSI(csi_data)

csi_matrix_trans = np.transpose(csi_matrix)

x = ATHCSIFrame(csi_matrix_trans, csi_matrix)

It throws up an error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-64-c1e4870f44fa> in <module>
     12 csi_matrix_trans = np.transpose(csi_matrix)
     13 
---> 14 x = ATHCSIFrame(csi_matrix_trans, csi_matrix)

/anaconda3/lib/python3.7/site-packages/CSIKit/csi/frames/ath.py in __init__(self, header_data, csi_matrix)
      9     __slots__ = ["timestamp", "csi_length", "tx_channel", "err_info", "noise_floor", "rate", "bandwidth", "num_tones", "nr", "nc", "rssi", "rssi_1", "rssi_2", "rssi_3", "payload_length", "csi_matrix"]
     10     def __init__(self, header_data: namedtuple, csi_matrix: np.array):
---> 11         self.timestamp = header_data.timestamp
     12         self.csi_length = header_data.csi_length
     13         self.tx_channel = header_data.tx_channel

AttributeError: 'numpy.ndarray' object has no attribute 'timestamp'

but when I get the info I suppose it data is correct because:

from CSIKit.tools.get_info import display_info

x = display_info('data_session2.dat')

gives the output:

Hardware: Atheros 802.11n-compatible Bandwidth: 40MHz Antenna Configuration: 2 Rx, 2 Tx Frame Count: 1 Subcarrier Count: 114 Length: 0.00s Average Sample Rate: 0.00Hz CSI Shape: (1, 114, 2, 2)

How can I get the output in form of the ATHCSIFrame(and all the parameters in it)?

[Gi-z]

Hi, in your first codeblock after this line: csi_data = my_reader.read_file('data_session2.dat')

csi_data is a CSIData object, which has a frames attribute. frames is a list of frames in the relevant CSIFrame class, so in this instance they are of ATHCSIFrame.

ath_frames = csi_data.frames This list contains ATHCSIFrames, which have the set of attributes defined in ath.py.

Hope this resolves your issue. Do let me know if you have any other questions.

[parthmishra1996]

Thank you for your reply!

I am using the tool to get data to be used in some machine learning tasks, for it I need sequential data say CSI information packet by packet. As you can see in the output:

Hardware: Atheros 802.11n-compatible
Bandwidth: 40MHz
Antenna Configuration: 2 Rx, 2 Tx
Frame Count: 1
Subcarrier Count: 114
Length: 0.00s
Average Sample Rate: 0.00Hz
CSI Shape: (1, 114, 2, 2)

The frame count is just 1 whereas it should be more, also in the CSI Shape the first dimension is 1, which I think just represents the frame so there is no sequential information. How do I get the CSI information of each packet, what am I missing?

Please help if you can.

Thanks in advance!

[Gi-z]

It is likely that the file is not being fully parsed correctly. The Atheros CSI Tool has a few variations of the .dat file format and I have been struggling to find examples of all of them. If you could provide me with either the file you are currently trying to parse, or another capture from your system, I can work out the issue.

[parthmishra1996]

Sure! this is the file. data_session2.dat.zip

[Gi-z]

Sure! this is the file. data_session2.dat.zip

Thanks for the data sample. I'm having some trouble interpreting the data, using the Atheros CSI Tool. Could you link me to the version of the Atheros CSI Tool you're using?

Strangely enough, it doesn't seem to parse correctly in any tool I've checked (including my own). The supplied MATLAB code for the tool crashes when attempting to parse it, and the values it extracts do not appear to be valid.

[parthmishra1996]

I think I am using the latest version, the last commit in their repo was made in 2018 and I installed the tool a couple of days back, here is the link https://github.com/xieyaxiongfly/Atheros-CSI-Tool. The tool is sending everything correctly. There is one thing I am doing a bit differently, the receiver has the has the modified "4.1.10+" but the sender does not everything else is exactly the same, do you think that can be a problem.

[parthmishra1996]

Hey, I am getting an error of list index out of range, can you help?

[Gi-z]

Apologies, I've been pretty busy this week. I am looking into further Atheros issues and have picked up an Atheros NIC to try and manually capture some data with the same version your specified.

As for this new issue, it seems like the file also isn't being parsed correctly. Is it another Atheros sample? Do you get any output when running csikit datamay25.dat?

[parthmishra1996]

Yup, this is another sample, collected the same way as the previous one.

[Gi-z]

Alright, seems to just be the same issue. The headers for the CSI data are not being parsed correctly. I've looked at the specification in the Atheros CSI Tool manual and while everything appears to be implemented correctly, this clearly isn't working. I'll take a proper look with these files tomorrow and see what I come up with. Thanks for bringing this issue up and I hope to resolve it shortly.

[parthmishra1996]

Thanks for replying!

Do you think I should try MATLAB using the file provided by the Atheros tool to read the data? To be honest, I wanted to stay away from MATLAB as my ultimate objective is to get the data into a nice data frame and use it for deep learning.

I have no idea of where I am going wrong in the data collection pipeline. As you can see even in the previous file the shape of the data was (1, 114, 2, 2) i.e there was just one frame do you have any idea of how to increase the frames because I am clearly sending more than one packet.

[parthmishra1996]

For your reference, this is the file that is having the issue. june_3_csidata.dat.zip

[Gi-z]

I would recommend trying the MATLAB parser for now, if you can get it to work. I had no luck when trying with the sample you posted earlier. You can export the parsed data to a .mat (which you can load into python with scipy's loadmat method).

One frame may be being parsed correctly, however the header is not, which means that the parser is unable to move on to the next frame. I'll take another look tomorrow and get it sorted. :+1:

[Gi-z]

Once again, I'm still having issues getting these files to parse in MATLAB, or to even manually parse the files just reading through the hex. I can't guarantee the files are incorrectly formed, but the csi_len attribute in each packet header reads 0. This indicates that the CSI payload is empty. I've been unable to get these files to parse (with valid CSI) in any tools I've found so far.

The only other thing I can try is setting up the Atheros CSI Tool myself on my own system to investigate the problem and generate my own data to see how it compares. I've just acquired the NIC and I'm waiting for the device to insert it into. Apologies for not being able to offer a quicker solution.

[parthmishra1996]

No problem, thanks for trying!

Do you know any tried and tested way to transfer CSI data, I really need the data to get to the next step of my pipeline. I have the intel NIC cards as well if that might help my cause.

[parthmishra1996]

Also, it would be great if you can let me know of any working configuration you know works. Thanks!

[Gi-z]

No problem, thanks for trying!

Do you know any tried and tested way to transfer CSI data, I really need the data to get to the next step of my pipeline. I have the intel NIC cards as well if that might help my cause.

Despite the older Linux versions needed to support them, I'd recommend setting up one of the Intel NICs as they're the most consistent CSI collection setup possible at the moment.

[Gi-z]

I see you had some help in citysu's repo. Their investigation is quite thorough. Apologies your issue was not resolved. By far the most consistent CSI collection setup you can use is the Intel 5300 NIC. I wish you the best of luck with your next endeavour.