Tools for extracting Channel State Information from formats produced by a range of WiFi hardware/drivers, written in Python with numpy.
Python 3.8+ required.
Documentation updates are incoming.
Don't have your own CSI data? Check out CSI-Data for a collection of public CSI datasets.
CSIKit is a framework aimed at assisting data scientists, researchers, and programmers with performing experiments and tests using CSI-capable WiFi hardware.
While the various public extraction toolkits do include scripts for parsing CSI data from their specific formats, these are largely written for MATLAB. Given the increasing focus on using deep learning in CSI experimentation, Python tools for parsing and processing CSI data may be more desirable for some. This is aimed at improving the accessibility of CSI research for those who may be interested in the area but have little experience with network engineering.
As is usually the case with research-oriented software, documentation is in-progress.
CSIKit provides a command line tool for parsing, processing, converting, and visualisation of CSI data, as well as libraries for use in other Python applications such as those working with Tensorflow, PyTorch, etc.
csikit [OPTIONS] file[.pcap/.dat/.csv/.csi]
CSIKit can be installed via pip or used directly from source.
pip install csikit
csikit log.all_csi.6.7.6.dat
csikit --graph --graph_type all_subcarriers log.all_csi.6.7.6.dat
csikit --csv log.all_csi.6.7.6.dat
Generic example:
from CSIKit.reader import get_reader
my_reader = get_reader("path/to/csi_file.dat/pcap")
csi_data = my_reader.read_file("path/to/my_csi_file.dat/pcap")
Hardware-specific (Intel IWL5300) example:
from CSIKit.reader import IWLBeamformReader
my_reader = IWLBeamformReader()
csi_data = my_reader.read_file("path/to/log.all_csi.6.7.6.dat")
CSIKit exposes key components for use in other Python applications: Reader
and csitools
. As other sections of code become more refined, this will likely grow over time.
Note: This documentation is initial and brief. More will follow shortly.
A Reader is used to read a given CSI capture file and generate parsed traces and matrices. As each file format differs a significant amount, different readers are necessary for each piece of hardware. Using the get_reader
method in CSIKit.reader
, the correct reader can be automatically selected for a given file. Optionally, hardware-specific readers can be imported, as ATHBeamformReader
, IWLBeamformReader
, NEXBeamformReader
, CSVBeamformReader
, PicoScenesBemaformReader
, FeitCSIBeamformReader
.
Once instantiated, a Reader can be used to read a file using the read_file
method. This method returns a CSIData
object, which contain frames (CSIFrame
objects), timestamps, and metadata from parsing (errors, device information, etc). Additional parsing options can be passed to read_file
, including scaled
(bool) to rescale CSI values from manufacturer's internal scaling.
CSIFrame
s will have a different type, based on the type of Reader used to parse them. All CSIFrame
s contain a csi_matrix
(being the most significant part of the payload), however they also contain additional attributes for each frame. Some attributes may be common across the different frame types, however they can vary significantly from one type to another. Next steps for this library are aimed at further integrating these standards. The full list of attributes associated with each frame are listed below.
CSI matrices contain complex numbers representing the gain and phase of a signal at a given subcarrier. These are output in their complex form in CSIFrame
s, however csitools
contains a function for retrieving CSI amplitude from them.
from CSIKit.reader import get_reader
from CSIKit.util import csitools
my_reader = get_reader("path/to/file.pcap")
csi_data = my_reader.read_file("path/to/file.pcap", scaled=True)
csi_matrix, no_frames, no_subcarriers = csitools.get_CSI(csi_data)
The returned tuple contains a modified matrix which contains CSI amplitudes in dBm, followed by the number of frames and subcarriers represented therein.
Once we have CSI amplitude data, we can also apply filters for preprocessing (as seen in many publications making use of CSI).
This example below loads a given CSI file, applies some basic preprocessing, and plots the data in a heatmap.
from CSIKit.filters.passband import lowpass
from CSIKit.filters.statistical import running_mean
from CSIKit.util.filters import hampel
from CSIKit.reader import get_reader
from CSIKit.tools.batch_graph import BatchGraph
from CSIKit.util import csitools
import numpy as np
my_reader = get_reader("/path/to/csi/file")
csi_data = my_reader.read_file("/path/to/csi/file", scaled=True)
csi_matrix, no_frames, no_subcarriers = csitools.get_CSI(csi_data, metric="amplitude")
# CSI matrix is now returned as (no_frames, no_subcarriers, no_rx_ant, no_tx_ant).
# First we'll select the first Rx/Tx antenna pairing.
csi_matrix_first = csi_matrix[:, :, 0, 0]
# Then we'll squeeze it to remove the singleton dimensions.
csi_matrix_squeezed = np.squeeze(csi_matrix_first)
# This example assumes CSI data is sampled at ~100Hz.
# In this example, we apply (sequentially):
# - a lowpass filter to isolate frequencies below 10Hz (order = 5)
# - a hampel filter to reduce high frequency noise (window size = 10, significance = 3)
# - a running mean filter for smoothing (window size = 10)
for x in range(no_frames):
csi_matrix_squeezed[x] = lowpass(csi_matrix_squeezed[x], 10, 100, 5)
csi_matrix_squeezed[x] = hampel(csi_matrix_squeezed[x], 10, 3)
csi_matrix_squeezed[x] = running_mean(csi_matrix_squeezed[x], 10)
BatchGraph.plot_heatmap(csi_matrix_squeezed, csi_data.timestamps)
Reference based on the Atheros CSI Tool User Guide.
IWLCSIFrame visualization can be count at ./docs
This format is based on the modified version of nexmon_csi (credit mzakharo) for BCM43455c0 with support for RSSI and Frame Control. If using the regular version of nexmon_csi
, these fields will not contain this data.
Format based on FeitCSI format documentation
If anything is wrong, let me know. I want to know why, and fix it!
I'm a PhD student working on several sensor data-focussed experiments, a few of which involve using CSI. I'm am by no means an RF engineer or particularly experienced this area. I have done and are doing as much as I can to make sure that anything produced with this is accurate. To that end, there are MATLAB .mat files included in the tests
folder which have been generated using IWLBeamformReader, NEXBeamformReader, and scipy's savemat
method. There are also MATLAB scripts in the scripts
folder which can be used to check the validity of the output from this tool. In my experience I have found these tools to produce identical output to the MATLAB scripts offered by the following developers. If this is not the case, let me know.
Further to that, if there are any assertions I have made within code comments or otherwise which are factually incorrect, again let me know. I want to learn as much about this area as I reasonably can.
The code in this project is licensed under MIT license. If you are using this codebase for any research or other projects, I would greatly appreciate if you could cite this repository or one of my papers.
a) "G. Forbes. CSIKit: Python CSI processing and visualisation tools for commercial off-the-shelf hardware. (2021). https://github.com/Gi-z/CSIKit."
b) "Forbes, G., Massie, S. and Craw, S., 2020, November. WiFi-based Human Activity Recognition using Raspberry Pi. In 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI) (pp. 722-730). IEEE."
@electronic{csikit:gforbes,
author = {Forbes, Glenn},
title = {CSIKit: Python CSI processing and visualisation tools for commercial off-the-shelf hardware.},
url = {https://github.com/Gi-z/CSIKit},
year = {2021}
}
@inproceedings{forbes2020wifi,
title={WiFi-based Human Activity Recognition using Raspberry Pi},
author={Forbes, Glenn and Massie, Stewart and Craw, Susan},
booktitle={2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI)},
pages={722--730},
year={2020},
organization={IEEE}
}