Simple Q-Learning / Classification Model for Signal Detection

[jasoninirio]

This will be a precursor to the machine learning model we will use for detecting jammers and jammed signals. For now, it will consist on a simple "on" or "off" sequence where the ML model will learn the difference between idle connection, when there is bidirectional ping traffic, and dead connection.

Although we can simply find this by using the srsGUI (as Stefan mentioned in the Constant/Barrage Jammer Detecter), we should utilize that issue and expand onto it with Q-learning. @yixiuzhu and I are experienced with Q-learning and can lead this task.

A way this would work is by using the modified srsRAN code we are working on to gather the PUSCH and PUCCH data. After so, we would preprocess the data to label each segment to match one of the states mentioned.

. . .
enb_pusch: 0 + 0i
enb_pusch: 0 + 0i
enb_pusch: 0 + 0i
enb_pusch: 0 + 0i
enb_pusch: 0 + 0i
enb_pusch: 0 + 0i
enb_pusch: -0.682389 + -0.72735i
enb_pusch: 0.744513 + -0.673355i
enb_pusch: 0.724487 + -0.676256i
enb_pusch: -0.680058 + -0.712481i
enb_pusch: 0.6545 + 0.739286i
enb_pusch: -0.695095 + -0.727533i
enb_pusch: 0.722327 + -0.700396i
enb_pusch: -0.722916 + 0.698978i
enb_pusch: -0.743311 + 0.690411i
. . .

Looking at the data from SampleSrsENBGraphData.txt, we can see that enb_pusch differs by some margin depending on the state it is in. So, how do we know what segment of the data is in what state? Controlling the graph's data output would help with this. We refer back to the ticket that Stefan assigned us, where we implement a jammer detector and try to recreate each different state individually then assign the data to a file with the state as a label.

TODO:

• Gain a deeper understanding of the srsRAN sourcecode and work with the srsRAN Mod team (@MichaelJAliberti and @jullianzz )
• Create a simple Q-learning model that can detect one of the states
• Expand on model to detect the rest of the states
• Use model as a foundation for the jamming detector model, then eventually misbehavior detection.

CONCERNS:

• Research if Q-learning is the most effective for this type of working. Since this is mostly classification and not detection, maybe we might want to use something like k-nearest neighbor or multi-label classification. It does not have to be a fully fledge algorithm, just something to classify the signals we decoded using srsRAN.
• Knowing what signal segment is doing what is crucial, there will be margin of human error for determining what's going on. Maybe we can have a way to manually select segments that we know for certain are in the determined state.

[gefa]

Thanks for creating the ticket @jasoninirio ! Let me add some of my thoughts to hopefully refine the action items.

Our ultimate goal is misbehavior detection with regards to resource usage scheduling. One of the scheduling algorithms for c-v2x is semi-persistent scheduling (SPS) algorithm which is studied in [1] whose cv2x_traffic_generator we plan to use to generate traffic. Misbehavior detection is kind of binary: either traffic is well behaved or not. Not well behaved case could be broken down into several different classes or types of jammers. Let's consider detecting single jammer type, mainly narrowband frequency hopping one describred in #6.

BACKGROUND There are several data abstractions in the LTE PHY layer, e.g. packets/frame, subframes, subchannels, resource blocks, bits, symbols, signals. Packets we see in Wireshark (or your application layer) may be made of number of resource blocks. To make things confusing, a resource block is further split into subcarriers and (OFDM) symbols (see [2]). This means that a resource block is a IQ data stream in itself, and can be represented on a constellation diagram (see [3]). (Question for @jullianzz and @MichaelJAliberti: Can we extract the iq signal / constellation of a resource block in particular not entire or random part of a packet)?

In [1] Eckerman et.al shows example grid of 100 resources. Note in this resource grid, columns are subchannels (frequency) and rows are subframes (time). Can you find the number of resource blocks per each resource grid (or pool?) used by cv2x traffic generator? Sidelink channel bandwidth used in [1] is 10MHz. It is divided in 1xx KHz subchannels. Number of subchannels = ? ... Subframes ... BTW, this configuration should match the receive end (pssch_ue).

DATA CONDITIONING / INPUT TO ML Let's assume for now each frame / resource grid has 100 resource blokcs (RBs). This parameter can be changed later. Now we don't want to forensically extract analog signal features but use discrete information: Is a given RB in use or not? If RB's signal shows proper constellation (e.g. 4 distinct sample groups for QPSK) then it's in use, and vice versa. Hence, you can represent usage of resource grid with a matrix of booleans (or ints with 1s and 0s maybe?). Notice that if we define the input to ML model to be matrix of boolean-s it resembles hand-written character classification. Except we weal with black and white "pixels", not gray scale.

SANITY CHECK How should this work intuitively? Assume following about jammer:

1. narrow band jammer completely blocks only single RB at a time (single point in the "image").
2. when RB is jammed it has no proper constellation, hence we detect it as unused (cant tell apart from noise).
3. jammer can move by only one resource block (up, down, left, or right) between two resource grid instances. I.e. assume frequency changes continuously over time, and doesn't jump around. Then trying to spot the narrow band jammer in a resource grid in real time is like tracking a pacman (that disappears some of the time, because we assume we can't tell apart jamming from noise) If you observe a single resource grid you just see a single snapshot of resource usage in time. Therefore, you may want to serialize multiple resource grids into a long array and feed that into ML model.

TESTING THE MODEL To test this idea, please craft a few synthetic examples:

1. list of matrices, each having random RB usage allocation (1s) --- > label as normal behaviour
2. list of matrices each having random RB allocation + use a jammer to sort of erase RB allocation (1s to 0s) in a random walk fashion. ---> label as mischievous behavior Train with training examples, then test on new examples. To start with, it may be easier to detect misbehavior when all RBs are in use (yellow in figure above). My suggestion is to try [4] or alternative toy model. You just need to slightly modify it for our purpose.

It would be helpful if you all come to the meeting tomorrow so that we can agree who is doing what.

[1] Eckermann, Fabian, and Christian Wietfeld. "SDR-based open-source C-V2X traffic generator for stress testing vehicular communication." 2021 IEEE 93rd Vehicular Technology Conference (VTC2021-Spring). IEEE, 2021. Available at: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9449043 [2] https://rfmw.em.keysight.com/wireless/helpfiles/89600b/webhelp/subsystems/lte/content/lte_overview.htm [3] https://twitter.com/ea4gpz/status/1487916526149804033/photo/1 [4] https://pythonprogramming.net/introduction-deep-learning-python-tensorflow-keras/

[gefa]

I was inpatient and added just a skeleton for artificial test case generator: https://github.com/C-V2X-Senior-Design/CV2X_MachineLearning/blob/main/sim.py But it's incomplete, see TODOs and assumptions. We should start simple and add more complexities to the model later.

[jasoninirio]

Thank you Stefan! This is exactly the direction I was thinking when implementing the preprocessing code for the data, I will work on it this weekend.