Realtime data visualization

[AsciiPro99]

Liveplot functionality. Requires TKAKK, MATPLOTLIB, Numpy.

[AsciiPro99]

Hope you can see new commit, im new to github.

I implemented a que system holding last 100 entries and plotting them. Your only comment that i did not implement was removing .pop which still is needed to work as far as i can tell.

Can you get this to run on your hardware, for some reason it only ran on one of my machines(windows)?

Something i cannot figure out is why the code is only able to read the 64 subcarriers of the legacy mode https://rfmw.em.keysight.com/wireless/helpfiles/n7617a/mimo_ofdm_signal_structure.htm The lines with 384 len (192 subcarriers) arent working. I can read them fine when i read from csv file with own parser. Do you have any idea why my code only reads the 64 subcarriers?

[AsciiPro99]

fixed variables, changed name and updated readme. Did u see my question about why this is only 64 subcarriers long and not 192?

[StevenMHernandez]

Thanks for this work, I have a few changes I would like to apply to this pull request. Can you edit the pull request and enable the "Allow edits from maintainers" option? Thanks!

[StevenMHernandez]

In regards to the 64 subcarrier question, by default we have the following option (accessible through idf.py menuconfig) https://github.com/StevenMHernandez/ESP32-CSI-Tool/blob/master/active_ap/main/Kconfig.projbuild#L27

This value is used here (https://github.com/StevenMHernandez/ESP32-CSI-Tool/blob/master/_components/csi_component.h#L54).

Setting this to No/False is an option for advanced users so it is disabled by default. Feel free to experiment with it and share your findings.

[AsciiPro99]

Thanks for this work, I have a few changes I would like to apply to this pull request. Can you edit the pull request and enable the "Allow edits from maintainers" option? Thanks!

This should already be on.

In regards to the 64 subcarrier question, by default we have the following option (accessible through idf.py menuconfig) https://github.com/StevenMHernandez/ESP32-CSI-Tool/blob/master/active_ap/main/Kconfig.projbuild#L27

This value is used here (https://github.com/StevenMHernandez/ESP32-CSI-Tool/blob/master/_components/csi_component.h#L54).

Setting this to No/False is an option for advanced users so it is disabled by default. Feel free to experiment with it and share your findings.

Much appreciated. The code should work with 384 byte length.

[AsciiPro99]

Small note the read me uses Linux notation for running commands some places. For instance in: idf.py monitor | python ./python_utils/serial_append_time.py > my-experiment-file.csv

On windows its ../ instead of ./

[AsciiPro99]

After some further testing having the NO/False option enabled breaks the plotting. Numpy converts it to an object instead of 2D array because of difference in list lengths. So it is a requirement at the moment to have it ON/Positive

[StevenMHernandez]

I am still having issue pushing my changes to this branch. I do not have time to figure this out immediately, but I promise I will get back to this and I will get this merged. Thanks again for your work.

On windows its ../ instead of ./

It should be ../ on all systems. Good catch.

So it is a requirement at the moment to have it ON/Positive

Thanks for checking this. I think we could handle both True/False for the different-list-length issue. In the visualization python script, we could remove the need for numpy during plt.plot (instead we would just have native python lists). Alternatively, when we call amplitudes.append and phases.append we can trim the size of the list to a default value (64 subcarriers). However I think this is not necessary for this visualization.

[AsciiPro99]

Thanks for checking this. I think we could handle both True/False for the different-list-length issue. In the visualization python script, we could remove the need for numpy during plt.plot (instead we would just have native python lists). Alternatively, when we call amplitudes.append and phases.append we can trim the size of the list to a default value (64 subcarriers). However I think this is not necessary for this visualization.

I feel stupid saying this but i couldnt get plotting right without converting it to numpy array. Lists within lists is not something i understand. However i am almost sure code would work with native python lists.

I will take a look around to see why you cant edit request.

[AsciiPro99]

Maybe this fixed the issue with not being able to edit. I sync the fork again. But again i might be screwing stuff up with this.

[StevenMHernandez]

Thanks for syncing. I think that could have helped. I also had some weird issues in my local git-repo that required me to restart it.

I tested this new code on macOS, can you test and confirm that the new code works well on Windows too? Thanks.

[StevenMHernandez]

I feel stupid saying this but i couldnt get plotting right without converting it to numpy array. Lists within lists is not something i understand. However i am almost sure code would work with native python lists.

Do not worry, there are always things that we do not know, but we can learn them when we need to. Maybe I did not explain well.

When I say a list of lists, I just mean we could store the amplitude values like this:

perm_amp = [
    [0, 1, 2, 3, 4, 5, 6, 7, 8],
    [0, 1, 2, 3, 4, 5, 6, 7, 8],
    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
    [0, 1, 2, 3, 4, 5, 6, 7, 8],
    ...
]

Notice how perm_amp is a list and perm_amp[0] is also a list? The key thing to notice is that len(perm_amp[2]) > len(perm_amp[0]) which shows that python-lists can handle different-length sub-elements. Numpy on the otherhand assumes that all sub-elements are the same length (which I think throws an error).

[AsciiPro99]

I will test it tomorrow. I know about nested lists, i just find them annoying to deal with!

[AsciiPro99]

Here are my notes for your changes:

For my machine to get it to run i need it to collect a few samples before plotting so this is needed for me. if(count>5): process(line) print("Number of lines read: " + str(count)) carrier_plot(perm_amp)

Also its funny that you corrected my mistake of adding x and y label inside carrier plot but you seem to have moved them back now. So these:

plt.xlabel("Time") plt.ylabel("Amplitude") plt.xlim(0, 100)

need to be moved to plot definition

It now does not crash when Advanced option is enabled, however it would be nice with some functinality to actually be able to plot these. I can work on a solution to this later but for now i think its fine to add this.

[StevenMHernandez]

Also its funny that you corrected my mistake of adding x and y label inside carrier plot but you seem to have moved them back now.

I found that the plt.clf() method clears the values that we set with plt.xlabel. So I added it to every frame. If it is a performance issues, we can evaluate different methods in the future. For now we are OK.

For my machine to get it to run i need it to collect a few samples before plotting so this is needed for me.

I will review this later and I will add back if(count>5):.

Thanks for checking this on windows. Once I make the above change, I will merge the code in.

[StevenMHernandez]

I tried this one more time and I found that the plotting was not real-time. So I have added some updates to the implementation to fix this. The problem was: matplotlib is not able to update at >100Hz. As such, the plotting became further and further behind in time (serial buffer kept growing, but serial_plot_csi_live.py was unable to consume the CSI quick enough).

This new method works on macOS. @AsciiPro99, do you mind testing one more time on your Windows machine? Thanks again.

[AsciiPro99]

Runs on my macihne! 👍

[citysu]

This is my implementation of live plotting. Maybe it can help you. csirealtime_esp32.py