lucask07 / covg_fpga

FPGA and Python experiment code for the digital ion channel amplifier project.
GNU General Public License v3.0
6 stars 2 forks source link

Plotting of DDR data streams is too memory intensive #27

Closed lucask07 closed 2 years ago

lucask07 commented 2 years ago

DDR data streams are very long and normally have a repeated value for many many points. We should create a method to find the unique points in a DDR sequence and plot lines between these rather than all points.

Ajstros commented 2 years ago

@lucask07 I think something like this plot_unique function would work.

import matplotlib.pyplot as plt
import numpy as np
import time

def plt_uniques(data):
    """Plot only the first and last of each group of unique data points to save time and memory.

    Parameters
    ----------
    data : np.ndarray
        Data to plot.
    """

    uniques, indices = np.unique(data, return_index=True)   # Get indices of first appearance of unique values
    indices = np.append(indices, indices - 1)               # Get indices from just before unique values to get first and last of each group of unique values
    indices = np.append(indices, len(data) - 1)             # Add last index so full data is plotted

    indices = np.sort(indices[indices >= 0])                # Sort data for plotting and remove negative indices (-1 from unique at 0)
    unique_data = data[indices]                             # Grab data at indices

    plt.plot(indices, unique_data, linestyle='dotted')      # Plot dotted line so we can see original underneath
    plt.show(block=False)

data = np.array([i // 100 for i in range(1000)])

start = time.time()
plt.plot(data)
plt.show(block=False)
end = time.time()
print(f'Normal plotting: {end - start} seconds')

start = time.time()
plt_uniques(data)
end = time.time()
print(f'Unique plotting: {end - start} seconds')

plt.show()

Here is the output when I run the above code. You can try it on your machine to see the graph as well, the lines match up.

Normal plotting: 0.15996980667114258 seconds
Unique plotting: 0.009996891021728516 seconds

I think we could put this in the utils module of the pyripherals repo. What do you think?

lucask07 commented 2 years ago

@Ajstros Wonderful. Yes, utils module seems like a good place for this. I will let you close this issue once the code is added.

Ajstros commented 2 years ago

Issue created at https://github.com/Ajstros/pyripherals/issues/11. Fixed in https://github.com/Ajstros/pyripherals/pull/12