Building Clean task

[agoodmanjerry]

• [x] Prepare the O3 data replay to test clean. Folder of low-latency data replay: /home/chiajui.chou/ll_data_O4_review
• [x] Make sure we load the correct weights of trained model and X_scalar.
• [x] Make sure we can update the 3-second online inference dataset correctly.
• [x] Validate the cleaned results.
• [ ] Plotting module
• [ ] Should we make sample_rate as a tunable parameter in train?

Data on CIT: /home/chiajui.chou/dc-demo/data Trained model: /home/chiajui.chou/dc-demo/results/test-O3-H1/lightning_logs/version_4

[saleem-muhammed]

Regarding plotting module

1. Within the projects/clean module, the data of most recent 3 s (and 1 s of their cleaned version) will be in memory. In order to make an ASD plot, we need a longer segment as we usually do at a rate > 1/4 Hz. If we transfer the data to slightly longer Buffers having 4 s duration, and one each for cleaned and unclean data, then we can produce the metric (ASDR) without having to load the data again. However, this will have to be parallelized otherwise, will increase the latency of cleaning.
2. Alternatively, we can have a different module projects/plotting with functions for plotting. This will run completely independently from projects/clean and will load the data independently from the original and cleaned directories.

The second approach will be easier for implementation and will be parallel by construction.

What is it that we want to plot? Make plots of ASD ratio vs frequency for every 4096 s for example? Or we compute bin-averaged ASDR from every 4s segment and plot them against GPS time? The latter one is a necessary thing to plot but do we also need the ASDR vs frequency?

@agoodmanjerry what do you think?