Spectrometer feature

[Vincenzo-Petrolo]

Hi, i am currently working on adding a spectrometer for Labrador, the feature is not yet stable because i have encountered some bugs, but i plan to fix them asap. I opened the PR, just to let you know and receive some feedback or help!

I'm working on Linux, and i think i will only add this feature for Linux Desktop, but you can port it on Windows/Android versions.

Although the channel is a bit noisy, you can see the result that i have obtained in the picture showing the 2 Dirac deltas given by the sinusoidal input function.

I plan to adjust the panel for visualizing it in a better way.

The spectrum gets enabled with a checkbox in the Oscilloscope menu, and it will show the amplitude value of the discrete Fourier transform of the signal.

The build is only supported for AppImage at the moment.

Here's the list of things to do:

• [x] Fix bugs.
• [x] Improve display.
• [x] Porting to other devices

[EspoTek]

Hi mate. First thing's first - that looks amazing! Thank you so much for your contribution. This is something people have been asking about for a long time and I'm sure they'll be happy to see it finally added as a feature.

That said, I do have a possibly-odd-sounding request: please do not fix the .deb (or if you do, please don't get the build server to generate and upload them to the releases page). One of the things I discovered working on this project is that distribution on Linux is fundamentally broken and unstable. Every package manager you support is just one more never-ending stream of busywork that, frankly, achieves nothing.

[Vincenzo-Petrolo]

After this train of commits, this is the result i have obtained: However, as you can notice there are some problems with the frequency mapping to the x-axis, in fact that impulse should be around 6000, but it is not. After some investigation in the code i discovered that it might be caused by the way the buffers for CH1/CH2 are read hence hinting a sinusoid of different frequency...

Do you have any hint?

I managed also to disable scrolling while in spectrum mode because it crashed and it is correlated to the way oscilloscope is managed, so in order to not modify anything in the code i disabled this feature only for spectrum mode.

[EspoTek]

To be honest with you, I'm not an expert in Fourier stuff; I haven't looked at it since I was a student half a decade ago!

That said I'm not sure that CH1 is the right input to the getDFTAmplitude() method. CH1 is basically a vector of processed samples used to plot the Y-axis of the time domain function. The vector will always be of length 1024, regardless of the time window (ie, those 1024 samples could represent anywhere from <1ms to 10s of signal - it's arbitrary!).

Interestingly, I noticed that when leaving the Frequency Spectrum mode, the time window was always around 350ms. Eyeballing your chart, it seems like your peak is at around 0.35 * 6000. Setting the frequency to 17142Hz (6000/0.35) gets me a clear peak at the 6000Hz mark! Definitely some off scaling there.

Another thing I noticed was that you used 65536 frequencies, but only 1024 input samples. This might explain why there's instability when you feed in a low-frequency signal (I can see pulsating at 1kHz) - but again I'm not an expert in this domain.

Let us know how you go!

[Vincenzo-Petrolo]

Hi there! After this commits the frequency spectrum is fully working!

I did the math again, and i discovered i was missing a scaling factor for frequencies! Thank you for hinting that!

Now as you can see in the picture frequencies gets displayed correctly.

But as I am not a man of UI, i managed it the fastest and simplest way i could. Now frequency spectrum is displayed on a range going from 0% to 100% of its maximum amplitude for each channel.

With these last commits, i should have finished the feature, so let me know if there are any bugs in the code!

[Vincenzo-Petrolo]

@EspoTek did you test the spectrometer? Was everything ok? Let me know!

[EspoTek]

It looks like the peak is in the correct spot now, but there's still odd noise in the other frequency bins. Check it out at 1kHz (strange oscillations), and also at 1Hz (occasional insane spikes in the high frequency bins).

[Vincenzo-Petrolo]

Hi Chris hope you're doing well, Thank you for spotting the problem, i didn't notice it and it definitely is related to the number of samples needed to perform a DFT on 'slow' signals.

Supposing we have a 1Hz sinusoidal signal and the sampling rate is 375ksamples/s then in 1 second we obtain around 375000 samples that represents only 1 period of the signal.

In order to compute a DFT, i chose to use 2^17 samples (around 131k samples) for two main reasons:

1. At the beginning is necessary to allocate memory and then allocating more than 2^17 samples slows the startup of the program.
2. At each cycle, it is necessary to compute DFT over N samples, so big number samples will require much more computing power (i'm currently on intel i5 and i hear it screaming when N=2^20), moreover all the UI interactions are slowed down dramatically.

So as you may already have noticed, computing DFT over a truncated signal won't give the correct answer in fact if sinusoidal signal is 1s periodic we get 375k samples but only use around 131k. For 2 Hz we get 2 periods of the sinusoidal signal in the buffer, for 10Hz we get 10 periods and so on...

Hence in order to have the DFT represented correctly it is necessary to have many periods of it inside the buffer and that explains why higher-frequencies are represented correctly...

I removed the 0-100 scaling of y-axis cause in this case it may give wrong feeling of what is being displayed expecially at low frequencies, now each value is represented as is and the scaling is done dynamically basing on some simple logic that you can check in getDFTAmplitude() method.

Eventually, i do not have a solution for this very-low frequency problem currently but on the other hand i think is not a big problem because for really low frequencies one should use expensive equipment for spectrum analysis.

[EspoTek]

Your explanation makes a lot of sense for extremely low frequencies (1Hz range), but it doesn't explain why, for example, 100Hz looks odd. You should be able to capture around 30 full cycles in the buffer at that frequency.

[Vincenzo-Petrolo]

Hi! I made some changes, during those days. Around 100 Hz, the behavior for me is almost fine, except for some times where the peak doesn't show up. I think this is somewhat related to the "slow" signals problem. In the meantime i added support for multi-threaded DFT computation!

[ajoyraman]

I had requested Chris to add the FFT and find that you are working on this. Keep up the good work adding to the fantastic 'Labrador' project. Ajoy Raman www.ajoyraman.in

[ajoyraman]

Please consider using alternate or every third sample of the 375k samples while creating the 131k samples for the FFT of low frequency signals.

[Vincenzo-Petrolo]

Please consider using alternate or every third sample of the 375k samples while creating the 131k samples for the FFT of low frequency signals.

That's a nice idea, but i should add some way to recognize a low freq. signal (for example 10 or 100 samples which are almost the same)

Btw thank you! I will continue my job, after my exams at University get done :)

[EspoTek]

Sounds great, Vincenzo. Really appreciate it. For what it's worth, if you do manage to get this up and running to a proper standard (with the ability to select frequency range and tested for correctness with complex signals), I'd be happy to send you 100 Euro and a letter of recommendation. This is a pretty commonly requested feature, so it only seems fair to put some sort of bug bounty on it.

[ajoyraman]

Adding to my earlier suggestion on FFT of slow signals. As there is no restriction on the buffer length on the PC side the user could display say 5 cycles of the waveform. As you know the timebase setting you can exteact uniform samples less than the 131k limit on your side. You could append zeroes to make up the desired length and the do the FFT. This would work for a at all frequencies as long as more than 5 cycles are displayed. (In practice 5 Cycles gives a decent FFT Spectrum)

[Vincenzo-Petrolo]

Yep, i already perform zero-padding when the buffer of samples cannot fill the arrays used for FFT, but the buffer length on PC is really critical as far as i know, because increasing the buffer length for FFT (which is currently 2^17) would add 2 problems:

1. Slow startup of the program, as it should allocate from the beginning the a large amount of an array (maybe some allocation optimization, or threaded allocation would do the trick)
2. Slow computation of FFT even if multi-threaded

You can try my code And increase the N value to upper powers of 2, and see what happens on your machine

The problem with slow signals i think it's the sample rate (375ksamples/s). I figured 2 solutions:

1. Increasing FFT buffer length to support at least 375ksamples (maybe more for zero-padding slow signals would be better, but slower -> maybe we could use massive multi-threading or 3 plans of 131k each)
2. Adding some logic (as you pointed earlier) to detect that the signal is slow frequency, by comparing for example 100 values and then selecting from the buffer each x samples instead of taking it all (as it would be useless to get duplicated samples)

Thank you for the suggestions, have a good day :-)

[Vincenzo-Petrolo]

Hi @EspoTek , I'm starting to work again for the spectrometer feature, and at the moment i re-implemented it in a much better way! Now as far as i can see we can achieve also 1Hz frequencies on the display.

It should be all good, but i still want to implement some way to visualize only a range of 2kHz and then by sliding with the fingers you can move the 2kHz window left and right. Does it sound good for you?

Also with this improvement i made, we can display signals up to 180kHz, but i have no analog signal generator, nor real oscilloscope to test it at home :-( Could you be so kind to test if the spectrum is correct for me? Report any problem!

Maybe there are still some minor bugs, but i think the spectrum is correctly displayed :-)

[EspoTek]

Thanks so much, man. That looks amazing. I'll definitely be sure to check it out over the coming few days. Just for what it's worth, if you do want to generate some exotic waveforms and see how the DFT performs with them, you can define custom waveforms for Labrador's signal generator: https://github.com/EspoTek/Labrador/wiki/Defining-a-Custom-Waveform

If the real-world spectrum of several odd waveforms matches the output of the fft functions in MATLAB/Octave or Numpy, that would be more than good enough for the patch to be shipped.

[Vincenzo-Petrolo]

I'll try custom waveforms, thank you :-) I'll be waiting for your news then!

[Vincenzo-Petrolo]

Hi @EspoTek I am validating the spectrum of custom waveforms on matlab.

First test

I used two cosine waves at different frequencies, and the spectrum totally agree to it as you can see from the two big delta at 10kHz and 20kHz, the rest is background noise.

Second test

I used an "Exponentially decaying" signal, and the output from the spectrum is correct, it should be F = 1/(1+jw) and it is:

Third test

I used a heaviside step function, and the output is correct again

As you can see, the output of the spectrum is quite accurate, of course it cannot respect the theoretic transforms but it correctly displays everything. Also, after the improvements i made, the continuos computing of DFTs in background is almost imperceptible which is a great improvement.

[EspoTek]

Awesome work. I've just patched it to work on Windows, and also only re-calculate the spectrum on every fourth frame (prevents slowdown).

The only thing left to do is check compatibility on OSX/32-bit Windows and add in some UI for setting the min/max frequency (I can do both of these tasks).

Please shoot us an email. admin@espotek.com :)

[Vincenzo-Petrolo]

@EspoTek I wrote you an email from vincenzo@kernel-space.org, please also check in the spam folder as i have some problems with the mail server :-)

[ModestMC]

I just wanted to say that first, this is a really useful feature, and second, reading the thread and watching how this feature evolved made my day. I ordered my first Labrador yesterday, when it finally shows I look forward to trying this out. Hats off to both of you.

[EspoTek]

And just to let you know, this is coming. I've been trying to get my Hackintosh working over the past couple of weeks so I can get it up and running on Mac. Easier said than done with a Ryzen...

[ModestMC]

My heart goes out to you, we wound up putting kubuntu on a box for that reason. Does the machine have a graphics card?

As a side note, I was reading the specs for the labrador (trying to see if it will work for some of the EE labs at my uni) and the arbitrary function generator says it can dump out 1 megasample per second. How high of frequencies can the board able to accurately dump out?

When I get the board I can see about testing out the windows build on my machine, I've been using vcvrack and an aux cable as a lazy way to get dummy data with multiple frequency components.

[EspoTek]

Yep, I've got an R9 280x (which I found a week ago in hard rubbish of all places, alongside an i7 2600k!!!)

I don't think it's the GPU that's getting me but some misapplied AMD Kernel patches. They do not seem to mix as smoothly as I'd like. I might have to try on my wife's 3570k...

Regarding accuracy of the signal generator, it really depends on how smooth you need it for be. For education purposes, anything up to 15kHz is more than usable (you get 64 samples per cycle). Above that and it will start to degrade. Anything over 30kHz will be rough.

If you have any other questions, it's probably best to start a new thread or shoot us an email at admin@espotek.com. We're starting to get a bit off-topic here!

[ModestMC]

If you wind up getting it all to work, I will have to try doing so myself. When we got the box GPU integration was iffy.

I'll follow up with you via email for more details on the signal generator. I was considering an email anyways but I realized that if I trawled through this thread looking for said information, I won't be the only one. Now the info is there for them.

The Labrador is supposed to arrive on the 2nd of august, so in about a week I'll be back with some initial results.

[EspoTek]

Couldn't end up getting the Hackintosh to work, but did find a MacBook on Facebook Marketplace for A$200 (125 EUR, 145 USD). The guy advertised it as having a broken screen, so the original plan was to use it as a ghetto Mac Mini.

However the screen seems to work just fine apart from some banding in the corners... Was the 2015 model with 256GB SSD, 8GB of RAM and i5 processor, too! I am definitely OK with this.

Have written the latest patch, software compiles and launches but I haven't tested it for functionality yet. That's the plan for this weekend, then x86 Windows libraries, then getting the whole thing building on the CI server!

[EspoTek]

This is now merged!