Upsampling/Oversampling for PhaseScope/PrettyScope

[elanhickler]

No idea what this feature involves, but you said you could add a feature to remove unrealistic jagged lines, sharp corners, etc. As seen here: https://www.youtube.com/watch?v=SvCfz-TY6Go

I will gladly pay for your time on this, I'd like to have it as soon as possible, assuming it's not a big investment.

[elanhickler]

class SmootherManager
{
public:
    SmootherManager() {}
    juce::Array<smoothedParameter *> CurSmoothingList;
    void add(smoothedParameter * sp)
    {
        CurSmoothingList.addIfNotAlreadyThere(sp);
    }

    void remove(smoothedParameter * sp)
    {
        CurSmoothingList.removeFirstMatchingValue(sp);
    }

    void doSmoothing()
    {
        for (auto & sp : CurSmoothingList)
            sp->incValue();
    }
};

the object adds itself to the manager, so you need to instantiate the manager and give it a pointer to that so it can add/remove itself.

add itself when value changed remove itself when value stops changing, or the difference is like... 0.000001. in that case, set the smoother's internal state to the target value, don't leave it at a difference obviously.

[RobinSchmidt]

yes, sounds like a good design. moreover, we could dynamically attach different kinds of smoothers if they have a common baseclass (exponential, gaussian, linear, whatever)

[RobinSchmidt]

class Smoother, subclasses: ExponentialSmoother, GaussianSmoother etc. the SmoothingManager would hold an array of pointers to the baseclass. ...or something

[RobinSchmidt]

perhaps Smoothers should be applicable to any kind of value (not only Parameters). we could have a class: SmoothableValue and a Smoother would operate on an object of that class. a SmoothableParameter would be subclass of SmoothableValue and Parameter (multiple inheritance)

[elanhickler]

whyyyyyyyyyy

[elanhickler]

every time I run the debugger, i crash at a different point for engineersfilter

[elanhickler]

ok I finally got it to work, I think it will crash randomly, and bessel filter is really nice for smoothing, the only problem is that changing the smoothing amount causes things to get weird... like delays in movement, values move on its own, values jumping around (even if no smoothing is going on).

edit: probably related to the fact that prettyscope crashes half the time using EngineersFilter

[elanhickler]

for smoothing mouse movements, exponentialsmoother vs bessel filter is like night and day.

[RobinSchmidt]

uuhh - random crashes? might this be a threading issue? i think, i will need to check this out in the proper context. changing the smoothing amount? is this the "cutoff-frequency" (i.e. the transition time)?. if so, i'm not surprised, if things go weird. this filter is not really made for "modulating the cutoff frequency", so to speak.

i'd still think, we may want to look into an optimized chain of 1st order lowpasses. this will probably solve the modulation weirdness and make the whole thing more lightweight

[elanhickler]

yes, run PrettyScope debugger. Half the time it crashes and doesn't load because of some issue in EngineersFilter code.

[elanhickler]

Can you get something working for my mouse smoothing? Really would like to have a better smoother for PrettyScope mouse sooner than later to impress customers. The problem is Engineer'sFilter doesn't work (just crashes PrettyScope). Bessel filter would work perfectly, but I can't use it due to crashes.

[RobinSchmidt]

wtf is this?

smoother.setTargetValue(*this);
//return smoother.getSample();
return smoother2.getSampleDirect1(*this);

you are passing a de-referenced this-pointer as function argument? into functions that take a double value as input?!

[RobinSchmidt]

how/why does that even compile? :-O

[elanhickler]

*this converts to jura::parameter::getValue();

did you not read my explanation on how my smoothing system works?

here: https://github.com/RobinSchmidt/RS-MET/issues/66#issuecomment-326500209

[RobinSchmidt]

ahh..ohh. ok, i see. so there's an implicit conversion from the "myparams" object to double due to having defined a double() operator? uuuuhhh.....that's....well.....let's say....non-obvious. i might say confusing. i'm even tempted to say obfuscated. i had initially only a glance at the code because i found it confusing and really trying hard to get my head around it would have been a distraction from what i was doing at the time (the mod-system). i'll take a closer look soon

[RobinSchmidt]

but anyway, i think, i could write a smoother class for you based on EngineersFilter which has the same interface as the ExponentialSmoother (plus some additional setOrder function, maybe also setShape - for later adding gaussian). i think, setting it to 1st order would reduce to the ExponentialSmoother. a 1-pole bessel-filter is the same as a one-pole butterworth, gaussian...or just simple rc filter. the differences between the shapes are only apparent for higher orders

btw. i never had any crahses. i tried to comment out the Exp-smoother stuff and uncomment the engineers-filter stuff. can you give me a version of the code which exposes the crash? i need to see it myself to figure out what's going on

[elanhickler]

I just committed a build "this build breaks prettyscope on purpose" with the filter enabled.

but you might run into this build issue: https://github.com/RobinSchmidt/RS-MET/issues/71

[RobinSchmidt]

ok - i'll check that tomorrow

[elanhickler]

ok, don't use the latest commit, use the commit:

"fixed build issues! yay (Prettyscope still broken on purpose, do not use this commit if your name is not Robin!)" daecfea937ecac27fbdd42718f517a184135f4b6

You know how to choose a commit, right?

[RobinSchmidt]

yes, i think so. it's not something i use a lot, though.

[RobinSchmidt]

ok - i think, i know what the culprit is. you do things like:

    paramStrIds =
    {
        &(FXMode = myparams(i++, BUTTON, "FXMode", 0.0, 1.0, 0.0)),
        &(Pause = myparams(i++, BUTTON, "Pause", 0.0, 1.0, 0.0)),

where you use the assignment operator = on objects of class type myparams. that implies that such objects are getting copied. you are actually creating temporary objects here myparams(i++, BUTTON, "FXMode", 0.0, 1.0, 0.0) and then assign them to the lhs FXMode = . after the assignment, the temp objects are deleted. when they contain other objects like my EngineersFilter, those get copied too - using the implicitly compiler-generated assignment operator. but the EngineersFilter class contains pointer variables, so the auto-generated trivial assignment operator is not suitable. it creates a shallow copy but we would need a deep copy:

https://en.wikipedia.org/wiki/Object_copying#Shallow_copy

[RobinSchmidt]

i could manually implement a more suitable assignment operator in EngineersFilter - but actually, i do not intend my dsp objects to be copyable/assignable like variables. things like

EngineersFilter filter2 = filter1;

do not seem to make much sense to me. you could instead of having an EngineersFilter member a pointer-to-EngineersFilter and create the actual object with new in the constructor of myparams and delete it in the destructor (or use a smart-pointer). this way, not the filter itself would get copied but only the pointer-to-it, which should work. i could perhaps explicitly disallow copying of such filter objects such that compiler would complain when you try to do this

[RobinSchmidt]

wait...no... i think, using a pointer still wouldn't work

[elanhickler]

I got something working, but can't really use engineers filter due to values jumping around when setting smoothing amount. Sometimes the value jumps around and then is permanently offset. Seems like the Jura plugin doesn't have this problem, strange.

So don't worry about this, will have to wait until you make a new filter class.

[RobinSchmidt]

I got something working

i think, the cleanest solution woud be to get rid of using the = operator here:

FXMode = myparams(i++, BUTTON, "FXMode", 0.0, 1.0, 0.0)

by replacing these assignments with something like:

FXMode.init(i++, BUTTON, "FXMode", 0.0, 1.0, 0.0)

this way, you would also avoid the wasteful creation and copying of temporary objects

[elanhickler]

ok ill make the change.

so this new gaussian filter you want to make... it could become my favorite filter, bessel filter has no ring. all other filters have ringing. Essentially it's the cleanest possible filter. I'd want to experiment with this filter for general audio/musical filtering purposes, assuming this new filter will be modulatable.

[RobinSchmidt]

i have now declared the copy-constructor and assignment operator as "deleted" in rsEngineersFilter, so now the compiler will balk on any attempt to copy rsEngineersFilter objects. i should really use this idiom consistently throughout the whole library for any class that can't be trivially copied...

i could make a dedicated smoothing filter based on a chain of 1st order lowpasses. eventually, this would also approach a gaussian shape with increasing order (by virtue of the central limit theorem), but however, it would presumably approach it not as quickly as a real gaussian filter design. so a real gaussian design would be a new design formula to be added to EngineersFilter (i have the formula in a book here). but that wouldn't be modulatable. i guess, i should prefer the 1st order lowpass chain then (i think, it would be modulatable)

[RobinSchmidt]

bessel filter has no ring. all other filters have ringing. Essentially it's the cleanest possible filter.

the design goal in the bessel filter is to make the phase response as close to linear as possible which means making the impulse response as symmetrical as possible. the plots in my book suggest that is has a tiny amount of overshoot. i think, a true gaussian would have none but be otherwise very similar

[elanhickler]

There's a ton of discussion on the smoothing filter here.

Get started on the upsampling thing for PrettyScope when you can.

[RobinSchmidt]

What I envision is more of an upsampling filter or a realtime bezier system / filter... like... your onepolefilter has an internal sample & hold, I guess instead of using a onepole you would use a system of a history of data points that you interpolate between.

yes - that's what i have in mind. my plan is to record 4 successive samples and put a cubic spline between the two inner ones such that at the joints (sample-points), the derivatives are matched (i.e. use a "hermite interpolation spline"). i describe that here:

http://www.rs-met.com/documents/dsp/TwoPointHermiteInterpolation.pdf

we would only use the 1st derivative and hence a cubic polynomial. the values of the derivatives (which are inputs to the hermite spline formula) are computed from differences of adjacent samples (we would use a finite difference numerical approximation to the derivative to supply target derivative-values to the formula). it would introduce a delay/latency of one sample because to find the desired value for the derivative at the "now" sample, i would need one future sample (no issue at all in this case). but: i can straightforwardly implement that spline drawing in the prototype cpu scope - but not in OpenGL. there, the most low-level rendering primitive is already the line (i would set single pixels inside my spline drawing function). but maybe i should do it like that anyway and we may later approximate my spline drawing prototype with (short) lines in OpenGL. edit: well, actually i think, we could also draw points in OpenGL to simulate my single pixel drawing (those "paintDot" functions that i would call in my (to be written) spline drawing routine). but that might be too inefficient. but maybe not.

...if you wonder why i dig up such an old quote, i've just been re-reading the whole thread in full

[elanhickler]

If you can't make it work for PrettyScope it is not useful to me. I thought we would ultimately just add more data points to the audio buffer. Also, that's why I posted the full code of dood.al oscilloscope so you could check how they are doing it. And if they are in fact doing something fancy in OpenGL then I don't expect you to do this for me. It looks like they are adding more data points.

[elanhickler]

it's not too inefficient to draw only dots btw. I want to actually move toward doing that for a high quality mode for PrettyScope.

[RobinSchmidt]

that's why I posted the full code of dood.al oscilloscope so you could check how they are doing it. [..] It looks like they are adding more data points.

you mean this: generateSmoothedSamples : function (oldSamples, samples, smoothedSamples) function? add more datapoints to the audio buffer in the sense of giving an oversampled signal? obtaining an oversampled signal is something that my spline-interpolation approach could also do. you would just have to evaluate the spline at intermediate positions. i.e. when the algorithm receives a new sample, it could produce 4 new output samples (if you want to have 4x oversampling, for example). but it's something that a bessel-filter could also do already, so you wouldn't necessarily need new code for that. the spline oversampler may have some advantages though - such as passing through the original samples. just adjusting a bessel filter to sr/4 and feeding it one sample (and 3 zeros) for every input sample would not guarantee that the oversampled signal values at the sample-points (of the original signal) are identical to the values at the same time-instants in the oversampled signal. ...sooo - maybe what you need is a spline-based upsampler class? something like my EllipticSubBandFilter...but as SplineSubBandfilter

i have to admit that i didn't really reverse engineer the code you posted. but the mentioning of a lanczos filter is something that i recognize from the world of image processing. it is actually an interpolation method:

https://en.wikipedia.org/wiki/Lanczos_resampling

[RobinSchmidt]

it's not too inefficient to draw only dots btw

that's good. then maybe my original idea of the spline drawing can be used. my algorithm would then tell you where to draw the dots. you would feed it an input sample pair and it would spit out a number of dot-coordinates to draw (the number of dots may vary from sample to sample, depending on how far apart successive sample-points are)

[RobinSchmidt]

i have already put a stub for the relevant function into the codebase. here's the roadmap:

template<class TPix, class TWgt, class TCor>
void rsImagePainter<TPix, TWgt, TCor>::drawDottedSpline(TCor x1, TCor x1s, TCor y1, TCor y1s, 
  TCor x2, TCor x2s, TCor y2, TCor y2s, TPix color, TCor density, int maxNumDots, 
  bool scaleByNumDots)
{
  // Not yet implemented. Here is what we would have to do:
  // -compute coeffs of the two polynomials:
  //  x(t) = a0 + a1*t + a2*t^2 + a3*t^3
  //  y(t) = b0 + b1*t + b2*t^2 + b3*t^3
  // -compute the total length of the spline segment (this will be some kind of analytic line 
  //  integral) to be used to scale the brightness of the dots
  // -compute a sequence of t-values at which to evaluate the polynomials and set a dot - these
  //  t-values should be chosen such that the spline segments between successive t-values have
  //  all the same length, t should be in the range 0..1
  // -the rest is conceptually similar to drawDottedLine
}

sooo..that's now what to do...

[RobinSchmidt]

ok...making some progress. the connections are smooth now ...but wiggly. i think my data points and derivatives are out of sync. will fix that tomorrow...

[RobinSchmidt]

yessss! old - connecting the dots with lines:

new - connecting the dots with smoooth splines:

ignore the artifact at the center - must have to do with initial conditions and should be irrelevant in realtime mode (i rendered these pics non-realtime using the rsPhaseScopeBuffer class (the data-points are exactly the same in both renderings) - still need to incorporate the new mode into the Scope - not sure, if i should make it switchable or always just use spline rendering)

[elanhickler]

If cpu usage is negligible make it non-optional. Also, if it is negligible, can you make the upsampling algorithm do even more upsampling for even nicer looking results? Or is that not how it works? (like how oversampling can be switched between x2 x4 x8).

Another question, would you be able to use this upsampling inside a synth audio path and get less aliasing? There would be no Gibbs (ripples) like in the antialiasing filter, which is neither good or bad, I would still think to combine it with an antialiasing filter... But the upsampling would perhaps create a smooth (no Gibbs) signal similar to polyblep oscillators but on an arbitrary signal.

Edit: excited to see this in action, can't wait to try it in the rsPhaseScope

[RobinSchmidt]

yeah..i did not yet measure cpu usage. and i'm not finished yet because i discovered another problem not apparent above. the plots above use 80 datapoints for one single roundtrip through the lissajous figure. reducing it to 35 datapoints gives this: ...the shape is still nicely smooth, but i get color discontinuities at the joints. i could trace this to having non-equidistant dots along the splines and having high-density segments join to low-density segments. i'm currently pondering how to remedy this (which may tax the cpu some more, depending on what i come up with....).

can you make the upsampling algorithm do even more upsampling for even nicer looking results? Or is that not how it works?

no, that's not how it works. i'm just connecting incoming datapoints with spline segments which are conceptually continuous between the datapoints (just as the lines are).

would you be able to use this upsampling inside a synth audio path and get less aliasing? There would be no Gibbs (ripples) like in the antialiasing filter

this should indeed be possible. actually, it would just be cubic-spline interpolation upsampling

[RobinSchmidt]

the math has become a bit nasty. in order to get the total length of the spline arc, i have to evaluate this integral (in the cyan box):

http://tutorial.math.lamar.edu/Classes/CalcII/ParaArcLength.aspx

from t=0 to t=1 (t is my spline parameter), where the term under the square root boils down to a quartic polynomial. the analytic formula for this integral is too complicated for wolfram:

http://www.wolframalpha.com/input/?i=integral+sqrt(a_0+%2B+a_1+t+%2B+a_2+t%5E2+%2B+a_3+t%5E3+%2B+a_4+t%5E4)+dt+from+0+to+1

i could find analytic formulas for quadratic polynomials under the square root - but even these formulas were a mess. and for a cubic, all computer algebra systems i tried throw the towel. ...so i have to resort to numerical integration. maybe it's possible to find an analytic formula - but that would be too messy to be useful anyway. moreover, i don't only need the total arc length but also the inverse function of the function which is defined by this integral (by varying the upper integration limit "beta" keeping alpha=0)......but i'm almost there. i've implemented a nice numerical integration function...now i need to invert the resulting function...almost there...this is fun! i love these kinds of problems.

[elanhickler]

oh god kill me.

edit: just let me know, what implications does this have for implementing inside prettyscope?

[RobinSchmidt]

it just makes it a bit complicated for me to compute the positions at which we should draw/accumulate dots. with a line, we would just take the total distance between two points (i'll now call the incoming data "points" and the to-be-drawn dots "dots"), get coeffs for a line equation between the points and draw some number of dots along this line. now, we do the same thing but with a spline (obtain spline-coeffs and put dots along this spline) this is what already works and what the pics show. the thing is that with a line, the dots along the line are naturally equidistant - but with a spline, they are not. naively incrementing the parameter t by a fixed stepsize leads to non-equidistant dots, like this (this is a single test spline segment): which may lead to these color mismatches at the joints of segments. i'm trying to figure out how to increment my parameter t in unequal amounts so as to have equidistant dots. that's the inversion of the integral....you don't have to worry about this - my function will just spit out the dot-positions for you

[RobinSchmidt]

looks like i have the math working. the dots are now equidistant:

that was a fun challenge and resulted in some new math code for the library which may be useful for other things, too. now, i have a lot of cleaning up, refactoring and optimizing to do

[elanhickler]

uhhhhhhhh would it be easy to enable in the toolchain phase scope so I can see how it looks before you do the clean up?

[RobinSchmidt]

ok - next to the AntiAlias button there's now a new menu with two entries for the interpolation mode: "Linear" (as before) and "Cubic". currently, the cubic mode does not yet use the density compensation (i.e. the nasty math). i'll probably make that optional (because i guess, it's going to be much more expensive than the spline-drawing itself)

will refactor now. i think, the interpolation functionality which is now scattered over the two classes rsImagePainter and rsPhasescopeBuffer should be consolidated in its own dedicated class. this will also facilitate a later port to OpenGL. you will then have to use dot-drawing in OpenGL instead of line-drawing (although, i may also add a hybrid mode that interpolates with splines and uses (short) lines to connect points on the spline - we'll see...)

[elanhickler]

cannot open file graphics/realtimespline

rebuilding the toolchain visual studio with jucer didn't work.

how did it manage to not be in the folder?

[elanhickler]

oh because you just added an include and you didn't actually add the file, you must be intending to but didn't do it yet. Commenting out the include got it working.

[elanhickler]

dood.al proof (interpolated)

your result (interpolated)

some differences can be seen like the O shape at the upper right. The proof seems to be more chaotic looking, not as clean as yours.

proof (perfect)

ugly!!!!

[elanhickler]

yours / dood.al / proof

Seems like your upsampling is more accurate, less wiggles.

[RobinSchmidt]

you didn't actually add the file, you must be intending to but didn't do it yet.

oh - right. fixed that. i just added it to the repo. that's where i want to factor out all the code that is related to the 2D spline interpolation which is now scattered over two classes

what do you mean by "proof" and "perfect"? and what is the right image in the last post? linear interpolation? or is "proof" juts the same signal scoped at a higher sample-rate/lower frequency?

Seems like your upsampling is more accurate, less wiggles.

i guess, the wiggles come from dood's lanczos kernel dropping below 0: https://en.wikipedia.org/wiki/File:Lanczos-kernel.svg ...but i actually have no idea, what my "kernel" looks like, because i think about the problem in different terms, but i may try to create some plots..