vitreo12 / omni

DSL for low-level audio programming.
https://vitreo12.github.io/omni/
MIT License
184 stars 6 forks source link

FFT support #44

Open vitreo12 opened 4 years ago

jamesb93 commented 4 years ago

one possible interface is to add a new kind of block that uses CONST arguments to define the fft settings. the hop: section defines what happens at each hop specified by HOP. For example, a basic script would look something like this.

fft:
    WINDOW = 1024
    HOP = 512

    hop:
        magnitude = hypot(real, imag)
        out1 = maginutes[0] # get the first bin's mag
jamesb93 commented 4 years ago

Thinking about the above interface and developing it a lit...

fft:
    WINDOW = 1024
    HOP = 512
    frame = fft.new()

    hop:
        frame = FFT(in1)
        magnitude = hypot(frame.real, frame.imag)
        out1 = maginutes[0] # get the first bin's mag
vitreo12 commented 4 years ago

One more thing to think about is IFFT: do we want the whole FFT -> processing -> IFFT chain in one object?

jamesb93 commented 4 years ago

Some way of storing previous frames to calculate differences...

could be used for onset detection or some spectral algorithms like median filtering

you could also extend this to making some kind of array of frames for doing bit smoothing jobs

fft:
    WINDOW = 1024
    HOP = 512
    prev_frame = fft.new()
    frame = fft.new()

    hop:
        frame = FFT(in1)
        magnitude = hypot(frame.real, frame.imag)
        prev_frame = magnitude
        delta = magnitude - prev_frame
        out1 = sum(magnitude)
jamesb93 commented 4 years ago

and for IFFT

frame would be an object that has two members - real and imaginary

fft:
    WINDOW = 1024
    HOP = 512
    frame = fft.new()

    hop:
        frame = fft(in1)
        polar = cartopol(frame.real)
        cartesian = poltocar(polar)
        out1 = ifft(frame.real, frame.imag)
        # or
        out1 = ifft(cartesian, frame.imag) # dunno just spitballing here
jamesb93 commented 4 years ago

One more thing to think about is IFFT: do we want the whole FFT -> processing -> IFFT chain in one object?

Not sure what you mean?

vitreo12 commented 4 years ago

One more thing to think about is IFFT: do we want the whole FFT -> processing -> IFFT chain in one object?

Not sure what you mean?

What I was wondering is what should be expected to come out of the outs. Would it be possible to output fft bins? Should that be disallowed and each fft object must provide an ifft counterpart? This, however, would prevent the system to be used as a drop-in fft effect in an already made fft chain (be it pfft, or SC's FFT, etc...)

jamesb93 commented 4 years ago

One more thing to think about is IFFT: do we want the whole FFT -> processing -> IFFT chain in one object?

Not sure what you mean?

What I was wondering is what should be expected to come out of the outs. Would it be possible to output fft bins? Should that be disallowed and each fft object must provide an ifft counterpart? This, however, would prevent the system to be used as a drop-in fft effect in an already made fft chain (be it pfft, or SC's FFT, etc...)

I think that you should contain the FFT to the ømni object rather than outputting some kind of special fft signal for max or FFT buffer for SC. The user would create their entire FFT process in omni and then extract bins, magnitudes etc from there as a signal. So...

out1 = someFftBuffer[24]

and not

out1 = someFftBuffer

If you need to output every bin then make a struct or do it programatically

vitreo12 commented 4 years ago

One more thing to think about is IFFT: do we want the whole FFT -> processing -> IFFT chain in one object?

Not sure what you mean?

What I was wondering is what should be expected to come out of the outs. Would it be possible to output fft bins? Should that be disallowed and each fft object must provide an ifft counterpart? This, however, would prevent the system to be used as a drop-in fft effect in an already made fft chain (be it pfft, or SC's FFT, etc...)

I think that you should contain the FFT to the ømni object rather than outputting some kind of special fft signal for max or FFT buffer for SC. The user would create their entire FFT process in omni and then extract bins, magnitudes etc from there as a signal. So...

out1 = someFftBuffer[24]

and not

out1 = someFftBuffer

If you need to output every bin then make a struct or do it programatically

I see your point. I am not sure I agree 100%. I'd like to keep a degree of interoperability with existing environments... Perhaps one thing doesn't exclude the other, and they could both be implemented.

jamesb93 commented 4 years ago

I can see why you would want this and I come from primarily (almost exclusively) a Max background where I really don't like or use the default methods for doing fft based algorithms. I would never want to pass the output to some pfft~ or jitter to deal with the arrays. Instead, what I would want from omni is to write a process that needs FFT but is wrapped up in the object.