FAUST integration

[ohmtech-rdi]

Our users expressed the need to have different language integration, and FAUST is one of them.

Overview

Faust (Functional Audio Stream) is a functional programming language for sound synthesis and audio processing with a strong focus on the design of synthesizers, musical instruments, audio effects, etc. Faust targets high-performance signal processing applications.

Epics

From Scratch

Avery has an idea of a new Eurorack module. They want to develop it in FAUST, because they are familiar with it, and they can leverage quickly the big standard library of dsp blocks available at their fingertips.

They started to draw the layout of the module on a piece of paper and a quick schematic of the audio flow. They use the GUI editor to make a first sketch of their idea and write the dsp code.

They iterate over the sketch by usually first modify the GUI to add or remove a control, and then update the dsp file accordingly.

From Existing

Brooklyn has been working for 10 years with FAUST and as a result has a big collection of dsp modules, some of which they would want to have as real Eurorack modules.

For each dsp code, They use the GUI editor to make a first sketch, that they will iterate on.

Workflow

FAUST code have presentation and model mixed into the source code. The faust program can be used to generate C++ code as well as declarative structured data representing some static properties of the DSP code (such as number of input/outputs, or parameters' description), for example in JSON.

faust -json -o plugin_faust.cpp ../Kick.dsp

Will generate:

• A JSON file that tells us about the number of audio inputs and outputs, parameters with their associated widgets (from which we can deduce a type), as well as minimum, maximum values, when applicable,
• A C++ file that can be instrumented with the use of an Architecture file.

Bindings considerations

Like the erbui/C++ bindings, we need to be able to map a erbui name to a corresponding C++ name.

This is done in C++ by making sure that a mismatched mapping will fail at build time rather than at runtime.

Since we can extract a static representation of the FAUST dsp code, we could apply the same principle as in C++, that is an unused erbui name in C++ would be silently ignored, but a name in C++ not defined on the erbui side would trigger an error.

However, using C++ allows a strict control about parameters of the DSP.

In the case of FAUST, including libraries may add widgets primitives to the generated DSP code, that the user wouldn't want to have visible in the erbui UI.

For this reason:

• A parameter in FAUST dsp code that wouldn't match its counterpart in erbui will be either silently ignored or at most emit a warning,
• A control in erbui code that wouldn't match its counterpart in FAUST code will be silently ignored.

Compound types

ERB has the concept of compound types, for example:

• A 3-state or 2-state switch is made of 2 contacts,
• A dichromatic LED has 2 pins for each color,
• More complex compounds will come like a fader with associated LED.

FAUST doesn't have this concept for its widgets. Furthermore, widgets are classified as passive or active whether they respectively take or provide a scalar value.

To remedy this, since the label is a string, we will use dot notation to bind a scalar to a compound element.

For example, considering the following erbui code:

module MyModule {
   control led LedBi { ... }
}

To access red and green components of the physical LED, we could use the same notation as in C++, that is led.r and led.g, but from within the primitive label:

... : vbargraph("led.r",min,max);
... : vbargraph("led.g",min,max);

Name mapping

On one side, erbui defines a name of a control as an identifier. Such an identifier matches the regex ([a-zA-Z]\w*), so that it needs to start with a letter, and then be followed a letter, number of underscore.

On the other side, labels are arbitrary. They can start with a number, contain spaces, and all sorts of other different characters.

Also, due the recursive nested structure with *group primitives, the same label may appear multiple times, but with a different group.

Given that OSC paths must be unique, "fully qualified labels" are unique for a given FAUST top-level source code.

Instead of breaking the isomorphism to match names between erbui and FAUST by applying some sort of normalisation of the name, which would also be very ugly for the user in some case, we will allow to create a string literal that aliases the erbui control name. This string literal will be the "address" of the widget, as defined by FAUST and available in the JSON description of the dsp code:

module Guitar {
   control attack Pot {
      faust { bind { address "/Distortion Effect/0x00/Compressor/Attack" } }
   }
}

When the faust address is not present, it will be crafted by taking the control name prefixed with a /:

module Guitar {
   control attack Pot {
      // would be implicitely: faust { bind { address "/attack" } }
   }
}

In the case of compound types, the control property identifier needs to be specified in the binding:

module Guitar {
   control led LedBi {
      faust {
         bind { property r address "/Distortion Effect/0x00/Compressor/Level X" }
         bind { property g address "/Distortion Effect/0x00/Compressor/Level Y" }
      }
   }
}

When the faust address is not present, it will be crafted with dot notation by taking the control and property names prefixed with a /:

module Guitar {
   control led LedBi {
      /* would be implicitely:
      faust {
         bind { property r address "/led.r" }
         bind { property g address "/led.g" }
      }
      */
   }
}

We consider that the order in which widgets are defined in the FAUST JSON generated file follows the same order in which widgets functions are called in the C++ generated buildUserInterface function: this allows for a very straightforward mapping.

Initial value

It is worth noting that a non-erbui-mapped name is valid, but the underlying parameter will stay to the FAUST defined default "init" value, which is probably not always good.

Also in this case, the generator issues a warning:

warning: non mapped hslider /PHASER2/0x00/Notch Depth (Intensity)

Those warnings are nice, because it allows to fetch easily the address of a FAUST widget, when using already made dsp, and when the widgget is buried deep into the dsp hierarchy.

But leaving warnings can be unsatisfactory, so we need a way to solve that nicely.

To solve those 2 problems (silenting a warning, and providing an initial value different from the FAUST defined one), we can simply add a syntax for this, for non-mapped active components:

module Guitar {
   faust {
      init { address "/PHASER2/0x00/Notch Depth (Intensity)" value 0 }
      init { address "/PHASER2/0x00/Vibrato Mode" value 0 }
   }
}

Finally for completion, we can do the same for the equivalent "passive widgets" (in FAUST terms) for an erbui control like a LED:

module Guitar {
   control led_bi LedBi {
      faust {
         bind { property r address "/Distortion/0x00/Output Gain" }
         init { property g value 0.5 }
      }
   }
}

Generator specifics

Sample data binding is done through the UI adapter, rather than being an adapter on its own.

Thought our adapter declaration and definition can be splitted, so that we have 2 definition files (one for the UI, one for the sample data).

This allows the erbui generator to only generate UI adapter functions, and the erbb generator to only generate audio sample adapter functions.

Parameters

All parameters (inputs or outputs), are represented as floating-point numbers.

Input parameters like sliders are associated to a function defining the minimum and maximum value as well as a mapping function defined for example as a meta/scale function. Since ERB considers only normalized or bipolar numbers, some sort of adaptation needs to be done there.

Conversely, output parameters are associated to a function defining the minimum and maximum value, and a mapping function. Since ERB LEDs are normalized, the inverse function will need to be done there as well.

The mapping functions are implemented in C++, but we will generate a flattened version of this C++ code from python.

The numerical entries widget nentry doesn't have an equivalent in the hardware world, but it is considered as a scalar value.

Audio inputs/outputs

We don't have labels for audio inputs and outputs, so we will rely on the order in which they are declared in the erbui file to bind them with the FAUST dsp code.

Audio files mapping

Since we don't have the concept of a filesystem, we need a system to match the erbb AudioSample definition to the corresponding FAUST soundfile expression.

To reuse the same concepts, we can use the name mapping defined for the erbui Language, but in the erbb language, since Data is generated from erbb, not erbui:

resources {
   data sample AudioSample {
      file "sample.wav"
      faust { bind { address "/Drum/0x00/Kick/sound" } }
   }

for the corresponding FAUST dsp code (in a library Kick.dsp):

process = 0,_~+(1):soundfile("sound[url:{'kick.wav'}]",2):!,!,_,_;

In particular, should there be a url option like "foo[url:{'foo.wav'}]", then the url information is either ignored or an error or warning is emitted if the file names do not match. At first, we will just ignore it.

As for name mappings with erbui, we will keep the same nice features, for example:

• A non-mapped sample will emit a warning
• If a faust binding is not done, then it is implicitely deduced from the module name and data name:

module Kick {
   resources {
      data sample AudioSample {
         file "sample.wav"
         // implicitely:
         // faust { bind { address "/Kick/sample" } }
      }
}

Audio files stream format

FAUST only supports for now planar (ie non-interleaved) sample formats.

Eurorack-blocks support this stream mode, and will use mode planar automatically for every multi-channel samples when using FAUST.

Planar representation most probably suffers from cache cohenrency on embedded platforms, as well as using SIMD instructions more difficult and less efficient, but we will leave this for now until we have collected execution performance data, and a interleaved mode would be supported in FAUST.

Audio files "parts"

FAUST has some sort of convenience system which allows to assemble multiple audio files into one consolidated file.

We don't have such a convenience system, and don't plan to use it. Rather we could use standard markers (such as "cue"/"splice" markers for WAV format), to detect parts in an audio sample file.

The current latest brew/apt release of FAUST doesn't include the number of parts in the SoundFile structure, so that the later structure is not closed (ie the information it provides are not sufficient to use it). But work has been already done to include this number in the SoundFile, and should be therefore available after version 2.39.1.

Memory management

Ideally the generated code should be portable for both the simulator and the embedded daisy platform.

However "allocating" big portions of memory, such as delay lines, are going to be handled differently on both platforms:

• On the desktop dynamic allocation can be used,
• On daisy the portion of memory needs to have a special section attribute to be placed in the SDRAM section.

Such "allocations" (the amount of memory is actually known at build time) appear when a @ time expression is used, for example.

For reference, FAUST has a custom memory manager, but it is only meant for rdtable primitives.

It is still unclear how to handle that, but it might be out of scope for this integration. The generated FAUST code will need somehow to place large amount of memories in the SDRAM section, and ERB should just make sure that enough space is available in SDRAM by decreasing the erb_SDRAM_MEM_POOL_SIZE to just what is needed for ERB (which is currently nothing).

In the worst case, provided the compiled code would need to be different for each platforms, we could wrap the code underlying target specific implementations:

// faust.cpp
#if defined (erb_TARGET_SIMULATOR)
   #include "faust_simulator.cpp"
#else if defined (erb_TARGET_DAISY)
   #include "faust_daisy.cpp"
#endif

Unsupported features

Layout informations, as this is already done by erbui, are ignored. This includes all "box" primitives, that is hgroup, vgroup and tgroup UI primitives.

While layout informations are ignored, the label of a group is used for name mapping as described above.

Tasks

• [x] Make a PoC manually
  • [x] Audio inputs/outputs
  • [x] Active parameters
  • [x] Passive parameters
  • [x] Parameter scaling
  • [x] Name mapping
  • [x] Compound types
  • [x] Sound files
  • [x] Memory management
• [ ] Update documentation
• [ ] Make Erbb FAUST generator
• [ ] Make Erbui FAUST generator
• [ ] Add sample code

[sletz]

To remedy this, since the label is a string, we will use dot notation to bind a scalar to a compound element. ==> can you explain mode this ?

Is FAUST expecting mono samples? : ==> samples can have any number of channels (see https://faustdoc.grame.fr/manual/syntax/#soundfile-primitive)

What is the "part" concept in Soundfile? : ==> a URL can describe a list of actual soundfile (up to 256). Each of them with then be a part (an index between 0 and 255) to access the wanted soundfile

For reference, FAUST has a custom memory manager, but it is only meant for rdtable primitives. ==> we plan to expand the model to have the memory manager also handle fast (SRAM) and slow (SDRAM) section

BTW; Faust C++ code could be generated a bit differently (different compilation options) to 1) run on the simulator, then 2) run on the daisy board with an adapter Memory Layout handling.

[sletz]

Ho do you plan to properly simulate CPU usage on VCV Rack ? (that is a given DSP that runs on the simulator may not be able to run on the real daisy board because of too high CPU use. So the simulator should have a kind of "CPU usage ratio" to warn the developer when CPU will be too high).

[ohmtech-rdi]

To remedy this, since the label is a string, we will use dot notation to bind a scalar to a compound element. ==> can you explain mode this ?

Let's say we have a dichromatic LED in erbui:

module MyModule {
   control led LedBi ( ... }
}

Then accessing the red and green scalar value would be done in FAUST:

vbargraph("led.r",min,max)
vbargraph("led.g",min,max)

[ohmtech-rdi]

BTW; Faust C++ code could be generated a bit differently (different compilation options) to 1) run on the simulator, then 2) run on the daisy board with an adapter Memory Layout handling.

Yeah, I guess in that case I would generate two files and assemble them, something like:

// faust.cpp
#if defined (erb_TARGET_SIMULATOR)
   #include "faust_simulator.cpp"
#else if defined (erb_TARGET_DAISY)
   #include "faust_daisy.cpp"
#endif

So maybe it's not a big issue whatever the technology choice to handle SRAM/SDRAM.

[ohmtech-rdi]

Ho do you plan to properly simulate CPU usage on VCV Rack ? (that is a given DSP that runs on the simulator may not be able to run on the real daisy board because of too high CPU use. So the simulator should have a kind of "CPU usage ratio" to warn the developer when CPU will be too high).

Well the problem is actually even more complicated. The time it takes to make a computation is more memory-speed bound that it is cpu-bound.

We thought about different way to address that (for example using cachegrind) but nothing really great came out yet.

Any idea on how to achieve that would be very welcomed!

[sletz]

Given that OSC paths must be unique, we will assume that "fully qualified labels" are unique for a given FAUST top-level source code. ==> this is the case: the compiler refuses to compile a DSP that would produce several same path.

what is the character encoding standard used in a FAUST dsp file? ==> TBH this is not really specific for now...

for both passive and active widgets, is there a function, defined on the FAUST side, and in C++, "safe" (that is no runtime failures like C++ exceptions) that can be used? ==> we have C++ written mapping functions code. Is that what you mean?

[ohmtech-rdi]

for both passive and active widgets, is there a function, defined on the FAUST side, and in C++, "safe" (that is no runtime failures like C++ exceptions) that can be used? ==> we have C++ written mapping functions code. Is that what you mean?

Yes I think that will be something in the line of it.

For example given the JSON content:

{
   "type": "hslider",
   "label": "Notch width",
   "address": "/FaustGuitarEffectChain/PHASER2/0x00/Notch width",
   "meta": [
      { "1": "" },
      { "scale": "log" },
      { "style": "knob" },
      { "unit": "Hz" }
   ],
   "init": 1000,
   "min": 10,
   "max": 5000,
   "step": 1
},

I want to know if there is a function in the FAUST project, written in C++ or python, that would convert a normalised value from 0 to 1 to the value returned by the hslider primitive upper.

And conversely the same for passive widgets @sletz

[sletz]

The conversion functions are indeed used when decoding the metadata [scale:log|exp], see this use-case in DAISY architecture https://github.com/grame-cncm/faust/blob/master-dev/architecture/faust/gui/DaisyControlUI.h and https://github.com/grame-cncm/faust/blob/master-dev/architecture/faust/gui/DaisyControlUI.h#L156