A conceptual framework and library for developing digital musical instruments that are replicable, readable, and reliable, using C++20 reflection and metaprogramming, a subcomponent-oriented development focus, and literate programming.
Read the documentation online at https://sygaldry.enchantedinstruments.com
[TOC]
Copyright 2023 Travis J. West, Input Devices and Music Interaction Laboratory (IDMIL), Centre for Interdisciplinary Research in Music Media and Technology (CIRMMT), McGill University, Montréal, Canada, and Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 CRIStAL, F-59000 Lille, France
SPDX-License-Identifier: MIT
The whole firmware for the T-Stick as of 2023-10-05 consists of a simple and readable list of its components in about 15 lines of code, of which 2/3rds are unavoidable boilerplate:
// includes
using namespace sygaldry;
struct TStick
{
sygse::Button<GPIO_NUM_15> button;
sygse::OneshotAdc<syghe::ADC1_CHANNEL_5> adc;
sygsa::TrillCraft touch;
sygsp::ICM20948< sygsa::TwoWireByteSerif<sygsp::ICM20948_I2C_ADDRESS_1>
, sygsa::TwoWireByteSerif<sygsp::AK09916_I2C_ADDRESS>
> mimu;
sygsp::ComplementaryMimuFusion<decltype(mimu)> mimu_fusion;
};
sygbe::ESP32Instrument<TStick> tstick{};
extern "C" void app_main(void) { tstick.app_main(); }
The substantive (i.e. non-boilerplate) code in this implementation requires fewer lines than the binding code for a single signal in the previous version of the firmware, thanks to the automatic protocol bindings.
Typically, protocol bindings require substantial development time and maintenance. In the previous T-Stick firmware, for every signal that should be sent over the network, we wrote:
// declaration of the signal value at global scope
T signal;
// declaration of the libmapper signal and metadata at global scope
mpr_sig lmsignal = 0;
float sigMax = 1.0f;
float sigMin = 0.0f;
// instantiation of the libmapper signal in the setup function
lmsignal = mpr_sig_new(dev, MPR_DIR_OUT, "sig/address", 1, MPR_FLT, "un", &sigMin, &sigMax, 0,0,0);
// reading the signal value
signal = get_signal();
// sending to the libmapper network
mpr_sig_set_value(lmsignal, 0, 1, MPR_FLT¸ &signal);
// sending over open sound control to the first destination address
oscNamespace.replace(oscNamespace.begin()+baseNamespace.size(),oscNamespace.end(), "sig/address");
lo_send(osc1, oscNamespace.c_str(), "f", signal);
// sending over open sound control to the second destination address
oscNamespace.replace(oscNamespace.begin()+baseNamespace.size(),oscNamespace.end(), "sig/address");
lo_send(osc2, oscNamespace.c_str(), "f", signal);
These 11 lines, dispersed over more than 500 lines of the main file, had to be written by hand and maintaned for every signal added, taking up the majority of the main file implementation.
When writing this code by hand, the burden of adding new signals is multiplied
by the number of lines needed for each communication protocol. Worse, the
burden of adding new communication protocols is multiplied by the number of
signals. The code size and burden increases M*N
, with the number of
protocols M
and signals N
.
Inspired by Avendish \cite celerier2022icmc_rage,
we use C++20 concepts, and C++'s limited reflection capabilities, so that all
of this code is now automatically generated by the compiler without manual
intervention. New signals and communication protocols can be added and existing
ones removed with minimal effort and minimal risk of introducing errors. The burden of
these changes is now decoupled and linear M+N
.
All of the components are developed in a way that limits their dependencies, makes them highly decoupled, and thus enables them to be reused easily to make new instruments. The firmware for a mubone reuses a subset of the components of a T-Stick, and took about 20 minutes to write:
// mubone.hpp
// includes
using namespace sygaldry;
struct Orientor {
sygse::Button<GPIO_NUM_15> button;
sygsp::DefaultICM20948 mimu;
sygsp::ComplementaryMimuFusion<decltype(mimu)> mimu_fusion;
};
struct Controller {
sygse::Button<GPIO_NUM_15> button0;
sygse::Button<GPIO_NUM_16> button1;
sygse::Button<GPIO_NUM_17> button2;
};
// orientor.cpp
#include "mubone.hpp"
// includes
sygbe::ESP32Instrument<Orientor> orientor;
extern "C" void app_main(void) { orientor.app_main(); }
// controller.cpp
#include "mubone.hpp"
sygbe::ESP32Instrument<Controller> controller;
extern "C" void app_main(void) { controller.app_main(); }
Because most of the library consists of decoupled portable software components, there are greater opportunities for automated unit testing, improving the reliability of the components and thus the instrument firmwares written with them.
The modular component-oriented design of the project also makes it easier to approach quality assessment of the components in the library. This could facilitate research that will improve the qualitative and quantitative performance of the library's components. Because the components are reusable across different instruments, these efforts will enable widespread improvement of the instruments developed with the library.
The project is written in a literate style, so that the most efficient way to understand inside and out how the components work is by reading the source code (emphasis reading, not struggling to decode and understand). This is hoped to reduce the urge to start over from scratch so as to better understand the problem, and encourage collaboration and reuse of the library.
Furthermore, the project aspires towards excellent documentation, including guides and tutorials, to further reduce barriers to adoption and encourage widespread collaboration on the project.
If you are still reading on Github, click here to switch to the online documentation. Github does not render doxygen markup correctly, so the following links will not show. Reading the documentation website is recommended.
Depending on your goals, consider starting with one of the following documents (work in progress):
Most digital musical instruments (DMIs) that are developed fall into disuse after a short amount of time \cite morreale2017nime_design-for-longevity. The poor longevity of DMIs is an important subject for DMI research, and numerous approaches have been proposed to enable long-lasting DMI design and use, from socio-ecologic approaches \cite mcpherson2012cmj_problem-of-the-second-performer, to musical \cite marquez-borbon2018nime_dmi-adoption-and-longevity and technical \cite zayas-garin2021nime_dmi-apprenticeship pedagogy, and implementation strategies that can improve longevity \cite franco2017prynth.
A major challenge facing long-term DMI use is technical failure. \cite morreale2017nime_design-for-longevity report that as many as 47% of DMIs may fall into disuse due to the instrument becoming broken or inoperable, such as due to software updates. \cite sullivan2021thesis found that reliability was essential for long-term DMI use. For a DMI to sustain long-term engagement, it must be maintained, updated, repaired, and eventually replaced \cite calegario2021nime_replicability. As technology platforms inevitably change and the tools and materials originally used to make a DMI become obsolete or otherwise unavailable, i.e. as a DMI ages, a maintainer must eventually replicate the design of the instrument using available tools and materials \cite calegario2021nime_replicability. Recent calls have argued that it is essential for the longevity of a DMI that its design be documented in sufficient detail to enable replication \cite calegario2021nime_replicability.
DMIs are almost always (e.g. 73% of DMIs reported at NIME from 2020 to 2022) implemented using an embedded hardware processor such as a microcontroller unit (MCU) or single-board computer (SBC), often (40\% ibid) in combination with another more powerful computer such as a laptop. Many different languages and protocols are employed, often simultaneously.
The combination of hardware devices, programming languages, and communication protocols makes the code that implements a DMI generally incompatible with a different combination of platforms. This can make it difficult to reuse code (whether one's own, or that of a third party) when developing a new DMI, leading to wasteful repetition of effort. As design requirements change or a DMI ages, code must inevitably be ported to new platforms and environments, and issues of portability and code reuse become issues of maintenance, replicability, and reliability, further harming instruments' longevity.
Current efforts to improve the portability and reuse of DMI software components are largely restricted to sound signal processing (e.g. Faust) and are generally not applicable to the development of DMI components such as sensor signal conditioning or mapping. Other approaches to portability often impose significant runtime performance impacts, safety issues, and can eventually become yet another platform to target for portable implementations \cite celerier2022icmc_rage-against-the-glue.
The possibility of scientifically advancing DMI design through evaluation of DMIs has recently been called into question \cite goudard2019nime_ephemeral-instruments \cite rodger2020nime_what-good. The assemblage of DMI components makes each DMI a unique system \cite goudard2019nime_ephemeral-instruments. The further embedding of these systems in co-constitutive socio-musical ecologies makes study and evaluation of DMIs deeply specific \cite rodger2020nime_what-good. Coupled with a general lack of consensus about how to evaluate DMIs \cite barbosa2015evaluation, this makes it extremely difficult to derive generalisable insight from evaluations of DMIs, and poses severe challenges for the scientific study and advancement of DMI design.
Sygaldry aims to explore a framework that addresses the replicability, portability, and generalisable study of DMIs by employing three interlocking development techniques: reflection and metaprogramming in C++ \cite celerier2022icmc_rage-against-the-glue; a paradigmatic approach that gives a strong emphasis to constructing reusable components primarily, with musical instruments emerging as a side-effect; and literate programming \cite knuth1984literateprogramming.
Reflection in C++ enables software components to be implemented in dependency-free C++ and then deployed to numerous heterogeneous computing environments, such as MCUs, SBCs, and host languages like Max/MSP and Pure Data. The approach proposed by Celerier \cite celerier2022icmc_rage-against-the-glue reduces the code-size complexity of implementing portable media processors, dependent on the number of components $N$ and target environments $M$, from quadratic $N * M$ to linear $N + M$. Originally proposed for binding media processing algorithms to plugin formats and software host environments, Sygaldry applies Celerier's approach to DMI development and maintenance.
Whereas whole DMIs are highly specific, DMI components such as sensors and mapping strategies are regularly found in many different DMIs, and can be assembled \cite goudard2019nime_ephemeral-instruments to make new DMIs. By focusing development and evaluation on DMI components, we should be able to build a library of reusable parts that can be leveraged by many designers. Focusing on modular components should also enable automated performance and correctness testing, improving reliability. Evaluations facing components should have better generality, and allow insights uncovered to be applied to any instrument that leverages the evaluated component. Using C++, especially with the techniques just described, provides the best possible code portability and reusability of these components for the least amount of effort, allowing the components developed to eventually be leveraged in a wide variety of programming languages, hardware platforms, and other runtime environments.
Modularity, reliability and portability, however, are insufficient qualities to enable reuse and replication. Literate programming \cite knuth1984literateprogramming advocates for computer code to be written with a human reader in mind. This practice improves the readability and understandability of source code, allowing low level details and design praxis to be clearly documented as part of the process of implementation. By acting as a kind of textual apprenticeship \cite zayas-garin2021nime_dmi-apprenticeship literate source code is hoped facilitate the transmission of research and design products that are often not reported and provide the necessary comprehension to encourage future maintainers and other researchers to adopt the reusable components thus documented, rather than starting over from scratch as is so often done.
Sygaldry consists of five main parts: the concepts and helpers libraries, the components and bindings libraries, and the instruments collection.
This library contains C++20 concepts that aim to operationalize the high-level
conceptual framework that guides the design of the project, as well as generic
methods for reflecting over and accessing the data and functionality associated
a component adhering to this conceptual framework. The concepts library depends
on boost::pfr
, boost::mp11
, and the utilities library, but should have no
other dependencies (except for its tests, which regularly make use of the
helpers library). It is most useful to binding authors, but may also be helpful
for component authors who wish to make use of plugins and throughpoints
generically. This library should be platform independent. It is defined in
the namespace sygaldry
.
Whereas the concepts library defines generic tools for inspecting and using
components, the helpers library defines specific tools that facilitate
authoring components. The helpers library is intended to be compatible with the
concepts library, but without any physical (i.e. compiled) dependency on the
latter. The helpers library depends only on the utilities library (except for
its tests), and should be platform independent. It is also defined in the
namespace sygaldry
; it is should be guaranteed that the concepts and helpers
libraries should not have any conflicting names, and if ones are encounter this
is a bug.
The components library contains a collection of sygaldry
components useful
for building digital musical instruments, implemented using the utilities,
concepts, and helpers libraries, and without any other dependencies (except for
its tests). Careful attention is given the the portability of these components.
A platform-independent component is allowed to directly physically depend only
on other platform-independent components. A platform-specific component is
allowed to directly physically depend on other components for the same
platform, and on platform-independent components. We have so far implemented
drivers for numerous different sensors in the namespaces sygaldry::sygsX
,
where X
is a platform-specific alphanumeric code or p
for portable
components. In the future, we hope to implement other useful components for
making mappings and sound synthesis signal chains.
The bindings library contains components that mainly reflect over other
components generically, providing functionality such as control protocol
bindings. It is defined in the namespace sygaldry::sygbX
where X
is
a platform-specific alphanumeric code or p
for portable components. Binding
components are allowed to directly utilize the interface of other binding
components, but must generically access other components, such as those in the
Sygaldry components library, using the generic methods defined in the
concepts library. The bindings library depends on the utilities, concepts, and
helpers library, and is not allowed to access the components library directly.
Like the component library, the bindings library provides both
platform-independent and platform-specific components in appropriately nested
namespaces.
Taken together, the utilities, concepts, helpers, components, and bindings
libraries make up the sygaldry
library. The instruments library contains a
collection of digital musical instruments implemented using the sygaldry
library, and completing the sygaldry
project.
When viewing the source code of the project, the above libraries account for most of the top level directories. The rest are briefly described below.
The sh
directory contains scripts used in the build process.
For more detail, refer to the build system document.
This directory contains 3rd party submodules used by other components,
including the boost
libraries required for reflection and metaprogramming,
and libraries meeting other platform- or component-specific requirements.
This directory contains resources used when generating the documentation website.
Related documents: