microsoft / pxt-microbit

A Blocks / JavaScript code editor for the micro:bit built on Microsoft MakeCode
https://makecode.microbit.org
Other
721 stars 593 forks source link

Unite all of the melody/sound string representations #4482

Open riknoll opened 2 years ago

riknoll commented 2 years ago

Across pxt-common-packages and pxt-microbit, we currently have four different ways of representing sounds/melodies

  1. The MicroPython format of arrays of strings (detailed here)
  2. The melody format used in pxt-arcade (detailed here)
  3. The SoundExpression string format used by microbit-codal (detailed here)
  4. The simplified melody format used by the playMelody block

None of these formats are compatible/interchangeable and there is a lot of overlap between them. It would be nice to come up with one format to rule them all, so to speak. I believe the requirements look something like this

  1. Ability to specify a string of note/octave/duration in a human readable way
  2. Ability to specify a custom sound/instrument with: a. Waveform b. Envelopes for different parameters (e.g. pitch, volume) c. Duration d. Effects (e.g. interpolation curves, tremolo/vibrato) e. Chains of all of the above (e.g. multipart sounds to simulate complex envelopes like ADSR)
riknoll commented 2 years ago

My dream format for sound effects (coming from a synthesizer background), would look something like this:

[base pitch] [base volume] [waveform] [effects...] [envelopes...]

The envelopes would include these parameters:

  1. parameter (P for pitch, V for volume, etc.)
  2. parameter modulation amount (-1024 to 1024). This represents the value added to the parameter at the apex of the envelope, after the attack stage and before the decay stage
  3. attack (milliseconds)
  4. decay (milliseconds)
  5. sustain (modulation amount, optional)
  6. release (milliseconds, optional)

Note that all of these parameters are relative rather than absolute; that's so that we can play them like instruments by pitching them up and down.

Jaqster commented 1 year ago

https://github.com/microsoft/pxt-common-packages/pull/1399