First look at tidal.pest

[unlessgames]

So, I skimmed through the mini notation grammar and tested a few things just using pest and the grammar file. Here are my first impressions:

"<>" is mislabeled as slow_sequence, this might just be a name I misinterpret but to be clear this represents an ordered deterministic choice over consequent cycles in tidal. For example a b <c d> is equivalent to a b c followed by a b d in the second cycle. Think it would be more fitting to call this "ordered", "alternating", "sequential" or something else as "slow" clashes conceptually with with the / operator.

A dot between patterns is parsed as a step when it should be a short-hand for grouping a b . c d e translates to [a b] [c d e]

An underscore doesn't expect anything after, it's more like a step where the preceding value is held down, even though synth_3 would be useful to shorten synth _ _ _, it could clash with underscores in sample names (which is why @ exists in tidal as a parametrizable alternative I guess).

The rest char "~" might be better defined as its own rule to let the parser do more.

op_replicate should be able to take a value on the right, a!3 b is valid and turns into a a a b.

Excuse my possible ignorance, but looking at how operators are defined I wonder if it wasn't bettter to define them as a pair of two sequences with an infix operator in the middle instead of x + tail? This would invalidate badly formed input on the parser level, currently 1 2 3 *2 parses when in tidal it is invalid. And I'd imagine it would make writing future generator code in a recursive manner much easier. Since nested structures are everywhere, recursion seems well-fitting. Another thing is that operators should allow for sequences on the right side, for example both a b*<2 4 8> and even a b*<2 4 8!3> is valid in tidal and quite useful.

[emuell]

I've used Strudel's krill PEG as reference here: https://github.com/tidalcycles/strudel/blob/main/packages/mini/krill.pegjs ... and can't really argue about the correctness or terminology used there.

My initial plan was not to support the entire mini notation's feature set. Also definitely not at once, but step by step. Starting with using notes as identifiers only, without Euclidean sequences and without Polymetric sequences.

The needed feature subset also depends a lot on how the notation actually gets integrated into the "rhythm".

---

Haven't worked out on how to the integration would work exactly, but I think there are two main ways of doing so:

1. Create a new Rhythm impl where the emitter and pattern are a strudel cycle, defined by a mini notation string. pattern isn't needed here as the mini notation then defines both, the pulse resolution, timing and actual emitted events. "resolution" and "unit" could define the cycle length then, but there also could be a new "cps" property.

2. allow using a mini notation as an argument to "emit" in existing rhythm impls only. Patterns would then skim through the events defined by the mini notation. This would not allow changing the pulse pattern and timing. The cycle would be interpreted as a flat list of events only.

For me, personally 2. would be all that I want and need. The mini notation would then only be used to create event patterns with random choices, variations and stacks. Also to avoid reinventing the wheel here. Do we really need a perfect tidal clone in Rust?

---

Also, in tidal you are using some "identifiers" as instrument of note references within the mini notation. We could either only use notes for a start or allow mapping them to specific note configurations. For example:

return rhythm {
  ...
  -- use bd as a shortcut for the value specified in the mapping:
  emit = mini("bd [bd sn]"):map {
    "bd" = note("c4 #2") 
  })
}

Thant could actually be useful for the rhythms as they are, without using the mini notation.

[unlessgames]

Also to avoid reinventing the wheel here. Do we really need a perfect tidal clone in Rust?

It would be cool to have, yes :) Joke aside, I understand that everything might not be needed, nor it could fit well with the structure and output here. That said, I think a mini notation just controlling the notes sounds a bit less than what would be ideal. Being able to express polyrhythmic structures is a very fundamental aspect of tidal which wouldn't be possible with option 2. So for example things like bd [hh hh hh] (a mix of 1 half-note followed by 3 quarter-note triplets) wouldn't work while bd [hh hh] would? What's the use of [ ] here if only the notes are taken as a flat list?

So I guess it either has to be more than option 2, or it would be so far from the concepts of tidal and its mini-notation that it might be better to just come up with a separate notation that fits the usecase better. From a user stand-point I'd say it's better to be presented with a new and ergonomic solution to a context than to get confused by a deceivingly similar but fundamentally different version of something you already know.

Maybe I misunderstand option 1, but wouldn't it be better to have some intermediate type that mini-notation is interpreted as and that could be converted to a complete rhythm, a flat pulse list or a flat note list as needed?

[emuell]

So for example things like bd [hh hh hh] (a mix of 1 half-note followed by 3 quarter-note triplets) wouldn't work while bd [hh hh] would? What's the use of [ ] here if only the notes are taken as a flat list?

That's a good point. We'd quite likely end up writing our very own version of a mini notation then. Let's try to keep as close to the official mini notation then - whatever we do.

Maybe I misunderstand option 1, but wouldn't it be better to have some intermediate type that mini-notation is interpreted as and that could be converted to a complete rhythm, a flat pulse list or a flat note list as needed?

Yes, basically all we need, in order to drive a rhythm from a mini notation, is a list of note events (feed to the -> emitter) with their relative pulse times (feed to the -> pattern). Such event lists could be fetches in batches of cycles, so they can then be consumed by the rhythm step by step until the cycle is exhausted and the next one is fetched.

Regarding integration:

A "cycle" could replace the pattern and emitter in the rhythm.

-- replace emit and pattern with a "cycle"
return rhythm {
  -- unit and resolution together define the length of a whole *cycle*
  unit = "beats",
  resolution = 4, 
  -- cycle replaces "pattern" and "emit"
  cycle = "c3 [c4 c5]" 
}

But it actually also might make sense to think of a cycle as a single pulse event in a pattern.

return rhythm {
  -- unit and beat again define the length of a cycle or a note event
  unit = "beats",
  resolution = 4, 
  -- define a single pulse"s time frame with a whole cycle
  emit = cycle("c3 [c4 c5]") 
  -- pattern can then still be used to trigger or skip cycles
  pattern = {1,0,{1,1},0}
}

I need to think about other consequences of the second approach, but this could actually be a good compromise between variants 1 and 2 , while still keeping it conform to the original specs.

One could use "emit" as you do now with patterns, but using e.g. the mini notation's select feature:

emit = cycle("[c4|c5|c6]")

Or also to sequence full blown cycles via pattern:

emit = cycle("c4 <[d4 d5] [g4 g5]>")

It would then also be possible to return cycles from generator functions, so cycles can be dynamic as well:

emit = function(context)
  if macro_value > 1 then
    return cycle("c4 <[d4 d5] [g4 g5]>")
  elseif macro_value > 2 then
    return cycle("c4 <[d#4 d#5] [a4 a5]>")
  else
    return note("c4'maj")
  end

Either way, the harder part for now is to get the PEG parsing working and to split out even lists with event times in batches of a single cycle.

[emuell]

Leaving the actual integration aside for now, I think all we need to make use of the mini notation, is a parser, that generates events in batches of one cycle.

The resulting cycle representation should contain information about which note or which event identifier, such as "bd", is played at which time. Time should be a relative measure here, where 0 is the beginning of the cycle and 1 the end (which actually never should be reached).

For example:

struct Event { note: String, time: f64 };

"bd <sn hi> bd" -> 
  [ Event { note: "bd", time: 0 }, Event { note: "sn", time: 0.333 }, Event { note: "bd", time: 0.666 } ]

This is actually very similar on how the Pulse Iterator works.

When called again, the result may change, depending on the sequence, which is how the cycle generator also will be consumed then. Cycle by cycle...

---

Regarding the necessary features of the mini notation. Of course it would be great to support everything eventually, but covering all the basics except Euclidean and Polyrythms would be a really good start.

---

Does that make sense to you? Personally, I would only do the initial parsing in unit tests, but if you want I could write a simple custom Rhythm impl, one that replaces the pattern and emitter with the cycle parser, so you can play around with it with sound.

[unlessgames]

Yes that mostly makes sense.

all the basics except Euclidean and Polyrythms would be a really good start

You mean polymeters (aka { x y z }%4 -> x y z x) right? I'd consider the polyrhythm notation of a a [b c d] or a a b*3 more part of the basics and it sounds doable if the output can be formed as you said.

I assume the parser would also get a cycle count as input (or keep track of this state internally) to be able to decide where it is in alternating groups like <1 2 3 4>.

Probably a later concern but for the random types [1 | 2 | 3 | 4] and [1 1 1 1]?0.5 I think it would be nice to be able to supply an optional seed to be able to lock generations of output instead of never to find some line again after regeneration.

There is : which can select sample index or note in the context of tidal where each name bass or crow corresponds to a folder with that name and N samples inside. When you say bass:2 it means to play the (2 % number of samples in folder). sample in the folder "bass" or play the synth like superpiano:3 at that note. It only makes sense to be used with identifier strings and then it will create a compound event that has both an identifier and a note.

Which brings me to the type of events, currently I am trying to improve on the smallest building blocks of the grammar first to make the parsed result more explicit. I suppose we could make a distinction in the following events right away instead of giving back a plain string as note (especially in the compound case above). I got these to parse as distinct rules already:

• valid note strings like a a3 contain pitch information and octave useful for the emitter
• integers are mostly used with operators in tidal like bass*4 but they also come handy as scale degrees for example, where you can express melodies by using the scale function with mini-notation input like n (scale " minor" "0 2 [4 5] [6 4]" ) # s "superpiano", without scales it translates to notes relative to middle C, or in the case of samples, the index of a sample in the directory. I think relative notes are better than having to write midi number like 48 and so on, especially if we can say "c4" for absolute notes. Something to consider.
• floats could be good for parameters, probability in patterns and so on, while it's not valid in tidal I think we could easily support ".2" notation for normalized values here.
• arbitrary string names for samples, instruments or for later replacement with map as you said previously.

So I'd make the output have an enum with the available type for each note instead of a String if that's alright, of course it would still need validation later but I think it would be helpful to have the target type already figured out by pest.

Now the outlier here is the rest or empty step, notated using ~ (note: I would make a space delimited - an alias for this as well, since ~ is one of those characters that non-programmers often don't even know how to type, especially if you aren't on US layout). This is a distinct "no event" that still takes up space. The issue is that it's not a note but it still contributes as it means the previously played note is stopped as opposed to held down. On the contrary there is _ which means holding the previous note for the step, so in c4 _ _ _ the c4 will take up the whole cycle where c4 ~ ~ ~ it will cover only the first quarter and then comes silence. What path should we take here? c4 _ _ _ could be just one event for the entire cycle where c4 ~ ~ ~ would be at least two (basically the note on and off for c4)? But I guess the simplest is to just output each step as events (_ as "hold" and ~ as "empty/rest") and deal with them later at integration, or always output events in the form of start and end time.

[unlessgames]

or always output events in the form of start and end time.

Looking at the strudel source now, the mini parser there generates events with start and end times expressed as whole fractions

"a b c d"

[ 0/1 → 1/4 | a ]
[ 1/4 → 1/2 | b ]
[ 1/2 → 3/4 | c ]
[ 3/4 → 1/1 | d ]

When it comes to _ and ~ these don't show up in the output, simply the start/end times are adjusted accordingly.

"a _ c d"

[ 0/1 → 1/2 | a ]
[ 1/2 → 3/4 | c ]
[ 3/4 → 1/1 | d ]

"a ~ c d"

[ 0/1 → 1/4 | a ]
[ 1/2 → 3/4 | c ]
[ 3/4 → 1/1 | d ]

Sampling cycles is done by providing a timespan to the pattern when querying events like

pattern.query(new State(new TimeSpan(Fraction(0.0), Fraction(1.0))))

which will return the first cycle. It's possible to query for more cycles, or parts of cycles, for example to get the second half of the first and the first half of the second cycle

pattern.query(new State(new TimeSpan(Fraction(0.5), Fraction(1.5))))

"a b c d"

[ 1/2 → 3/4 | c ]
[ 3/4 → 1/1 | d ]
[ 1/1 → 5/4 | a ]
[ 5/4 → 3/2 | b ]

I guess this granularity mostly comes handy when using the higher level functions that operate on patterns via distorting time and might be irrelevant to us, just putting it here for reference.

[emuell]

You mean polymeters (aka { x y z }%4 -> x y z x) right? I'd consider the polyrhythm notation of a a [b c d] or a a b*3 more part of the basics and it sounds doable if the output can be formed as you said.

If you think that's an easy one to do, then I won't stop you here :)

I assume the parser would also get a cycle count as input (or keep track of this state internally) to be able to decide where it is in alternating groups like <1 2 3 4>.

I think it's easier to just keep this internal and do it iter-style - fetching one cycle after the other. Otherwise you also need to support walking/parsing backwards or in multiple cycle steps, if cycles are not fetched in ascending order.

Probably a later concern but for the random types [1 | 2 | 3 | 4] and [1 1 1 1]?0.5 I think it would be nice to be able to supply an optional seed to be able to lock generations of output instead of never to find some line again after regeneration.

We are already passing a seed to rhythms, so you can use that one then too: See https://github.com/emuell/afseq/blob/867b1f2b61a50896a39df5088ff2585de9441dda/src/rhythm/generic.rs#L57

There is : which can select sample index or note in the context of tidal where each name bass or crow corresponds to a folder with that name and N samples inside. When you say bass:2 it means to play the (2 % number of samples in folder). sample in the folder "bass" or play the synth like superpiano:3 at that note. It only makes sense to be used with identifier strings and then it will create a compound event that has both an identifier and a note.

That could be translated to an instrument id it the NoteEvent: https://github.com/emuell/afseq/blob/867b1f2b61a50896a39df5088ff2585de9441dda/src/event.rs#L45

Which brings me to the type of events, currently I am trying to improve on the smallest building blocks of the grammar first to make the parsed result more explicit. I suppose we could make a distinction in the following events right away instead of giving back a plain string as note (especially in the compound case above). I got these to parse as distinct rules already:

Ideally the parser should return time tagged Event structs as defined here and not strings (well Option\<Event> - see below): https://github.com/emuell/afseq/blob/867b1f2b61a50896a39df5088ff2585de9441dda/src/event.rs#L307

So note strings should create note events by default, numbers by default probably too. I think everything else needs to be mapped by the user. So if an identifier is not mapped and can't be converted to an Event without mapping, this would be an error.

Same for things like n (scale " minor" "0 2 [4 5] [6 4]" ) # s "superpiano": First, let's keep in mind that we need to do such mapping in Lua and plain Rust. In Rust there are currently no scales, and we likely also don't want to hardcode integers to use scales, and also don't want to add more specific context around the mini notation. Instead we should let the user map the integers to whatever they want:

In Lua this could like this:

local cmin = scale("c", "minor")
return rhythm {
  cycle("0 2 [4 5] [6 4]"):map(function (event) return cmin.notes[event] end)
}

-- or for chords, where the numbers are degrees
return rhythm {
  cycle("0 2 [4 5] [6 4]"):map(function (event) return cmin:chord(event) end)
}

-- note events actually could be remapped then too
return rhythm {
  cycle("c4 c6 [c5 c5] [c3 c6]"):map(function (event) return note(event):with_volume(0.5) end)
}

The Parameter events are currently a placeholder and likely will be useful when using floating numbers in the mini notation later, so I agree that it would be nice to allow them for now.

So, in summary. A valid "event" in the mini notation could be:

• a note string or identifier as already defined by tidal
• an integer or a floating point number

Note strings and integers could by default mapped to NoteEvent. Everything else must be mapped with a user defined callback, else this will be an error.

Now the outlier here is the rest or empty step, notated using ~...

I think it's fine to either extend the event's time or explicitly emit a None Event. That's actually a good point, because then the parser's output should not be a time tagged Event, but actually a time tagged Option<Event> . That's how it works in the rhythm in general as well.

[emuell]

Looking at the strudel source now, the mini parser there generates events with start and end times expressed as whole fractions

As mentioned above, I don't think we need fractions here and simply would move the parser by a full cycle with each call, but else that looks exactly like what we need too.

[unlessgames]

If you think that's an easy one to do

Subdivision in itself is fine, but it does get harder if we want to allow for groups on the right side of expressions like a b*[2 3 4]. Strudel restricts to numbers on the right so this would be invalid but it does work in Tidal. With a subdivision group it is a bit weirder to predict what this does but with alternation for example it is very useful and concise like a b*<2 3 4> translating to a <b*2 b*3 b*4>, which would be nice if we could support eventually.

Note strings and integers could by default mapped to NoteEvent. Everything else must be mapped with a user defined callback, else this will be an error.

So we'd need a new event type that contains a string that can be mapped later, right? It would also need to be a list like NoteEvents.

---

local cmin = scale("c", "minor")
return rhythm {
  cycle("0 2 [4 5] [6 4]"):map(function (event) return cmin.notes[event] end)
}

It's a tangent but I think it would be convenient to supply helpers for these things and only force the user to define their own functions if they want to do something unconventional. Lambdas in lua are a bit inconvenient because the syntax takes up so much space. As a user I'd rather do

cycle("0 2 [4 5] [6 4]"):scaled("minor"):transposed("d")

Or have scale contain those common mapping functions

local cmin = scale("penta", "c")
cycle("0 2 [4 5] [6 4]"):map(cmin.notes)
cycle("0 2 [4 5] [6 4]"):map(cmin.chords)

---

In the meantime I've made some good progress on this, I am hoping to clean up the code a bit (it's going to be rough still) and share it soon.

[emuell]

So we'd need a new event type that contains a string that can be mapped later, right? It would also need to be a list like NoteEvents.

If it helps, create your own data structure for now. We can then port it and merge it with the rest later?

Lambdas in lua are a bit inconvenient because the syntax takes up so much space.

Indeed, but we can surely come up with some common, most likely used helper mapping functions later.