use keys for events to deal with vertical structure (chords)

[polymorphicengine]

currently it is not possible to regard the stack construct as an alternative to lists, since too much information is lost in the transformation to signals, for example a lookup operator wouldn't be possible:

"[1,2 3,4]" !!! 2 --> "2 3"

there is just no way of knowing that the events corresponding to value 2 and 3 respectively "belong together".

my suggestion is to add a new field in the Event type representing this information, i'm thinking of the type

data Key = Key [Int]

where the list of integers represents the path in the "stacktree" (for dealing with nested stacks). for example the keys in

"[1 2, [3, 4 5], 6" would be

1 -> [0]
2 -> [0]
3 -> [1,0]
4 -> [1,1]
5 -> [1,1]
6 -> [2]

since tidal doesn't have many functions dealing with vertical structures it is a pretty easy addition, i think only overlay and stack should be really affected, which can be defined as:

overlay :: Pattern a -> Pattern a -> Pattern a
overlay p p' = Pattern $ \st -> map (\e -> e {key = addKey 0 (key e)}) (query p st) ++ map (\e -> e {key = addKey 1 (key e)}) (query p' st)

stack :: [Pattern a] -> Pattern a
stack xs = Pattern $ \st -> concatMap (\i -> map (\e -> e {key = addKey i (key e)}) $ query (xs!!i) st ) [0..length xs - 1]

i made a working branch of the additions (without tying the changes into the parser) here:

https://github.com/polymorphicengine/Tidal/tree/1.9-stack-try

let me know what you think! i will happily add these changes to the current tidal 2.0 branch if everyone agrees

[polymorphicengine]

thinking about this a bit more, it is probably even better to switch from [Event a] to Tree (Event a) instead of tagging the events with the keys in the tree, here Tree would be somthing like

data Tree a = Leaf a | [Tree a]

[yaxu]

Hey @polymorphicengine, I wonder if this is a bit of a complex solution, compared to adding more support for patterns of lists?

So I think it would be good to look at the underlying use cases to check there isn't a cleaner solution.

[polymorphicengine]

i thought about this quite a bit and i don't really see a way to make it simpler unfortunately.

unless we want to sacrifice some of the nice flexibilities of stacks (for example, not allowing stacks within stacks etc.)

i think moving to a own tree like datastructure will actually make things a bit simpler as to dealing with lists of events, as i see it right now the semantics of combining multiple stacked patterns are a bit unpredictable, it would be really nice to make them more explicit. for example, right now if you have multiple stacks in binary operators on patterns it will produce all possible combinations (i think), this will lead to an exponential amount of events, it would be much nicer to zip the events by default:

"[1,2]" + "[10,[20,30]]" --> "[11, [22,32]]"

since in the current list structure everything is flat, this is quite hard to do.

there are pontentially many use cases like being able to design own arpeggiators or chord modifiers, applying functions only to certain layers of patterns etc., i've seen people wanting such features in discord or on the forum multiple times.

i think this would be a good step towards tidal being able to handle more interesting horizontal structures :)

i'm ofcourse open to other suggestions, but right now i'm convinced that this is a good and not overly complex solution, but it could well be that there will be difficulties i don't see yet :)

[bgold-cosmos]

How does the parser decide what tree structure to give

"[a <b ~>,  c d, e f g]"

when queried for Arc 0.3 0.6? Some things that I might call parent nodes only partially exist on that interval. There's probably a way of dealing but I suspect it's generally going to be a bit arbitrary, and I haven't even gotten very complex with pattern transformations yet.

[polymorphicengine]

ah yes i forgot to take into account that tidal deals in arcs of time, i.e. a pattern represents for every arc all the events active during that time, then probably trees wont do the trick alone, but possibly a list of trees, where the list represents the horizontal and the tree the vertical part of the pattern

i will have to think about it a bit more, please do let me know if you have other ideas to tackle the vertical structure of patterns!

[mindofmatthew]

Assuming that we're in 2.0 land, a thought I've had previously that might also work here: instead of returning a list of events, might a query return a Sequence instead? If I'm reading the code correctly, the Atom constructor includes all of the relevant information that an Event would, and I think the other constructors provide all the structural information that @polymorphicengine is asking for.

In addition to allowing "polyphonic" functions like stack to encode that operation in the structure of their output, using a Sequence rather than a list of events offers more potential information about the structure of the gaps between events. (#721, #760, and #761 all discuss the potential of this)

[polymorphicengine]

instead of returning a list of events, might a query return a Sequence instead?

yes, this is basically also what i'm suggesting, sequences are basically trees in their structure, at least when i last looked at their definition. i think it might be an even more complex solution than going with simple trees, because Sequences have both vertical and horizontal structure themselves (i think there is a stack constructor for sequences)

today someone on the forum/discord asked again about how to deal with vertical structures, would be really nice to come up with a good solution!