Add place sensitive cue points in editor

[capital-G]

A place for discussion and sketches on how to implement location based services in the editor of Gencaster

Elementary

• Fire "triggers" based on location which jump to a direction within the graph?
• Circles are enough for now - polygons would be top
• Is there a way to include relative positions?
• Can a POI create a trigger? Or something like a magnet?

ToDo

• [ ] @vin-ni Sketches for editor
• [ ] @capital-G How to intertwine the current (absolute/single execution) graph based approach with "location based triggers"

See also

• https://github.com/Gencaster/gencaster/issues/614

[vin-ni]

[capital-G]

We had an exchange today on how to tackle the difficulties of generative playback, absolute positions on a map as well as relative/generated positions on a map.

The main idea is to

• create a map node, which can be entered like any other node and exited based on conditions such as
  • minutes passed
  • exited area
  • visited all areas
• A map consists of multiple areas (polygons) - these can be entered and if entered, the associated script cells will be executed in a blocking manner. There should be the possibility to exit early (aka not execute all script cells) in case we exit the area. This allows for
  • having overlapping areas in sound as an async playback can be triggered or SuperCollider code can be executed
  • respecting the succession of areas visited by having the ability to block a node through e.g. sync playback
  • We still follow the graph approach: There is one node responsible and one node responsible to figure out what happens next - no triggers which would allow to jump in the graph to an arbitrary position
• In order to reflect the idea of a OSM query, we should have something like a "node factory", which creates polygons on the map derived from a OSM query. A factory will then create a polygon "on the fly" for each object on the map and attach the script cells of the factor to each of these nodes.

See the sketch above

Open questions

• Is it sufficient that an area is represented by a script cell or should it be more generic like a "sub-graph"

[capital-G]

I think it would be a good idea to create sequence diagrams which cover the cases of

• Audio-walk / sequenced
• Overlaying audio walk / async
• OSM factory nodes

I started with a simple mermaid diagram for now (yay, github finally has a way to display graphs which are stored as code)

sequenceDiagram
    create actor User
    create participant Start
    create participant BackgroundMusic
    create participant LocationB
    User-->>+LocationB: user entered location B
    Start->>BackgroundMusic: NormalFlow

    create participant MapNode
    BackgroundMusic->>MapNode : NormalFlow

    create participant LocationA
    %% create participant LocationB
    MapNode->>+LocationA : check if entered
    MapNode->>LocationB : check if entered
    LocationB->>MapNode: user entered B
    MapNode->>LocationB: run script cells
    LocationB->LocationB: execute script cell 1
    LocationB-->>-User: user left location B
    LocationB->>MapNode: script cell 1 executed - terminate?
    MapNode->MapNode: is exit condition met? -> No
    MapNode->>LocationB: continue
    User-->>+LocationA: user is in location A
    LocationB->LocationB: execute script cell 2
    LocationA->>MapNode: user entered A
    LocationA-->>-User: user leaves location A
    LocationB->>MapNode: finished
    MapNode->MapNode: is exit condition met? -> Yes

    create participant StopBackgroundMusic
    MapNode->>StopBackgroundMusic: jump to next node

    %% Alice->>+John: John, can you hear me?
    %% John-->>-Alice: Hi Alice, I can hear you!
    %% John-->>-Alice: I feel great!