Refactor display into multiple programs; make shaders available from userland; run code on subprocesses

[osnr]

The display primitives (text, triangles, circles, lines, polygons) are now in separate programs in virtual-programs/display/, which is easier to read and patch and more idiomatic.

(Virtual programs can now live in subfolders like /display/ and will be picked up by the system, other than the _archive subfolder.)

They all use wishes as their interface; all Display:: calls have been deprecated and produce a visible warning above any program that uses them. The new wishes generally have optional parameters (color, fill, radians, etc).

Examples from shapes.folk:

  Wish $this draws a circle
  Wish $this draws a triangle with color skyblue
  Wish $this draws a triangle with color green offset {280 0}
  Wish $this draws a pentagon with color gold offset {200 0}
  Wish $this draws an octagon with color red offset {250 80}

  When the clock time is /t/ {
    set offsetVector [list [sin $t] [cos $t]]
    set offsetVector [::vec2::scale $offsetVector 105]
    Wish $this draws a circle with color palegoldenrod offset $offsetVector
  }

  # This toggles a square between filled and unfilled
  When $this has region /r/ & the clock time is /t/ {
    lassign [region centroid $r] x y
    set fill [expr {round(sin($t) * 2) % 2 == 0}]
    set y [- $y 150]
    Wish to draw a shape with sides 4 center [list [- $x 100] $y] radius 60 color white filled $fill
  }

(Arcs and shapes and sprites and fonts have all been ported to the new way of doing things.)

---

dict_getdef has been added, which is used pervasively through the new display wishes, along with rest patterns from #94, to implement those optional parameters.

---

Subprocess code deployment has changed. You boot a subprocess now with Start process NAME, like:

Start process cool {
  Wish $::thisProcess shares statements like [list /someone/ wishes /z/ is labelled /x/]
  Wish $this is labelled "Hello!"
}

You can send more code to an already-existing subprocess with On process NAME:

On process cool {
  Wish $this is labelled "Wow"
}

This is used to deploy code from individual programs to the GPU process without it all having to be in one monolithic file anymore.

There are some catches / ugly bits: process names are global & you now have to explicitly opt into all sharing (either share or receive); there's no sharing by default anymore.

---

You can now write arbitrary shaders from programs. There aren't great helpers for this yet, like shading a region or anything like that, so you need to work in projector coordinates by hand for now. Let me know what you think would be useful to have.

Here is a very basic and not very useful shader that you can just paste as a virtual program (or print). It will color a triangle underneath itself green-blue.

Wish the GPU compiles pipeline "$this's triangle" {
    {vec2 p0 vec2 p1 vec2 p2} {
        vec2 vertices[4] = vec2[4](p0, p1, p2, p0);
        return vertices[gl_VertexIndex];
    } {
        return vec4(0, 0.1, 0.1, 1);
    }
}

Wish $this is outlined blue
When $this has region /r/ {
    set r' [region move $r down 100%]
    Wish the GPU draws pipeline "$this's triangle" with arguments [list {*}[lrange [region vertices ${r'}] 0 2]]
}

See https://github.com/FolkComputer/folk/pull/93 for more info about the shader source object that you pass to the pipeline compilation wish. Also see the implementations of drawing primitives in virtual-programs/display/*.folk.

Notice how you compile the pipeline once (when the program is put down), and then it gets executed repeatedly (whenever the program's region changes).

---

I'm noticing a moderate performance regression here, where this branch hovers between 50-55fps instead of being solidly at 60fps; the evaluator on the display subprocess seems to be much slower (10-12ms per frame) now. Would like to fix this before merging... let me know how much of a problem it is for you.

[nmichaud]

I see startup errors on virtual-programs/shapes.folk:

Error in virtual-programs/shapes.folk, match m89:0: wrong # args: should be "apply lambdaExpr this shapes p a shape options"
wrong # args: should be "apply lambdaExpr this shapes p a shape options"
    while executing
"apply $lambda {*}$env"
    invoked from within
"time {set ret [apply $lambda {*}$env]}"
    ("uplevel" body line 1)
    invoked from within
"uplevel [list time $body]"
    (procedure "baretime" line 1)
    invoked from within
"baretime {set ret [apply $lambda {*}$env]}"
    (procedure "runInSerializedEnvironment" line 9)
    invoked from within
"runInSerializedEnvironment $lambda $env"

Is shapes.folk supposed to work with the new approach?

[osnr]

I see startup errors on virtual-programs/shapes.folk:

Error in virtual-programs/shapes.folk, match m89:0: wrong # args: should be "apply lambdaExpr this shapes p a shape options"
wrong # args: should be "apply lambdaExpr this shapes p a shape options"
    while executing
"apply $lambda {*}$env"
    invoked from within
"time {set ret [apply $lambda {*}$env]}"
    ("uplevel" body line 1)
    invoked from within
"uplevel [list time $body]"
    (procedure "baretime" line 1)
    invoked from within
"baretime {set ret [apply $lambda {*}$env]}"
    (procedure "runInSerializedEnvironment" line 9)
    invoked from within
"runInSerializedEnvironment $lambda $env"

Is shapes.folk supposed to work with the new approach?

Yes -- I think that's an overzealous match where shapes and sprites collide, it should be ignorable (but we should fix it)

[nmichaud]

This is great! It was pretty easy to port my current set of interesting programs

[osnr]

Open questions / notes (will update this over time):

• You can invoke Start process without a name, Start process { ... } if you don't want to deploy code to it later
• Should shader syntax get disaggregated / become a DSL with subcommands like vert, frag, fn, etc, rather than just having like 4 positional arguments in a literal? Should it look more like C where there's a compiler object that you build up?
  • Fn upreferences in shaders are kind of sketchy, don't have a good way to pass them among Folk programs yet
  • Shader names are global which is sketchy
  • Shader compilation stalls the whole system, should do async
  • Need to fix boolean args
• Maybe Wish to draw should be Wish the display draws -- you can think of there being levels of abstraction Wish $this draws (object-centric) > Wish the display draws (natural-language wish) > Wish the GPU draws pipeline (GPU-arglist / shader wish) > Gpu::draw (Tcl imperative) > Gpu::drawImpl (C) > vkCmdDraw (Vulkan/GPU driver)

Things to do:

• [x] Fix spurious startup error (due to sprites match collision?)
• [x] Get performance back up to par
• [ ] Dispose shaders when the wish unmatches (or 5 minutes later or something)
• [ ] Make it possible/easy to define functions from main thread
• [x] Fix Web editor error reporting issues
• [x] Add backward compatibility for Display:: with prominent warning

[nmichaud]

I'm partial to the Wish to draw syntax as being a bit agnostic to what is doing the drawing. I feel like most folk drawing should be done composed as drawing programs (which would then make it easy to federate?)

[osnr]

I'm going to punt on some of the user-facing shader language stuff until next PR (including resource management questions). The main remaining thing is performance (currently 45-55fps instead of a solid 60).

The bottleneck is the glyph / text shaping loop in display/text.folk (you can see for yourself by returning before the actual GPU render wish -- it's just as slow -- and then returning before the whole loop -- performance gets way better). 'Easy' solution is to rewrite in C. Will think about it tomorrow.

[osnr]

Performance is now comparable (within 5fps better/worse depending on context) to main for me, so I'll merge it.