curv3d / curv

a language for making art using mathematics
Apache License 2.0
1.14k stars 73 forks source link

Value Scrubbing / Graphical Value Pickers #45

Open doug-moen opened 6 years ago

doug-moen commented 6 years ago

"Value Scrubbing" is the ability to modify a value in a Curv program using direct manipulation. For example, tweaking a numerical value using a graphical slider. While you drag the indicator on the slider, the shape in the Viewer window updates in real time. This is an important "live coding" feature that makes it easy to explore the effect that different parameter values have on a parametric shape.

Here are two approaches that we could take in Curv. They aren't mutually exclusive, and they serve different types of users.

Value Pickers in the Viewer Window

In one approach, you modify the source code and declare which parameters can be "scrubbed" using an interactive value picker. The declaration includes the type of value picker, and additional information like the start and end values of a numeric slider. When the program is evaluated and the shape is displayed, a collection of value pickers is displayed in the Viewer window along with the shape. This is similar to the Thingiverse customizer, the OpenSCAD customizer, and Fragmentarium.

In this approach, a developer can design a parametric shape, and create a "user interface" for tweaking the parameters. Then another user, who may not understand the code, can use the value pickers in the Viewer window to customize the shape.

Value Pickers in the Editor Window

In the other approach, you select a literal constant in the source code, and a value picker appears above the constant for scrubbing the value. Examples:

Used by developers, for writing Curv programs.

doug-moen commented 6 years ago

How to implement value pickers in the editor window

If you are using a conventional text editor to edit Curv programs, then it isn't possible to implement value pickers in the editor window. The default gedit isn't powerful enough, and vim running in a terminal window can't do this either. Maybe an existing IDE could be used (with what plugins/extensions?), and/or maybe we implement a text editor using Qt, as part of an integrated Curv GUI.

@doug-moen said:

I've thought about embedding Curv in a conventional IDE. But the features that will turn Curv into a reactive visual programming environment, like the glslEditor slider mechanic and other Brett Victor-inspired features, are not found in IDEs.

That kind of points to writing a GUI in C++, probably using Qt, just like OpenSCAD and libfive Studio.

@s-ol replied:

as for the IDE debate - as a vim user I really enjoy curv's current editor-independent live-reloading and general application paradigm. Building 'yet another IDE' sounds a bit "silly" to me - LSP sounds like a better choice for "standard level support". For scrubbing values etc. maybe the LSP protocol can be extended - after a quick look at the specification it seems the protocol would easily support something like that:

the rename/prepareRename mechanism seems similar for example; the client sends a prepareRename to find out whether a location in the text is rename-able, if so, it can preset renaming-UI, and when the user enters a name the rename request can go through. In this second phase the language server generates the project-wide change-sets and the editor only commits them.

Mapping this to scrubbing, there could be a prepareScrub request that yields information about the UI to be displayed (value type, value range etc.). When this request succeeds a corresponding UI is activated that sends previewScrub requests that allow the preview (doubling as the language server) to preview the value changes in real time and finally a finishScrub request to finalize and write back the changes. (previewScrub and finishScrub might also be the same).

This extension sounds general enough to propose to LSPs at large and simple enough to fork and maintain in one 'officially-curv-supported' IDEs LSP implementation even if the proposal doesn't become part of the specification.

doug-moen commented 6 years ago

Implementing value pickers in the viewer window

This is my current project. I'm building an experimental proof-of-concept. The idea is to first create working code, then improve the design.

A parametric shape is a shape value containing additional metadata which describe its parameters. This allows the Viewer window to display graphical sliders for modifying numeric shape parameters. The shape animates as the sliders are moved. (A primary design goal is to represent parametric shapes as first class values, rather than by placing a special interpretation on source files that contain specially formatted comments.)

Right now, you can construct parametric shapes using code like this:

make_parametric << {
    diam :: slider(0.5,4) = 1;
    len :: slider(2,8) = 4;
} -> let
    candy = sphere diam >> colour red;
    stick = cylinder {h: len, d: diam/8} >> move(0, 0, -len/2);
in
    union(candy, stick)

diam and len are the two parameters that you control using sliders. This code runs, but the sliders do not appear in the Viewer window yet. And we might want to replace make_parametric with a better syntax.

Parametric shapes are supported by the following language features:

Parametric Records

A parametric record remembers how it was constructed. It remembers its construction parameters, and it allows you to selectively change those parameters and make a modified copy of the record. This metadata is stored in two record fields: parameter and call.

make_parametric is the current API for constructing a parametric record. (Although you can also create one "by hand", I wanted a more convenient, higher level syntax.) make_parametric is a built-in function, which takes a function F as an argument, and returns a parametric record as a result. The function argument F looks like this:

{ d :: slider(1,10) = 2 } -> cube d

On the left of the -> is a record pattern, with one field pattern for each named parameter in the parametric record. Each field pattern contains the parameter name (eg, d), then :: and a value picker predicate like slider(1,10), then = value specifying the initial value of the parameter.

On the right side of the -> is an expression that use the named parameters to construct a record value, which in this case is a shape, cube d.

Predicate Patterns

A predicate is a function that returns true or false. Predicates are used to classify values. For example, there are some built in predicates like is_bool and is_num that classify values according to their type.

Within a function definition like this:

incr f (n :: is_num) = n + 1;

the phrase n :: is_num is a predicate pattern that restricts the parameter n to being a number. This is similar to a type declaration in other languages.

Value Picker Predicates

To support parametric shapes, we are defining a set of value picker predicates. These are predicate functions that carry extra metadata that describe the type of a value picker, and additional parameters specifying the range of values supported by the value picker.

The initial prototype will just support sliders: slider(lo,hi) specifies a slider that ranges between the numbers lo and hi. In the future, I would also like to support boolean checkboxes, colour pickers, drop down lists giving a fixed set of alternatives, and more.

s-ol commented 6 years ago

I would like to see logarithmic sliders on the Todo list here!

If you don't want to deal with all the edge cases of UI inputs and get a lot of different input means "for free" you might want to look into dear imgui, it's a well known, robust and variable UI framework that stays out of your way and doesn't need to own or invade your data structures, which makes integrating it very easy.

doug-moen commented 6 years ago

@s-ol Thanks for the recommendations. Logarithmic sliders are a good idea.

I am looking into GUI toolkits. I haven't tried any yet.

doug-moen commented 5 years ago

There is now an experimental implementation of slider controls in the Viewer window. See: examples/lollipop.curv.

doug-moen commented 5 years ago

The syntax is now more convenient: the old make_parametric function has been replaced by a parametric keyword for declaring picker parameters. There is now a checkbox value picker, and more in the pipeline. Look in examples/picker for working example code.

doug-moen commented 5 years ago

First public preview, plus documentation: https://github.com/doug-moen/curv/blob/master/examples/picker/README.md