GUI/HUD Feature

[mattdesl]

I'm working in a feature/hud branch to test out the idea of a built-in GUI system for canvas-sketch. It would be like dat.gui but a bit more opinionated, a lot more declarative, and will integrate easily with canvas-sketch as well as features like exporting/importing serialized data (for example, making prints parameters reproducible).

Here is an example, no fancy styling yet:

And a video in this tweet.

It would be built with Preact to keep the canvas-sketch library small.

Syntax

I don't want to introduce a lot of new API concepts to users, and I want it to be declarative and in line with the rest of the ethos of pure render functions. My current thought is to have something like this:

• When params is passed to settings or update() function, the sketch will be added to a global HUD panel, and the panel will be made visible if it has one or more sketches attached to it.
• If the param is an object it can include settings like display name, min/max, etc.
• In the sketch & renderer functions, the param prop is converted into literal values (e.g. instead of a descriptor with options, you just get the raw number value).

const canvasSketch = require('canvas-sketch');

const settings = {
  dimensions: [ 640, 640 ],
  params: {
    background: 'pink',
    time: {
      value: 0.5,
      min: 0,
      max: 1,
      step: 0.001
    },
    number: 0.25,
    text: 'some text',
    download: ({ exportFrame }) => exportFrame()
  }
};

canvasSketch(() => {
  return ({ params }) => {
    const { background, radius } = params;
    console.log('Current background:', background); // e.g. #ff0000
    console.log('Current radius:', radius); // e.g. 0.523
  };
}, settings);

Motivation

It won't be all that different than dat.gui, but:

• It will work well out of the box, and will require zero "wiring" to get properties hooked up to GUIs or render events
• It will be declarative, so it can technically be stripped away by a higher-order function that passes props down (like React props & components)
• It will integrate well with canvas-sketch already, but can also make some nice considerations for exporting, e.g. serialize JSON to a file so that each frame/export is saved with the parameters, or adding a --params flag to the CLI tool to override params with a JSON file
• The global HUD can be a place for other requested features to canvas-sketch, like a play/pause, export button, etc

Features

Should support:

• Text input, number spinners, sliders, color input, 2D XY pad, 3D orbit rotation (?), drop-down, checkbox, buttons, file drag & drop (images etc), ...?
• Fuzzy searching of parameters to quickly drill down big lists?
• Folders or some other way of letting the user organize things?

Questions

• Syntax for buttons/click events? Often users will want to handle the event within the sketch function, rather than before the sketch loads. Maybe some sort of event system?
• Should the parameters be persisted with localStorage? etc.
• How to serialize/unserialize the parameters?
• How to map params to built-in sketch properties like time, dimensions, duration etc?
• Is this a can of worms I don't even want to get into?

[nkint]

👏 nice! I love the concept to have a minimal gui serializable as the save-seed-and-commit works.

Is this a can of worms I don't even want to get into? 😅

[kellymilligan]

Love this idea @mattdesl! I've tried to use ControlKit in my environment before with loads of issues, this would be a real game changer.

• Persisting to localStorage would be nice. Requesting a "reset" control to easily revert to default values without having to manually localStorage.clear(). Additionally, would these values already persist between hot-reloads? That's been a big frustration of my previous attempts.

• On the serialization, I'd love to be able to save a particular config as a snapshot. Similarly to how you can save out iterations of artwork. Could this provide you with a list of previous snapshots that you can return to in a dropdown or something?

[guidoschmidt]

Your approach looks promising, great work 🎉. I love the idea of the fuzzy searching, folders would be definitively nice, too.

To have custom change handlers, something like the following would be convenient from my point of view:

background: {
  value: 'pink',
  onChange: newValue => { /* custom code that should run on change */ }
}

Also I was wondering if it would be possible to have interconnected parameter changes (e.g. change the background color which then will effect another parameter), ... just popped into my head as a maybe nice feature for generative art 🤔.

👍 +1 for persisting localStorage, which @kellymilligan suggested

[mattdesl]

Thanks guys!

Some more questions:

• How to handle serialize / deserialize?
• How to handle onChange event syntax so that user can specify them within the scope of their sketch function?
• How to handle folder syntax?
• How to handle special properties like duration, dimensions, animate etc which link directly with update() function

Possible answers for each...

Serialize

If you specify { serialize: true } in your params, then each time you export a frame, it will also export a [filename].params.json to be associated with that artwork. This means serialize is a reserved word you can't use in your own UI. In the case of animations, it exports a single JSON for a sequence of PNG frames.

To deserialize, you can drag and drop a JSON file onto the page...? Or maybe also specify a --param-file in the CLI to override default params. Maybe there should also be some sort of programmatic way of doing it?

Reacting to onChange and button events

One tricky thing is that most of the time when you react to changes and button presses, you will need to be in the scope of your sketch. What is the syntax for that? Take for example:

const { createRandom } = require('canvas-sketch-util/math');

const settings = {
  params: {
    seed: 0
  }
};

canvasSketch(({ params }) => {
  const random = createRandom(params.seed);

  // how to do random.setSeed(newSeed) with param change?

  return ({ context }) => {
    // render...
  };
}, settings);

Folder Syntax

I'm not sure any clean way to tackle this, really. I could use ES7 decorators but I'd rather not introduce non-standard syntax. One way is just looking for { type: 'folder' } and if found, all keys that are not reserved (keys like open, visible can't be used for example) will be made into UI elements:

const settings = {
  params: {
    circle: {
      type: 'folder',
      background: 'red',
      lineWidth: { min: 0, max: 10, value: 3, step: 0.1 }
    }
  }
};

canvasSketch(({ params }) => {
  console.log(params.circle.lineWidth) // 3
}, settings);

Special Properties

Some properties like dimensions or duration you will just want to 'expose' but not have to wire up yourself manually. In the case of dimensions it could be nice to have canvas-sketch populate a drop-down with paper size presets and so forth. In that case, maybe using special type keys.

const settings = {
  params: {
    // Expose a dimension selector
    dimensions: { type: 'dimensions' },
    duration: {
      // Expose a duration slider
      type: 'duration',
      // In case you wanted to give a specific constraint to the loop duration
      min: 1, 
      max: 10
    } 
  }
};

canvasSketch(mySketch, settings);

[dmnsgn]

I see you are using your own flavour of localStorage in the examples/util/controls.js. Have you ever tried dat.GUI's gui.remember(settings.params); which allegedly saves the current state to localStorage?

[mattdesl]

Yeah! I ended up doing a few things differently:

• It saves state with a local storage key based on the file name you are editing, so the same params in two different sketch files will be saved separately.
• If the code value doesn’t match the locally stored code value, we let it override the local storage. This way if you change a colour manually in code from say ‘red’ to ‘blue’, the GUI will begin to use blue even if it had locally remembered a different colour. This is better UX in my tests so far.

[terkelg]

I would like to help with this one. I've been thinking about building a lightweight HUD library for a while as a separate module but It can be nice to make sure it works great with canvas-sketch – we can design the API based on the needs of canvas-sketch

[onetwothreebutter]

I'm trying to add a control panel/settings to an app I've built using the canvas-sketch and modulating orb thing @mattdesl built for his Frontend Masters course. However, it seems like canvas-sketch may be preventing my mouse events from reaching my form fields. And I've never been able to right-click to inspect the page, even before adding my control panel... is canvas-sketch preventing event bubbling somehow? If this is too off-topic, I can start a new issue or ask this question in a different venue!

[mattdesl]

Hey @onetwothreebutter do you mind starting a new issue and also providing some code samples? It shouldn’t capture mouse events by default.

[RafalWilinski]

I've managed to add https://github.com/dataarts/dat.gui with minimal amount of effort.

1. npm install dat.gui --save

2. Instantiate GUI

   import * as dat from 'dat.gui';
   const gui = new dat.GUI();  

3. Extend settings Object with something like params

   const settings = {
   suffix: Random.getSeed(),
   dimensions: 'A4',
   orientation: 'portrait',
   pixelsPerInch: 300,
   params: {
   myControlledVar: 10
   },

4. Add params to be controlled by datGui

   gui.add(settings.params, 'myControlledVar', 1, 50);

5. Reference it inside render function returned by sketch:

   const sketch = (props) => {
   
   return (props) => {
   const { myControlledVar } = settings.params;
   ...

6. Profit

[mattdesl]

Here's an example showing basically the same thing, but also re-rendering the frame when a GUI parameter changes (fairly useful for static sketches).

https://gist.github.com/mattdesl/04ceca544e637ce1da4d2cf5200d71af