davidfoerster / KaleidOK-examples

KaleidOk invites participants to use a new kind of interactive media tool and take part in an emerging experience which explores speech recognition, media retrieval and visuals generating in a collaborative context (between people, and between people and machines).
http://www.kaleidok.co/
5 stars 0 forks source link
affective-computing art java linguistics processing-library speech-processing synesthesia

Kaleidoscope

Note: Image discovery and retrieval is defunct since Chromatik, the web service behind this function, went defunct itself.

Setup

  1. Request a Flickr API key and consult the Configuration section on where to put it for Kaleidoscope to find.

  2. Similarly request a Google API access key and provide it to Kaleidoscope.

  3. Prepare a configuration for Kaleidoscope.

  4. Some libraries need to be set up, but I'm not going into that now.

Usage

  1. Run KaleidoscopeApp as stand-alone Java application or use the included run IntelliJ configuration.

  2. Click the recorder button in the controls window or press I to start recording from the default microphone line; click the recorder button again or press O to stop. The speech in the recorded section is transcribed, the resulting text synesthetised and the synesthetiation result used to search for images with Chromatik.

  3. If you're unwilling or unable to speak to Kaleidoscope through a microphone, use the two text fields below the recorder button in the controls window. The upper one is for a message to synesthetise, the lower one for search terms for Chromatik's image search. You can run perform a synesthetiation and image search by having the upper text field in focus and pressing ENTER/RETURN.

  4. F11 (Windows/Linux) or +F (macOS) toggles full screen display of Kaleidoscope.

Configuration

The most convenient way to configure Kaleidoscope is through the integrated configuration editor:

  1. Click on the Configuration window tool button at the bottom of the controls window.

  2. Double-click the cells in the right column of the configuration table to edit their value. Most importantly, add the required API keys (see below).

Below is a description of some the most important options. All parameters except API keys are optional and have sensible defaults.