IDyOM: Information Dynamics of Music

Version 1.7

Copyright 2001-2023, the IDyOM development team (see CREDITS).

This program is distributed under the terms of the GNU General Public License. See COPYING for details.

Documentation

Online documentation is available at: https://github.com/mtpearce/idyom/wiki

Scientific presentations of IDyOM can be found in Pearce (2005, 2018), copies of which are provided in the directory docs/ - please use these as citations in any publications using this software.

Pearce, M. T. (2005). The Construction and Evaluation of Statistical Models of Melodic Structure in Music Perception and Composition. Doctoral Dissertation, Department of Computer Science, City University of London, UK.

Pearce, M. T. (2018). Statistical learning and probabilistic prediction in music cognition: mechanisms of stylistic enculturation. Annals of the New York Academy of Sciences, 1423, 378-395. https://doi.org/10.1111/nyas.13654

Requirements

• SBCL http://www.sbcl.org/
• Emacs http://www.gnu.org/software/emacs/
• Quicklisp http://www.quicklisp.org/
• Sqlite http://www.sqlite.org/

See the online documentation for further details on installation.

News

2023-05-22: v1.7

• Kern import: improved warnings when things go awry (thanks to Bastiaan van der Weij)
• MIDI import: fix errors in note timing
• MIDI import: restore import of midi dynamics and tempo
• MIDI output: write key and time signature information if requested
• MIDI output: preview (PDF and midi play) for database and music objects (thanks to Peter Harrison)
• Database: db: is a nickname for the package qualifier idyom-db:
• Database: copy-datasets allows user to specify composition-ids to include as well as exclude
• Utils: new dataframe package for idyom output (thanks to Peter Harrison)
• Utils: interactive messaging (thanks to Peter Harrison)
• Utils: Cartesian product works with zero arguments (thanks to Bastiaan van der Weij)
• Utils: remove-by-position allows inverting the supplied positions
• Music objects: use initializer for music-sequence to avoid code duplication (thanks to Bastiaan van der Weij)
• Music objects: new package name music-objects + nickname mo:
• Viewpoints: improved support for harmony viewpoints
• Viewpoints: new General Chord Type algorithm for root finding (Giannos & Cambouropoulos, 2021)
• Viewpoints: new Parncutt (1988) root-finding algorithm (thanks to Peter Harrison)
• Viewpoints: new harmony viewpoints (following Harrison (2019) and Sears et al. (2018))
• Viewpoints: new function viewpoint-name-string and argument keep-undefined for viewpoint-sequence{,s} (thanks to Peter Harrison)
• Viewpoints: viewpoint-sequence and viewpoint-sequences now accept a symbol designating a viewpoint (or the viewpoint object itself as before)
• Viewpoints: modify onset alphabets so as not to predict ioi = zero (thanks to Bastiaan van der Weij)
• Viewpoints: new package nickname vp:
• PPM*: replace make-ppm with the more general make-instance (thanks to Bastiaan van der Weij)
• Idyom: Add training-set-size and null-token top-level args to idyom:idyom (thanks to Peter Harrison)
• Idyom: fix bug in formatted output when dur/bioi/deltast/onset had null value
• Idyom: top-level command can now return entropy as well as information content
• Segmentation: add facility to detect boundaries based on both IC and entropy
• Generation: new arguments for output filename and initial context
• Testing: new tests for the Utils and Viewpoints packages (including rhythm and harmony)

2020-04-27: v1.6

• PPM*: update excluded counts are now incremented at the highest-order matching state (reported by Peter Harrison)
• PPM*: with update exclusion enabled, if the highest-order matching state is deterministic and there are no lower-order deterministic states, the update excluded count is used, rather than the full count
• PPM*: virtual states are now allocated correctly when more than one exists on a string transition
• PPM*: new parameter for the top-level idyom function to enable/disable exclusion
• Viewpoints: basic temporal viewpoints (onset, bioi) are now imported correctly from Kern files when there are rests in the first bar
• Viewpionts: the metaccent and posinbar viewpoints are undefined when the time signature is not specified
• Viewpoints: added threaded viewpoints for scale degree and used clearer naming conventions for threaded viewpoints
• Utils: added SD and Pearson correlation
• Data import: don't increment voice number for empty MIDI tracks
• Apps: new top-level function in the segmentation code: (segmentation:idyom-segmentation ...)
• Apps: new similarity modelling function: (similarity:idyom-dissimilarity ...) ;; see (Pearce & Mullensiefen, 2017, https://doi.org/10.1080/09298215.2017.1305419)
• Apps: new app for Krumhansl-Schmuckler key-finding (key-finding:find-keys ...)
• Apps: new control parameters for generation (generation:idyom-generation ...)
• Output: output information gain as the KL divergence between the predictive distribution before and after a note is processed
• Output: when predicting onset, label the distribution with ioi values rather than onset values

2018-07-12: v1.5

• New top-level option to specify the separator to use in output files (thanks to Peter M. C. Harrison)
• new output formatting code for greater efficiency and accuracy, fixing misidentified columns when predicting onset (thanks to Peter M. C. Harrison)
• fixed uneven splits in the creation of resampling sets (thanks to Peter M. C. Harrison)
• new top-level option to specify whether output files are written during viewpoint selection
• fix bug preventing viewpoint selection (reported by Niels Chr. Hansen and Ben Gold)
• viewpoint selection works for max-links = 1 and more than one target viewpoint (reported by Ben Gold)
• fix compilation issue with recent versions of SBCL (reported by Ben Gold, tested on SBCL 1.4.8)
• added timebase parameter in the music-objects code to allow for different timebases to be specified (thanks to Bastiaan van der Weij)
• more efficient implementation of CARTESIAN-PRODUCT (thanks to Bastiaan van der Weij)
• new macro DEFINE-THREADED-VIEWPOINTS for flexibly creating threaded viewpoints
• new utility functions: CUMSUM, RANGE, MD5-SUM-OF-LISTS, COPY-SLOT-VALUES, INITIALISE-UNBOUND-SLOTS, SHUFFLE, ANY-DUPLICATED, RANDOM-SELECT
• midi track numbers are indexed from 1 rather than 0 (for consistency with Kern import)
• avoid negative deltast in midi import when successive notes overlap in a melody
• add random-subset argument to IDYOM-DB:COPY-DATASETS to allow copying random subsets of a specified size
• extended documentation on running IDyOM, especially with respect to viewpoint selection (thanks to Ben Gold)

2016-05-03: v1.4

• new feature: new function IDYOM-DB:COPY-DATASETS for copying and merging datasets
• new feature: new function UTILS:REMOVE-BY-POSITION in utilities
• new feature: new module for information-theoretic detection of grouping boundaries (see Pearce et al., 2010, Perception, 39, 1367-1391).
• new feature: new top-level variable in IDYOM:IDYOM to control whether existing output-files are overwritten (:overwrite nil)
• enhancement: greater randomness in the generation of resampling sets (the random state is now reset in the call to random)
• enhancement: allow the user to pass a random-state as a top-level argument to dataset-generation
• enhancement: add model configuration (STM, LTM, LTM+, BOTH or BOTH+) as a top-level argument in the generation module
• enhancement: the root and earth nodes in the PPM* module are no longer global variables
• enhancement: add a variable to control whether MIDI pitch bends affect pitch in MIDI import module (apply-pitchbend)
• enhancement: ignore empty compositions when importing data
• optimization: improved efficiency in generating the order-1 distribution in the PPM module (thanks to T Hedges)
• optimization: improved efficiency in generating viewpoint sequences
• optimization: generate transition counts more efficiently in the PPM module
• bug-fix: greater elegance in the viewpoint generation macros

2015-03-25: v1.3

• package mtp-admin renamed to idyom-db
• clearer printing of information during viewpoint selection
• add min-links to allow specifying a minimum number of constituent viewpoints in a link
• improvements to handling of voices/spines in Kern import
• ensure that phrase boundaries are correct before tied notes in Kern import
• fix last-element and penultimate-element for empty sequences
• add utility functions for permutations and rotation

2014-12-31: v1.2

• remove verbose output during data import
• linked viewpoint creation is now agnostic to order of constituent viewpoints
• user may omit trailing forward slash when specifying directories

2014-09-27: v1.1

• an extended and improved representation of music objects
• a time protocol for music objects
• extraction of melodies from polyphonic pieces using skyline algorithm
• some generalisation of features in the generation code
• greater efficiency in accessing music-objects from the database
• allow specialisation of viewpoints on user-defined classes
• a caching parameter to control whether LTM resampling sets are cached
• some fixes to Lilypond export (thanks to J Forth)
• installation directories are created if need be (thanks to J Forth)

2014-06-04: v1.0

First public release. I am grateful to Jeremy Gow and David Lewis for their contributions to this release, and also to Niels Chr. Hansen and Roger Dean for user feedback.