MELONS: generating melody with long-term structure using transformers and structure graph

[tatsuropfgt]

MELONS: generating melody with long-term structure using transformers and structure graph [ICASSP22]

Abstract

• conditional long melody generation based on a graph representation of music structure
• factor melody generation into 2 sub-problems: structure generation and structure conditional melody generation
  • these sub-problems are solved by the transformer which can generate long consistent sequences

Method

from the above melody, the structure extraction module generates the following relation structure graph

then using the above graph, 3 generation module is trained

• Structure Generation Module
  • Compound word transformer
  • token triples like {(i1, j1, t1),(i2, j2, t2), · · · ,(in, jn.tn)},
  • bar, bar, relation
• Unconditional/Conditional Melody Generation Module
  • event-based music token representation
  • {(content of the melody context), (content of the related bar), [relation type],(content of the target bar)}

These models are implemented on the linear transformer model

when generation, the Structure Generation Module first generates the structure graphs. next, the Unconditional/Conditional Melody Generation Module generates the melody in one bar. these steps are repeated(Auto-refression).

[tatsuropfgt]

memo

One bar can have only 1 relation