search

openai / jukebox

Code for the paper "Jukebox: A Generative Model for Music"
https://openai.com/blog/jukebox/
Other
7.81k stars 1.4k forks source link

"lyrics-container" as in https://jukebox.openai.com/ samples? #96

Open kcrosley-leisurelabs opened 4 years ago

kcrosley-leisurelabs commented 4 years ago

OK, so how does the timed highlighting trick work on https://jukebox.openai.com/?

I don't understand how to replicate this in one's own jukebox-generated tracks. We get align.png and align.json and lyrics.json in Level 2, but what's the connection? Is the code available for this?

I suspect that align.json is the indices into lyrics.json (right?) but how does one map the timing between these two files. Is there code available for that? (I'm prolly being stupid and it's "obvious" from the browser, but help a brother out here...)

Anyone?

prafullasd commented 4 years ago

The sampling run should also generate an html file with the aligned lyrics under {name}/level_{level}/index.html, which you can open in the browser. The js code inside it shows how it animates the lyrics from align.json and lyrics.json. You can also find the code for it here https://github.com/openai/jukebox/blob/master/jukebox/save_html.py#L92