search

johnephillips / auto-music-video

Create a music video automatically using generative AI tools
MIT License
2 stars 1 forks source link

cuts and tempo #3

Open fromtoulouse opened 1 year ago

fromtoulouse commented 1 year ago

hello congratulations for your project. the goal is ambitious. on the tempo part, cutting at the right time is important in a video. how to improve this part of the code ?

johnephillips commented 1 year ago

I implement the code correctly down to the precision of a single frame, assuming the .lrc file is correct. The mistimed transitions in the demo video are caused by the .lrc file I found online being imperfect. In my limited testing I found that the timesteps in most .lrc files online are slightly incorrect.

To improve it further manually, one could improve the times in the .lrc file themselves manually using software like https://www.lrcgenerator.com/. To do this automatically, there would need to be some sort of voice to text processing and then matching the results with the pre-determined lyric chunks in an .lrc file and replace the timings.