tnfe / FFCreator

A fast video processing library based on node.js (一个基于node.js的高速视频制作库)
http://tnfe.github.io/FFCreator/
MIT License
2.85k stars 405 forks source link

tts和字幕无法同步的问题 #254

Open zjqiudan opened 2 years ago

zjqiudan commented 2 years ago

开始以为是标点符号的问题,但是去掉标点符号也还是无法精确同步(比如一段60秒的视频,在第20秒的时候就会突然出现,这句话字幕没有念完,字幕就跳到下一句话了),请问有什么办法可以解决,谢谢!

drawcall commented 2 years ago

Oh sorry, I've been busy lately. question check later

xsstomy commented 2 years ago

@zjqiudan

这个文件下

https://github.com/tnfe/FFCreator/blob/master/lib/node/subtitle.js

原代码 ` punctuation() { const fps = this.rootConf('fps'); const duration = this.conf.duration; const totalLength = this.conf.text.length; const list = this.text.split(SIGN).filter(elem => elem); let start = 0; this.totalFrames = duration * fps + this.frameBuffer;

forEach(list, text => {
  // text += sign;
  const time = (text.length / totalLength) * duration;
  const frames = start + time * fps;
  this.textList.push({ start, frames, text, draw: false });
  start = frames;
});

} `

修改之后的代码

` punctuation() { const fps = this.rootConf('fps'); const duration = this.conf.duration; const totalLength = this.conf.text.length; const list = this.text.split(SIGN).filter(elem => elem); let start = 0; this.totalFrames = duration * fps + this.frameBuffer;

forEach(list, text => {
  // text += sign;
  const time = ((text.length+1) / totalLength) * duration;
  const frames = start + time * fps;
  this.textList.push({ start, frames, text, draw: false });
  start = frames;
});

}`

主要是修改了这里 const time = ((text.length+1) / totalLength) * duration;

我看了一下源码,标点符号应该也有一个长度的,这里添加了一下,大概能同步吧,相差几十毫秒。 出现以下情况也会音画不同步。

  1. 如果字幕里面一个地方有多个标点符号,音画也不同步。 2.我测试的,如果 2 个字就有标点符号,音画也不同步,如果是一段多长度的字,音画差不多同步。 3.如果音频里面有比较长的静音部分,也会音画不同步。
drawcall commented 2 years ago

Great, please submit a pr.