LipSync without spectrum file

It might be possible to implement lipsync without using our current method of spectrum files:

Advantages of not using spectrum files:

Reduce the number of files on disk - currently there is one spectrum file per each voice file
No need to regenerate spectrum files if new voice files are added
Since we don't have access to the tool which was used to generate the spectrum files, we'd need to re-write it again (not sure if this has been done already), although it shouldn't be that complicated
Can adjust parameters on-the-fly (sensitivity etc) which would allow for easier tweaking

See this unity question for info on doing this..

edit: I did a quick test, and you can get the audio data - I did a quick test which prints every 1000th sample of channel 0, when inserted into the WaitForLoad() function in AudioLaterUnity.cs:

if(audioClip == null)
{
    Debug.Log($"Audio {filename}: audioClip is null!");
}
else
{
    Debug.Log($"Audio {filename}: audioClip is not null! Samples: {audioClip.samples} Channels: {audioClip.channels}");
    float[] samples = new float[audioClip.samples * audioClip.channels];
    audioClip.GetData(samples, 0);

    System.Text.StringBuilder sb = new System.Text.StringBuilder();
    sb.Append("[");
    for (int sample = 0; sample < 100*1000; sample += 1000)
    {
        int stride = audioClip.channels;
        int index = sample * stride;
        if (index > samples.Length)
        {
            break;
        }

        sb.Append($", {samples[index]}");
    }
    sb.Append("]");

    Debug.Log($"Audio {filename}: Every 1000 samples (tot 100 samples) CH1: {sb}");
}

07th-mod / higurashi-assembly

LipSync without spectrum file #62