search

hecomi / uLipSync

MFCC-based LipSync plug-in for Unity using Job System and Burst Compiler
https://tips.hecomi.com/entry/2022/01/30/152519
MIT License
1.07k stars 128 forks source link

Lipsync won't play in Play mode, when AudioSource is connected to Timeline #3

Open lehtiniemi opened 3 years ago

lehtiniemi commented 3 years ago

Using the following workflow, I can't get uLipSync play in Play mode: -create timeline -setup uLipSync object -create Audio track on Timeline -drag the audio source to timeline Audio Track -drag an audio clip to the track -> At this point, the lipsync plays correcly when I play timeline (I have added [ExecuteAlways] to uLipSync.cs to be able to use it in Editor mode

-now hit Play to play the timeline in Play mode.

-> The mouth doesn't move at all. The callback on uLipSync triggers and OnAudioFilterRead seems to trigger as well on console (I put a debug there). But the LipSyncInfo.phoneme is empty or null in the callback.

I am only using the lipsync callback for analysis and not the mouth blending component - I do that in my own script.

What could could cause this incompability with Timeline?

hecomi commented 3 years ago

It seems that if you specify the game object to play the sound, it will play during play mode without any problem.

image

lehtiniemi commented 3 years ago

Related to the answer you wrote in #4 , did you get it to work after all? OnAudioFilterRead didn't seem to work properly in playmode when using Timeline? In our experiments, we kept on getting empty phonemes from uLipSync and figured it's related to Timeline and OnAudioFilterRead. Or did you mean something different with this solution?

fumiodev commented 3 years ago

@lehtiniemi I am able to work around this problem by using Signal. On the Timeline Window, I create a Signal Track and assign a GameObject with an AudioSource. Then I add Signal to Signal Receiver, and set the following 2 Reactioms:

  1. AudioSource.Clip(AudioClip)
  2. AudioSource.Play() (I try to use AudioSource.PlayOneShot(AudioClip), but it became same result in this issue)

I hope that this information is helpful to you.