collabora / WhisperFusion

WhisperFusion builds upon the capabilities of WhisperLive and WhisperSpeech to provide a seamless conversations with an AI.
1.45k stars 101 forks source link

Race condition in text_to_speech func #47

Open felipemuhamed opened 3 months ago

felipemuhamed commented 3 months ago

I believe there might be a race condition in the following section of the code:

# clip audio if the current chunk exceeds 30 seconds, this basically implies that
# no valid segment for the last 30 seconds from whisper
if self.frames_np[int((self.timestamp_offset - self.frames_offset)*self.RATE):].shape[0] > 25 * self.RATE:
    duration = self.frames_np.shape[0] / self.RATE
    self.timestamp_offset = self.frames_offset + duration - 5

samples_take = max(0, (self.timestamp_offset - self.frames_offset)*self.RATE)
input_bytes = self.frames_np[int(samples_take):].copy()
duration = input_bytes.shape[0] / self.RATE
if duration < 0.4:
    # If the audio duration is short, release the lock and wait
    self.lock.release()
    time.sleep(0.01)    # 5ms sleep to wait for some voice active audio to arrive
    continue

You are accessing a shared resource between threads without proper synchronization in this section. Could you please review this code?

makaveli10 commented 2 months ago

@felipemuhamed thanks for reporting the issue. We are taking a look at this.