pyannote / pyannote-audio

Neural building blocks for speaker diarization: speech activity detection, speaker change detection, overlapped speech detection, speaker embedding
http://pyannote.github.io
MIT License
6.45k stars 792 forks source link

Declaring pipeline variables causes torch.jit model fail to execute #1756

Open WelkinYang opened 2 months ago

WelkinYang commented 2 months ago

Tested versions

pyannote.audio 3.1

System information

Ubuntu A800 GPU pyannote.audio 3.1

Issue description

Declaring pipeline variables causes torch.jit model fail to execute

minimal code to reproduce

import os
import numpy as np

import torch
from pyannote.audio import Pipeline

sd_model = Pipeline.from_pretrained("pyannote/speaker-diarization-3.1", use_auth_token="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").to(torch.device('cuda'))

device = torch.device('cuda')
local_file = 'model.pt'

if not os.path.isfile(local_file):
    torch.hub.download_url_to_file('https://models.silero.ai/denoise_models/sns_latest.jit',
                                   local_file)

model = torch.jit.load(local_file)
model.to(device)

a = torch.rand(1, 48000).to(device)
out = model(a)            

with the line "sd_model = Pipeline.from_pretrained("pyannote/speaker-diarization-3.1", use_auth_token="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").to(torch.device('cuda'))" this code will fail to execute, the error message is: RuntimeError: The following operation failed in the TorchScript interpreter. Traceback of TorchScript (most recent call last): RuntimeError: The following operation failed in the TorchScript interpreter. Traceback of TorchScript (most recent call last): RuntimeError: default_program(30): error: name followed by "::" must be a class or namespace name void kernel_0(IndexType totalElements, const TensorInfo<c10::complex,2> t0, const TensorInfo<float,1> t1, const TensorInfo<float,1> t2 ) { ^

default_program(30): error: expected an identifier void kernel_0(IndexType totalElements, const TensorInfo<c10::complex,2> t0, const TensorInfo<float,1> t1, const TensorInfo<float,1> t2 ) { ^

default_program(30): error: invalid combination of type specifiers void kernel_0(IndexType totalElements, const TensorInfo<c10::complex,2> t0, const TensorInfo<float,1> t1, const TensorInfo<float,1> t2 ) { ^

default_program(30): error: too few arguments for class template "TensorInfo" void kernel_0(IndexType totalElements, const TensorInfo<c10::complex,2> t0, const TensorInfo<float,1> t1, const TensorInfo<float,1> t2 ) { ^

default_program(30): error: expected a type specifier void kernel_0(IndexType totalElements, const TensorInfo<c10::complex,2> t0, const TensorInfo<float,1> t1, const TensorInfo<float,1> t2 ) { ^

default_program(34): error: name followed by "::" must be a class or namespace name c10::complex t0_buf[4]; ^

default_program(34): error: expected an identifier c10::complex t0_buf[4]; ^

default_program(34): error: expected a ";" c10::complex t0_buf[4]; ^

default_program(52): error: identifier "t0" is undefined size_t t0_dimIndex1 = t0_linearIndex % t0.sizes[1]; ^

default_program(73): error: identifier "t0_buf" is undefined for(int i = 0; i<4; i++) t0_buf[i] = t0.data[t0_offset + i]; ^

default_program(78): error: name followed by "::" must be a class or namespace name

but the code will work well when we remove the line which declare the pipeline. it's so weird and amazing.

Minimal reproduction example (MRE)

https://colab.research.google.com/github/pyannote/pyannote-audio/blob/develop/tutorials/MRE_template.ipynb#scrollTo=SCS4Nr0vgzoc&line=1&uniqifier=1

clement-pages commented 1 month ago

Hey @WelkinYang , could you please provide a MRE of your issue (in google colab), so we can reproduce it ?