Closed tuanad121 closed 5 years ago
In the inference, you do not need to prepare anchor
...
The general inference process is as follows:
1) Prepare data:
Get the audio, and extract MFCC features, just similar the training preprocess. Please view details in function extract_features
which is located in pre_process.py
2) Predict on model :
embedding = model.predict(input)
Then you will get 512-dimensional embedding vector.
3) Use this embedding vector to verification or classification tasks.
Suppose there are two utterances that need to be verified if they were made by the same person.
We first get the embedding vector for 2 utterances
emb1= model.predict(utt1); emb2= model.predict(utt2)
If the similarity between the two embeddings is greater than a certain threshold (e.g. 0.5), we can determine that the two utterances originated from the same person.
utt1 and utt2 has the same speaker if sim(emb1,emb2)>threshold
Thanks for your quick response. Much appreciated. I will try as you suggested. ^^
cool, I can it. Here's the script:
import constants as c
from pre_process import extract_features
from models import convolutional_model
from utils import get_last_checkpoint_if_any
from scipy.io.wavfile import read
import numpy as np
def clipped_audio(x, num_frames=c.NUM_FRAMES):
if x.shape[0] > num_frames + 20:
bias = np.random.randint(20, x.shape[0] - num_frames)
clipped_x = x[bias: num_frames + bias]
elif x.shape[0] > num_frames:
bias = np.random.randint(0, x.shape[0] - num_frames)
clipped_x = x[bias: num_frames + bias]
else:
clipped_x = x
return clipped_x
if __name__ == '__main__':
model = convolutional_model()
last_checkpoint = get_last_checkpoint_if_any(c.CHECKPOINT_FOLDER)
print(last_checkpoint)
model.load_weights(last_checkpoint)
_, utt1 = read('demo/87-121553-0002.wav')
utt1 = utt1 / (2**15 - 1)
feat1 = extract_features(utt1)
feat1 = clipped_audio(feat1)
feat1 = feat1[np.newaxis, ...]
_, utt2 = read('demo/103-1240-0002.wav')
utt2 = utt2 / (2**15 - 1)
feat2 = extract_features(utt2)
feat2 = clipped_audio(feat2)
feat2 = feat2[np.newaxis, ...]
print(feat1.shape, feat2.shape)
emb1 = model.predict(feat1)
emb2 = model.predict(feat2)
# similarity
mul = np.multiply(emb1, emb2)
s = np.sum(mul, axis=1)
print(s)
I've tried the above code but each time i get a different similarity value. How is this possible?
I've tried the above code but each time i get a different similarity value. How is this possible?
Hi,
I think the main reason is the random clip in def clipped_autio
in the code. The audio was random clipping before feeding into the model if the audio is too long.
You can modify the def clipped_autio
to a deterministic clip. (clip the deterministic middle frames of the long audio, or clip different parts of the long audio then average the different results).
Or fixed the random seed during running the code, according to reproducible-results-neural-networks-keras.
That's cool! Thank you very much.
Hi there, I'm looking for how to embed data of one speaker with the pre-trained model. In the training process, the input data has
anchor, positive, and negative speaker
. Each speaker has 32 sentences with 160 frames for each sentence. I wonder how we use the model to embed one speaker. Do we need to prepareanchor, positive and negative speaker
as in training? Thanks for spend your time on my question. ^^