cropped_frames variable not found

[EmreOzkose]

Hi, thank you for sharing such a great work.

When I run the Colab demo file, cropped_frames variable not found.

What should be cropped_frames ?

[EmreOzkose]

I think some cells after displaying video cell are missing, because landmarks variable is also missing for Y-Net-gr (landmarks and Graph CNN) part.

[JuanFMontesinos]

Yes you are right. There were some incompatibilities with latest pytorch version and it seems I deleted a whole cell when fixing it. Note that the new cell may take a while to run as the face detector has to be downloaded and runs on cpu.

The code should be fixed. Feel free to re-run. Thanks anyway for pointing it out! In case you are interested the missing cell should be

# Reading and processing the data

from scipy.io.wavfile import read
from VnBSS.utils import normalize_max
from flerken.audio import np_int2float
import imageio
import torch

audio = np_int2float(read('audio.wav')[1][:expected_adur]) # Reading audio and converting int16 -> float 
audio = normalize_max(audio) # Normalizing wrt abs max
reader = imageio.get_reader('video.mp4')
frames = np.stack([x for x in reader])[:expected_vdur]

# Check the video length is adequate
assert len(audio)==expected_adur,f'Audio length is {len(audio)} but should be {expected_adur}'
assert frames.shape[0]==expected_vdur,f'Video length is {frames.shape[0]} but should be {expected_vdur}'

# Face detection, cropping and alignment 
print('Face detection, cropping and alignment ')
print('This may take a while')
cropped_frames,landmarks = compute_landmarks(frames)
imageio.mimwrite('cropped_frames.mp4', [x for x in cropped_frames],fps=25)
inputs = {'src':torch.from_numpy(audio).unsqueeze(0).to(DEVICE)}
display_video('cropped_frames.mp4')