inference and produce sequences for custom videos?

[Tony363]

Hi,

I would like to know whether its possible to perform inference and produce sequences with my own custom videos? What are some of the core preprocessing methods that I have to keep in mind? I don't have rPPG labels for my videos tho.

Tony,

[zizheng-guo]

Inferencing your own custom video requires some code tweaking. It's essential to incorporate face detection and cropping during preprocessing, with the algorithms located in the 'baseloader', which can be directly invoked. Additionally, normalization could also impact the model's performance. Therefore, it's preferable to maintain the same preprocessing procedures as during training.

For your reference:

1. Add a dataloader for your video, removing the label processing part (you can simplify by setting labels to a fixed value).
2. Adjust the output in the trainer's test or metrics.

[Tony363]

What should I do with the preprocess_dataset_subprocess function? Something like the below works?

` def preprocess_dataset_subprocess(self, data_dirs, config_preprocess, i, file_list_dict): """ invoked by preprocess_dataset for multi_process.""" filename = os.path.split(data_dirs[i]['path'])[-1] saved_filename = data_dirs[i]['index']

    # Read Frames
    if 'None' in config_preprocess.DATA_AUG:
        # Utilize dataset-specific function to read video
        frames = self.read_video(
            os.path.join(data_dirs[i]['path'],"vid.avi"))
    elif 'Motion' in config_preprocess.DATA_AUG:
        # Utilize general function to read video in .npy format
        frames = self.read_npy_video(
            glob.glob(os.path.join(data_dirs[i]['path'],'*.npy')))
    else:
        raise ValueError(f'Unsupported DATA_AUG specified for {self.dataset_name} dataset! Received {config_preprocess.DATA_AUG}.')

    # Read Labels
    bvps = self.generate_pos_psuedo_labels(frames, fs=self.config_data.FS)

    frames_clips, bvps_clips = self.preprocess(frames, bvps, config_preprocess)
    input_name_list, label_name_list = self.save_multi_process(frames_clips, bvps_clips, saved_filename)
    file_list_dict[i] = input_name_list`

or just set bvps to an arbitrary array?

[zizheng-guo]

Yeah, like this:

def preprocess_dataset_subprocess(self, data_dirs, config_preprocess, i, file_list_dict):
      saved_filename = data_dirs[i]['index']
      # Read Frames
      frames = self.read_video(data_dirs[i]['path'])
      bvps = np.ones(frames.shape[0])
      frames_clips, bvps_clips = self.preprocess(frames, bvps, config_preprocess)
      input_name_list, label_name_list = self.save_multi_process(frames_clips, bvps_clips, saved_filename)
      file_list_dict[i] = input_name_list

[Tony363]

Thanks for the function! I am currently facing a multiprocessing deadlock. I have some ideas how to fix it but would you have any thing immediate? I consider this issue closed? I forked this repo. I could make a pull request once I fix the multiprocessing deadlock?