search

ubicomplab / rPPG-Toolbox

rPPG-Toolbox: Deep Remote PPG Toolbox (NeurIPS 2023)
https://arxiv.org/abs/2210.00716
Other
458 stars 111 forks source link

rPPG-Toolbox on webcam #249

Closed PalmDomenico closed 7 months ago

PalmDomenico commented 7 months ago

hi, how can I use one of the methods developed to extract ppg from the webcam?

girishvn commented 7 months ago

Hi @PalmDomenico,

There are a few needed steps:

  1. read in your video into some numpy array
  2. Write a function similar to preprocess in BaseLoader.py. The original function expects a ppg label (bvps), you can adapt the function just for input frames.
  3. This new function should crop_face_resize (defined in BaseLoader.py) to face crop / resize your video to the expected size (generally 72x72).
  4. This new function should then normalize your data.
  5. You can find configurations for both cropping and normalization in existing config files. I suggest configs/infer_configs/PURE_UBFC-rPPG_DEEPPHYS_BASIC.yaml as a start as this input can be used for a number of networks (namely DeepPhys and TSCAN).

Take the processed input (it can still be in the form of a numpy array), and write code to load a existing model checkpoint from ./final_model_release. I suggest starting with PURE_DeepPhys.pth. You can then pass the data through the model. If you need reference for how this is done I would inspect the training scripts (start with neural_methods/trainer/DeepPhysTrainer.py)

Hope this helps.

girishvn commented 7 months ago

Closing this issue. Feel free to open again if need be.

PalmDomenico commented 7 months ago

What do the input models take? single frames or multiple frames?