search

ubicomplab / rPPG-Toolbox

rPPG-Toolbox: Deep Remote PPG Toolbox (NeurIPS 2023)
https://arxiv.org/abs/2210.00716
Other
414 stars 99 forks source link

Performance of Bigsmall #250

Closed 408550969 closed 4 months ago

408550969 commented 4 months ago

Hi, I just learned about rPPG. In the BigSmall paper, it outperforms other methods such as EfficientPhy, but its performance is not as good as EfficientPhy in Remote Biosensing tests: https://github.com/remotebiosensing/rppg?tab=readme-ov-file, which confuses me. I want to know how BIGsmall performs in the rPPG-Toolbox. Also, I want to know what the current SOTA method is, or what the current general method is, like YOLO for face detection.

girishvn commented 4 months ago

Hi,

The contribution of BigSmall is its ability to efficiently multitask physiological signals (rppg, breathing, and facial action). The provided link does not utilize BigSmall as a multitask architecture, but instead only trains it on the single task of rppg. Note, that BigSmall also highlights its contributions when operating on small context window inputs (~3 samples), where-as this repo does report the input frame window length. This makes it hard to compare the results in this repo to the paper results.

Realistically, the high parity in rppg research evaluation results mean that its quite possible that you will see variance in performance for difference methods / models across different training / evaluation regimes and data distributions. This can be seen in our toolbox paper. You will see huge parity between models, and will not observe a complete dominance of one architecture over any other. You can read more about the BigSmall evaluation in this toolbox in our paper as well. Note, the implementation in this toolbox is added to show how to extend this toolbox to a multitask problem, and does not run the same evaluation as the original BigSmall paper.

What do you mean in terms of SOTA method? Do you mean in regards to rPPG, or physiological multitasking? With regards to face detection, neither BigSmall nor any other work implemented in this toolbox claim or are correlated with that field.

Hope this helps!

408550969 commented 4 months ago

What I mean is what sota method is in the rPPG field, Thank you very much for your answer!

408550969 commented 4 months ago

I would also like to ask, currently I have both RGB and infrared cameras. For rPPG, which type of camera performs better, or is there not much difference between the two?

girishvn commented 4 months ago

Hi,

The following two will give you some initial insight into SOTA.

  1. Non-Contrastive Unsupervised Learning of Physiological Signals from Video
  2. PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer

I unfortunately am not extremely familiar with IR ppg, though I am sure there are existing works, and that the multimodal approach likely leads to some performance benefit.

408550969 commented 4 months ago

Thanks!

girishvn commented 4 months ago

Anytime! Hopefully this toolkit will be of use! I'm closing this issue, but feel free to (re)open an issue if something else arises.