search

ZitongYu / STVEN_rPPGNet

Main code of "Remote Heart Rate Measurement from Highly Compressed Facial Videos: an End-to-end Deep Learning Solution with Video Enhancement"
MIT License
155 stars 44 forks source link

Experimental effect #8

Closed yinruonan closed 3 years ago

yinruonan commented 3 years ago

Can the author explain the processing of the input data? I trained rPPGNet separately as described in the paper, but the effect is not very good. The output rPPG signal of the model is almost a straight line. In the experiment, I used the following steps to process the data:

  1. For a single input sample, 64 frames are randomly sampled in the video. The dimension of each video segment is [3, T=64, 128, 128]. After noise reduction, the ECG signal samples 64 pieces of data at the corresponding time. This piece of training data is normalized to make it obey a normal distribution.
  2. For the face mask, use bob.ip.skincolorfilter to set the threshold to 0.3 to get a [T=64,64,64] tensor with values ​​0 and 1. Can someone help me please? The training loss of each sample is between 3-4, and there is no downward trend.
ZitongYu commented 3 years ago

Hi,

I suggest you train on COHFACE or UBFC-RPPG dataset first as they have the BVP ground-truth, which is easier to converge.

Thx. --Zitong

yinruonan notifications@github.com 于2020年12月9日周三 上午6:41写道：

Can the author explain the processing of the input data? I trained rPPGNet separately as described in the paper, but the effect is not very good. The output rPPG signal of the model is almost a straight line. In the experiment, I used the following steps to process the data:

  1. For a single input sample, 64 frames are randomly sampled in the video. The dimension of each video segment is [3, T=64, 128, 128]. After noise reduction, the ECG signal samples 64 pieces of data at the corresponding time. This piece of training data is normalized to make it obey a normal distribution.
  2. For the face mask, use bob.ip.skincolorfilter to set the threshold to 0.3 to get a [64,64] tensor with values ​​0 and 1. Can someone help me please?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ZitongYu/STVEN_rPPGNet/issues/8, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIU3A4LYACBAFSQMJJHJ2G3ST35ZBANCNFSM4US5YN7Q .

yinruonan commented 3 years ago

Thank you! I have got the results of the experiment. When processing data, a function was written incorrectly, so that the result is always a straight line.

Biinzy commented 3 years ago

Can you please post the code for preprocessing the data?