yahskapar
commented
1 year ago Hi @GaoXu007,
Thanks for using the toolbox!
- That's a strange error that I've never seen before, are you able to reproduce it? It seems that I'm unable to reproduce this. The terminal output I get for using
UBFC_UNSUPERVISED.yamlto get MAE, RMSE, MAPE, and Pearson correlation after using POS is below:
Preprocessing dataset...
5%|██████▏ | 2/42 [00:38<10:50, 16.26s/it]Warning: More than one faces are detected(Only cropping the biggest one.)
7%|█████████▎ | 3/42 [00:38<05:54, 9.08s/it]Warning: More than one faces are detected(Only cropping the biggest one.)
19%|████████████████████████▊ | 8/42 [00:40<00:47, 1.39s/it]Warning: More than one faces are detected(Only cropping the biggest one.)
26%|█████████████████████████████████▊ | 11/42 [01:12<03:30, 6.80s/it]Warning: More than one faces are detected(Only cropping the biggest one.)
64%|██████████████████████████████████████████████████████████████████████████████████▉ | 27/42 [02:21<02:28, 9.89s/it]Warning: More than one faces are detected(Only cropping the biggest one.)
Warning: More than one faces are detected(Only cropping the biggest one.)
83%|███████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 35/42 [02:40<00:28, 4.08s/it]Warning: More than one faces are detected(Only cropping the biggest one.)
Warning: More than one faces are detected(Only cropping the biggest one.)
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 42/42 [02:58<00:00, 4.26s/it]
Total Number of raw files preprocessed: 42
Cached Data Path /playpen-nas-ssd/akshay/UNC_Google_Physio/preprocessed_gold/test/UBFC_SizeW72_SizeH72_ClipLength180_DataTypeRaw_LabelTypeRaw_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len180_unsupervised
File List Path /playpen-nas-ssd/akshay/UNC_Google_Physio/preprocessed_gold/test/DataFileLists/UBFC_SizeW72_SizeH72_ClipLength180_DataTypeRaw_LabelTypeRaw_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len180_unsupervised_0.0_1.0.csv
unsupervised Preprocessed Dataset Length: 42
===Unsupervised Method ( POS ) Predicting ===
100%|███████████████████████████████████████████| 42/42 [04:10<00:00, 5.96s/it]
Used Unsupervised Method: POS
FFT MAE (FFT Label):3.9969308035714284
FFT RMSE (FFT Label):7.5831059532071405
FFT MAPE (FFT Label):3.8622851481891742
FFT Pearson (FFT Label):0.9224921893686093I've only ever seen that error when using ffmpeg for other purposes, so I'm not sure how you managed to see this error within the context of the rPPG-Toolbox. Perhaps OpenCV can also report a similar error in certain scenarios. You may benefit from starting with a fresh copy of this repo, re-running setup.sh, and re-installing the required dependencies as per the README. Please make sure you are also using the latest version of this toolbox by checking the commit history using git log.
Also, can you paste your entire terminal output when doing some step with the toolbox such as pre-processing? I'm a bit confused by the output you have provided since it doesn't seem like you are getting the usual terminal output I would expect when doing pre-processing steps such as face cropping.
-
Those results look good to me and are different from the table you referenced due to numerous changes to the rPPG-Toolbox (code re-factoring, bug fixes, etc) having been made since that table was generated. That table in the arXiv paper is outdated and will be updated in the coming few months. Here's a picture (for reference purposes only) of our new table from the ongoing draft:
-
That doesn't sound normal at all. Please give us more information regarding your computing environment. For example, what kind of GPU do you have?
-
That shouldn't affect the results significantly.
-
Assuming you're using the default FFT evaluation method as specified by the YAML config file, two heart values are calculated for each video - one based on the FFT of the ground truth PPG waveform and another based on the FFT of the FFT of the predicted PPG waveform. Then the final MAE is calculated after taking the mean of the absolute values of differences between the predicted HRs and the label HRs. You can refer to
evaluation/metrics.pyandevaluation/post_process.pyin the toolbox for more details, take a look at thecalculate_metrics()function in particular. -
I believe it is necessary in order to allow for fair comparisons between various baselines in the rPPG-Toolbox. If you wanted to and had enough memory, you could use the raw size by changing those width and height parameters in the YAML config file to a width of 640 and a height of 480 (which is the resolution of UBFC-rPPG videos). My understanding is the main reason it is set to a default of 72x72 is so that it can allow for a fair comparison to various neural methods that rely on a 72x72 pre-processed input, such as TS-CAN.
girishvn
GaoXu007
Prachiiitd
Dear author, thanks for your dedication, I believe such a valuable tool will promote the development of the rPPG field and allow researchers to more effectively build upon existing work.
I have encountered a few issues while using the tool, and I was hoping to seek your guidance in resolving them.
When I follow the steps in readme.md to preprocess the UBFC dataset, I encountered some warnings/errors, but the program can still continue to run. What caused this? Do these warnings have an impact on the experimental results?
Apart from DO_PREPROCESS & DATA_PATH & CACHED_PATH, I followed your default parameters in UBFC_UNSUPERVISED.yaml without making any changes. But the results were not as good as yours (see the figure below). For example, the MAE and RMSE of POS and CHROM in my result: POS: 4.00 7.58
Am I ignoring some details? Or your parameters are different from mine?
CHROM: 3.98 8.72
When using pre-trained models PURE_UBFC_TSCAN_BASIC.yaml, the inference/predict process is very slow, lasting up to 2 hours. Is this normal? I ran the program on the server.
I noticed that the framerates of different videos in the UBFC dataset are inconsistent. For example, subject12’s framerate is 29 and subject23’s framerate is 28.9, but in .yaml, the value of the FS parameter is fixed to 30. Does this affect the results?
Each video will calculate an MAE value, and the MAE in the final result is the average of these video MAEs. Am I correct?This may be a naive question...
Is the Frame Resizing operation (cv2.resize) necessary? What is its role? Can I use a raw frame? How are W=72 and H=72 determined?
I appreciate any guidance you can provide. thanks.