Closed Dylan-H-Wang closed 11 months ago
I don't think the warnings or errors are the reason. If no face is detected, cropping simply isn't performed and the whole video is used for re-sizing to, in this case, 72x72. In the case of the warning you got, that's simply a case of multiple faces being detected (even if there is literally only one face in the video) because of poor face detection. I plan on updating this toolbox's face detection significantly sometime by late November after some other conference deadlines of mine, hopefully then many of these errors and warnings that pop up will disappear completely due to a stronger face detection approach being present.
Can you copy/paste the exact config you're using (e.g., with your file paths and all)? Also, how is the UBFC-PHYS folder you are pointing to organized? Does it look like this below?
RawData/
| |-- s1/
| |-- vid_s1_T1.avi
| |-- vid_s1_T2.avi
| |...
| |-- bvp_s1_T1.csv
| |-- bvp_s1_T2.csv
| |-- s2/
| |-- vid_s2_T1.avi
| |-- vid_s2_T2.avi
| |...
| |-- bvp_s2_T1.csv
| |-- bvp_s2_T2.csv
|...
| |-- sn/
| |-- vid_sn_T1.avi
| |-- vid_sn_T2.avi
| |...
| |-- bvp_sn_T1.csv
| |-- bvp_sn_T2.csv
If your UBFC-PHYS dataset is organized in the same way and you are sure you downloaded the UBFC-PHYS videos correctly (e.g., you can open and view the folders aside from s56 and s24, as well as their ground truth files aren't empty), I would recommend double-checking your terminal output. The only other possible thing I can think of at the moment is limited CPU or memory causing threads to somehow silently crash, but I would expect your progress bars in the terminal would look off then too and your program itself would crash at some point.
Here is the config I am using
# use default train/val split with ratio 8/2
BASE: ['']
TOOLBOX_MODE: "train_and_test"
TEST:
USE_LAST_EPOCH: False
DATA:
FS: 35
DATASET: UBFC-PHYS
DO_PREPROCESS: True # if first time, should be true
DATA_FORMAT: NCDHW
DATA_PATH: "./data/UBFC-Phys/raw_data" # Raw dataset path, need to be updated
CACHED_PATH: "./data/UBFC-Phys/preprocessed_data" # Processed dataset save path, need to be updated
FILE_LIST_PATH: "./data/UBFC-Phys/preprocessed_data/data_file_list" # Path to store file lists, needs to be updated
EXP_DATA_NAME: ""
BEGIN: 0.0
END: 1.0
PREPROCESS:
DATA_TYPE: ['Standardized']
DATA_AUG: ['None']
LABEL_TYPE: Standardized
DO_CHUNK: True
CHUNK_LENGTH: 180
CROP_FACE:
DO_CROP_FACE: True
USE_LARGE_FACE_BOX: True
LARGE_BOX_COEF: 1.5
DETECTION:
DO_DYNAMIC_DETECTION: True
DYNAMIC_DETECTION_FREQUENCY : 35
USE_MEDIAN_FACE_BOX: True # This should be used ONLY if dynamic detection is used
RESIZE:
H: 72
W: 72
and I just downloaded the dataset a week before. Below is the data structure which I believe is correct and contents are non-empty.
./data/UBFC-Phys/raw_data
├── s1
│ ├── bvp_s1_T1.csv
│ ├── bvp_s1_T2.csv
│ ├── bvp_s1_T3.csv
│ ├── eda_s1_T1.csv
│ ├── eda_s1_T2.csv
│ ├── eda_s1_T3.csv
│ ├── info_s1.txt
│ ├── selfReportedAnx_s1.csv
│ ├── vid_s1_T1.avi
│ ├── vid_s1_T2.avi
│ └── vid_s1_T3.avi
├── s10
│ ├── bvp_s10_T1.csv
│ ├── bvp_s10_T2.csv
│ ├── bvp_s10_T3.csv
│ ├── eda_s10_T1.csv
│ ├── eda_s10_T2.csv
│ ├── eda_s10_T3.csv
│ ├── info_s10.txt
│ ├── selfReportedAnx_s10.csv
│ ├── vid_s10_T1.avi
│ ├── vid_s10_T2.avi
│ └── vid_s10_T3.avi
├── s11
│ ├── bvp_s11_T1.csv
│ ├── bvp_s11_T2.csv
│ ├── bvp_s11_T3.csv
│ ├── eda_s11_T1.csv
│ ├── eda_s11_T2.csv
│ ├── eda_s11_T3.csv
│ ├── info_s11.txt
│ ├── selfReportedAnx_s11.csv
│ ├── vid_s11_T1.avi
│ ├── vid_s11_T2.avi
│ └── vid_s11_T3.avi
├── s12
│ ├── bvp_s12_T1.csv
│ ├── bvp_s12_T2.csv
│ ├── bvp_s12_T3.csv
│ ├── eda_s12_T1.csv
│ ├── eda_s12_T2.csv
│ ├── eda_s12_T3.csv
│ ├── info_s12.txt
│ ├── selfReportedAnx_s12.csv
│ ├── vid_s12_T1.avi
│ ├── vid_s12_T2.avi
│ └── vid_s12_T3.avi
.....
I have also checked the terminal outputs line by line, the above warning and errors messages were all I get.... and the program did not crash, it iterated all files. However, there is a difference compared to preprocessing other datasets. when processing other datasets, the outputs are
26%|█████████████████████████████████████████▋| 44/168 [58:59<2:46:13, 80.43s/it]
Warning: More than one faces are detected(Only cropping the biggest one.)
26%|█████████████████████████████████████████▋| 45/168 [58:59<2:46:13, 80.43s/it]
but UBFC-Phys procssing gave
Warning: More than one faces are detected(Only cropping the biggest one.)
Warning: More than one faces are detected(Only cropping the biggest one.)
...
Warning: More than one faces are detected(Only cropping the biggest one.)
100%|█████████████████████████████████████| 168/168 [58:59<2:46:13, 80.43s/it]
not sure if this can help to identify the issue?
Your dataset layout seems fine to me, try this config of yours that I modified below:
BASE: ['']
TOOLBOX_MODE: "train_and_test"
TEST:
USE_LAST_EPOCH: False
DATA:
FS: 35
DATASET: UBFC-PHYS
DO_PREPROCESS: True # if first time, should be true
DATA_FORMAT: NCDHW
DATA_PATH: "./data/UBFC-Phys/raw_data" # Raw dataset path, need to be updated
CACHED_PATH: "./data/UBFC-Phys/preprocessed_data" # Processed dataset save path, need to be updated
EXP_DATA_NAME: ""
BEGIN: 0.0
END: 1.0
PREPROCESS:
DATA_TYPE: ['Standardized']
DATA_AUG: ['None']
LABEL_TYPE: Standardized
DO_CHUNK: True
CHUNK_LENGTH: 210
CROP_FACE:
DO_CROP_FACE: True
USE_LARGE_FACE_BOX: True
LARGE_BOX_COEF: 1.5
DETECTION:
DO_DYNAMIC_DETECTION: True
DYNAMIC_DETECTION_FREQUENCY : 35
USE_MEDIAN_FACE_BOX: True # This should be used ONLY if dynamic detection is used
RESIZE:
H: 72
W: 72
All I did was remove the explicit file list path specification and changed the chunk length back to 210. Neither of those changes should matter, but I'm more curious if this would work on your end with these changes that make the situation the same as the original config. If it does work out with the above config, maybe there is a bug of some kind that I can start looking into soon when I have time.
Also, assuming you're using multi-processing, modify the multi_process_quota
in the below line from 8 to 1:
That will be a good sanity check to see if it's something related to the default allocation of multi-processing threads, which in this case might've only caused issues for you with UBFC-PHYS since the videos are quite longer than most other supported datasets (each UBFC-PHYS video is 3 minutes long).
Lastly, I should stress, if you're running into this issue even after trying the two recommendations in my previous reply, 1) double-check your compute resources and see if anything strange might be going on there even when running a single thread (maybe by just monitoring htop
to begin with), and 2) please make sure you're trying this with the main version of the rPPG-Toolbox (e.g., not a heavily modified version of this toolbox that you might have locally or on another fork).
If you are testing this out on a heavily modified version of this toolbox as a part of your own codebase or some other fork, it will be very difficult to help you narrow the issue down and it's best you first test with the main, unaugmented version of the toolbox first.
Thank you for the suggestions.
I only tried to change the multi_process_quota
from 8
to 1
and all outputs became normal. I guess there must be something wrong with the the multi-processing method.
There's definitely improvements we can make with multi-processing, especially with datasets that may require it on certain machines (e.g., UBFC-PHYS) due to their longer raw video lengths. We'll keep that in mind for future improvements where we may handle longer raw video lengths differently when using multi-processing.
Hi,
I used this official config to pre-process UBFC-Phys, but I can only get preprocessed data files for S56 and S24.
There are warnings and errors when running the program
Is this the reason?