yahskapar
commented
4 months ago Hi @ZacheryZhang,
Can you share 1) your exact dataset folder setup and its structure (e.g., at your path /media/ps2/zexing/UBFC-PHYS) and 2) the exact terminal output when you preprocess the dataset?
My guess is either 1) you need to make sure the dataset is organized as described here or 2) due to the length of the UBFC-PHYS videos, resolution, and ultimately the computational load it takes to process them, it's possible processes as a part of multi-processing are dying on you during the dataset processing. This can happen depending on your exact compute environment and if it's shared with many other users (e.g., on a cluster). If somehow the terminal output doesn't make this obvious, you can try experimenting with lowering the multi_process_quota here (maybe try 4 instead of 8 first, and then 2 or 1 if needed).
ZacheryZhang
Configuration file is as follows, the code has not been modified, reproduce the problem only need to create a new configuration file. ''' BASE: [''] TOOLBOX_MODE: "unsupervised_method" # "train_and_test" or "only_test" or "unsupervised_method" UNSUPERVISED: METHOD: ["ICA", "POS", "CHROM", "GREEN", "LGI", "PBV", "OMIT"] METRICS: ['MAE', 'RMSE', 'MAPE', 'Pearson', 'SNR', 'BA'] DATA: FILTERING: USE_EXCLUSION_LIST: True EXCLUSION_LIST: [ 's3_T1', 's8_T1', 's9_T1', 's26_T1', 's28_T1', 's30_T1', 's31_T1', 's32_T1', 's33_T1', 's40_T1', 's52_T1', 's53_T1', 's54_T1', 's56_T1', 's1_T2', 's4_T2', 's6_T2', 's8_T2', 's9_T2', 's11_T2', 's12_T2', 's13_T2', 's14_T2', 's19_T2', 's21_T2', 's22_T2', 's25_T2', 's26_T2', 's27_T2', 's28_T2', 's31_T2', 's32_T2', 's33_T2', 's35_T2', 's38_T2', 's39_T2', 's41_T2', 's42_T2', 's45_T2', 's47_T2', 's48_T2', 's52_T2', 's53_T2', 's55_T2', 's5_T3', 's8_T3', 's9_T3', 's10_T3', 's13_T3', 's14_T3', 's17_T3', 's22_T3', 's25_T3', 's26_T3', 's28_T3', 's30_T3', 's32_T3', 's33_T3', 's35_T3', 's37_T3', 's40_T3', 's47_T3', 's48_T3', 's49_T3', 's50_T3', 's52_T3', 's53_T3'] SELECT_TASKS: True TASK_LIST: ['T1', 'T2', 'T3'] FS: 35 DATASET: UBFC-PHYS DO_PREPROCESS: True # if first time, should be true DATA_FORMAT: NDHWC DATA_PATH: "/media/ps2/zexing/UBFC-PHYS/" # Raw dataset path, need to be updated CACHED_PATH: "/media/ps2/zexing/rPPG-Toolbox/PreprocessedData/" # Processed dataset save path, need to be updated EXP_DATA_NAME: "" BEGIN: 0.0 END: 1.0 PREPROCESS : DATA_TYPE: ['Raw'] LABEL_TYPE: Raw DO_CHUNK: False CHUNK_LENGTH: 180 CROP_FACE: DO_CROP_FACE: True BACKEND: 'HC' # HC for Haar Cascade, RF for RetinaFace USE_LARGE_FACE_BOX: True LARGE_BOX_COEF: 1.5 DETECTION: DO_DYNAMIC_DETECTION: True DYNAMIC_DETECTION_FREQUENCY : 35 USE_MEDIAN_FACE_BOX: False # This should be used ONLY if dynamic detection is used RESIZE: H: 72 W: 72 INFERENCE: EVALUATION_METHOD: "FFT" # "FFT" or "peak detection" EVALUATION_WINDOW: USE_SMALLER_WINDOW: False # Change this if you'd like an evaluation window smaller than the test video length WINDOW_SIZE: 10 # In seconds ''' The saved preprocessed file is only: “s6_T3_input0.npy”, ‘s6_T3_label0.npy’, ‘s6_T1_label0.npy’, “s6_T1_input0.npy”, ‘s11_T3_input0.npy’, ”s11_T3_label0.npy” , “s47_T1_label0.npy”, “s47_T1_input0.npy”. The downloaded UBFC-PHYS dataset was checked to be complete.