ubicomplab / rPPG-Toolbox

rPPG-Toolbox: Deep Remote PPG Toolbox (NeurIPS 2023)
https://arxiv.org/abs/2210.00716
Other
482 stars 117 forks source link

Possible issue/bug in BaseLoader.crop_face_resize #296

Closed boehis closed 3 weeks ago

boehis commented 3 months ago

TLDR: I think in BaseLoader.py function crop_face_resize, the resized frame should be initialized with the correct datatype.

Current:

resized_frames = np.zeros((frames.shape[0], height, width, 3))

New:

resized_frames = np.zeros((frames.shape[0], height, width, 3), dtype=frames.dtype)

First of all, thanks for the amazing toolbox.

I came across this issue bc I noticed that viz_preprocessed_data.ipynb expects frames to be either floats in a [0,1] range or ints in a [0,255] range:

        # Clip frame to a valid range
        if np.issubdtype(frame.dtype, np.floating):
            frame = np.clip(frame, 0, 1)  # Clip float pixel values to [0, 1]
        elif np.issubdtype(frame.dtype, np.integer):
            frame = np.clip(frame, 0, 255)  # Clip integer pixel values to [0, 255]
        else:
            raise ValueError("Unsupported pixel value data type.")

But crop_face_resize will actually turn any array to a float array which for me resulted in my clips being float in range [0,255] which then leads to issues in the visualiser. The visualiser issue is easily fixable, but it made me wonder what the expected and acceptable data format is, specifically, should 'raw' data be in the range [0,255] or [0,1].

I believe this might be reason for #295 and #275, but not sure.

yahskapar commented 3 weeks ago

The original visualization problem should be fixed as soon as #323 is reviewed and merged. Thanks for pointing this out, and please create a new issue if somehow that gets merged and the original visualization problem persists for you.