search

tianyili2017 / HEVC-Complexity-Reduction

Source programs to test the deep-learning-based complexity reduction approach for HEVC, at both intra- and inter-modes.
125 stars 80 forks source link

The purpose of y_image_valid_32 and y_image_valid_16 #9

Open rohan0615 opened 5 years ago

rohan0615 commented 5 years ago

Hi,

According to the code in net_CTU64.py, it is necessary to get the ground truth for 6464, 3232 and 16*16 with the HCPM map as input, so you first reshape the input y into y_image and the get 3 maps y_image_16, y_image_32 and y_image_64. My question is why we still need y_image_valid_32 and y_image_valid_16? What are the meanings of these two terms since they are all used in cross entropy loss and accuracy? Thanks!

Best Regards.

tianyili2017 commented 5 years ago

Hi,

We need these two variables, y_image_valid_32 and y_image_valid_16, to represent whether each element in arrays y_image_32 and y_image_16 is valid. Considering the CU partition in HEVC, the labels for 64x64 CUs always exist, each representing whether one 64x64 CU is split. However, the labels for 32x32 CUs and 16x16 CUs sometimes do not exist. For example, if a 64x64 CU is non-split, the labels for its 32x32 sub-CUs are not valid; and similarly, if a 32x32 CU is non-split, the labels for its 16x16 sub-CUs are not valid, too. So, when considering the ground-truth of 32x32 and 16x16 CUs, we introduce these two variables y_image_valid_32 and y_image_valid_16, which are then flattened into y_flat_valid_32 and y_flat_valid_16 and finally used in loss_32 and loss_16. More details of the loss function in our approach, can be found in Equation (1) of our paper.

Best Regards, Tianyi Li