search

tianyili2017 / HEVC-Complexity-Reduction

Source programs to test the deep-learning-based complexity reduction approach for HEVC, at both intra- and inter-modes.
124 stars 80 forks source link

Could you tell me why LSTM_MAX_LENGTH is 1 in HM-16.5_TEST_LDP? #11

Open Liam-zzZ opened 5 years ago

Liam-zzZ commented 5 years ago

Hi, in train mode, LSTM_MAX_LENGTH is 20. But in HM-16.5_TEST_LDP, LSTM_MAX_LENGTH is 1 . Could you tell me why LSTM_MAX_LENGTH is different?

tianyili2017 commented 5 years ago

Hi, Sorry for late reply. In train mode, the 20 residual CTUs on all time steps together, are fed into the LSTM structure at one time. So, LSTM_MAX_LENGTH = 20. However, in test mode, the frames are encoded one by one, that is to say, only if the current frame is encoded, the residual CTUs for the next frames can be available. The 20 residual CTUs from 20 frames can not be obtained together at once, and thus the LSTM should be run step-by-step. Here, on each step, only one LSTM cell needs to be run forward, whose input contains one residual CTU and the LSTM state for the last frame, and the output contains the splitting probability of 21 CUs (HCPM in paper) and the updated LSTM state for the current frame. The above procedure explains why LSTM_MAX_LENGTH = 1 in test mode.