Entropy Model is different between training and encoding

[PnZheng]

When i run the code in test/trainDDP_Full.py, i found EntropyNet model is different from EntEncoder/EntDecoder in pseudo_codec.py.
Will there need extra train for entropy model ?

[limuhit]

It's a just another implementation for entropy coding. No extra training are needed. In training, we conduct parallel context-based predition for all the codes. However, in coding, we should serialization this prediction according to the coding order. Thus, the operators in trainDDP_Full.py is the parallel implementation, while the operators in pseudo_codec.py is the serialization version. They share the same prarameters.

[PnZheng]

Thank you, It is a great method. But I have no idea how to convert the parallel version to serialization version, Would you mind open the related code ?

[limuhit]

All the codes are given in Github. Please check the PCONV_operator and extension directories.

[PnZheng]

Thank you，I have solved the problem. But when i test the bitrate-VPSNR, it will more less 3-4dB than value shown in paper. The code is training in GeForce 3090, will that be some problem incur?

[limuhit]

The Pseudo Convolution Network needs initialization. Please check if you follow the following steps:

1. Initialize the encoder and decoder with the normal 2D convolution for only distortion loss. This step produces the "mse_192.pt" used in "trainDDP_base.py". You may produce your own "mse_192.pt" by changing the "PCONV_operator/base.py" line 31 from "rt=1." to "rt=0.". Then, change the opt flag in "trainDDP_base.py" line 169 to "default=False". Run "trainDDP_base.py" and save the best model as "mse_192.pt".
2. Fineturn the parameter of encoder and decoder from step 1 with the pseudo convolution for only distortion loss with "trainDDP_base.py", remember to set the opt flag to True.
3. Initialize the encoder and decoder with the parameter from step 2, and train the whole model with respect to rate-distortion loss with "trainDDP_Full.py".

[PnZheng]

Thanks for your kind reply。Exclude the step “rt=0.”and “opt=false”，others is same. Will it influence heavily？ And the ''aplha'' in ''trainDDP_Full.py'' is set 3.0,10.0and16.0 in the validim=56， the final vpsnr is between 27-28dB in training .

[limuhit]

By the way, step 2 also works to initialize parameters for different code channel settings, i.e., 192, 112, 56.

In "trainDDP_Full.py", setting different alpha to produce models for different bitrates. You may change the bitrate by code channels and the alpha. channel 192 for the high bitrate region. channel 112 for medial and channel 56 for the low bitrate regions.

[limuhit]

Directly training on pseudo convolution is unstable. It is quite important to initialize with the normal convolution. You could view this step as a kind of Knowledge Distillation.

[PnZheng]

yep，i have changed the dim to 56，112，192 before. The final vpsnr is about 28，29，30 dB which is deelpy different with the paper. It confuse me for months.

[PnZheng]

oh, i see. thank you for your reply. I will folllw the step to train one more.

[limuhit]

This is the key problem. I met it at the beginning.