InterDigitalInc / CompressAI

A PyTorch library and evaluation platform for end-to-end compression research
https://interdigitalinc.github.io/CompressAI/
BSD 3-Clause Clear License
1.21k stars 232 forks source link

When I wanted to migrate that program to sequence signal data, I found that @register_model("mbt2018") The compress and decompress methods of class JointAutoregressiveHierarchicalPriors are very complex #294

Open wei-mei opened 5 months ago

wei-mei commented 5 months ago

When I wanted to migrate that program to sequence signal data, I found that The compress and decompress methods of class

@register_model("mbt2018")
JointAutoregressiveHierarchicalPriors

are very complex. Why is it so complex compared to the compress function and the decompress function of the previous models?

s = 4  # scaling factor between z and y
kernel_size = 5  # context prediction kernel size
padding = (kernel_size - 1) // 2

y_height = z_hat.size(2) * s
y_width = z_hat.size(3) * s
y_q = self.gaussian_conditional.quantize(y_crop, "symbols", means_hat)

Also, do I have to train the aux loss very small to get similar results for the forward function and the compress and decompress functions?

Thank you very much for your answer!

YodaEmbedding commented 5 months ago

The autoregression portion requires a loop of many steps at runtime. This is because the needed information for decoding only becomes available as the tensor is being decoded pixel-by-pixel from top-left to bottom-right. In contrast, during training, all the information about the tensor is immediately available, so that "decoding" can be done in one small step.

Autoregressive loop
The above operations are repeated in a loop done in raster-scan order (top-left to bottom-right). Previously decoded (purple) pixels are used to help predict the current pixel (yellow).

Also, previous models contain some amount of code for runtime decoding too, but it's hidden inside the EntropyBottleneck and GaussianConditional classes.