why the prediction of the 7x7 ac coefficients needs "abs" ?

[xiaoxiongli]

Dear danielrh: @danielrh

for the prediction of 7x7 ac coefficients of the block, in the paper I find that the prediction method is like this: but in the lepton code, I find that the predction method is just like below:

So I feel confuse that why we need to have a "abs" operation. I change this implement to what the paper describe, but the result get worse.

Could you please help to explain this? ^_^

[danielrh]

You indeed have found a bug in the paper--I'll try to update the arxiv version when I get a chance; the lepton algorithm uses only the magnitude of the coefficients to predict the next. It's generally accepted that DCT coefficients are difficult to predict in frequency space, however there have been some advances in the area including https://www.spiedigitallibrary.org/conference-proceedings-of-spie/6497/64970L/Prediction-of-signs-of-DCT-coefficients-in-block-based-lossy/10.1117/12.713872.short which takes place in pixel space and https://ieeexplore.ieee.org/document/6738915

However, lepton does not make use of these advanced techniques and chooses a simple prior for the sign bits