search

pinterf / mvtools

mvtools plugin for avisynth
155 stars 17 forks source link

Assert in MDegrainN line 224 #47

Closed DTL2020 closed 6 months ago

DTL2020 commented 3 years ago

I frequently get assert debug break in MDegrainN: line 224: assert(!(blockWidth % 8) != 0 && blockWidth != 4);

Today it looks I found the way to reproduce: With 1920x1080 (YV12) source: tr = 6 # Temporal radius super = MSuper (mt=false, chroma=true, pel=1) multi_vec = MAnalyse (super, search=3, searchparam=2, multi=true, delta=tr, chroma=false,mt=false, optSearchOption=0, optPredictorType=0, levels=1) MDegrainN (super, multi_vec, tr, thSAD=400, thSAD2=150, mt=false)

It happens when pel=1 in super, not happens when pel=default. If disable this assert looks nothing bad happens, but may be it is not good ?

pinterf commented 3 years ago

Thanks for the report. Assert was misleading indeed. Reason: bad parenthesis. But I noticed that for blkwidth==4 we'd better read only 4 bytes instead of 8. Then I extended the sse2 support for mod4 as well. That means that now blkwidth=12 is supported as well, as the remaining non mod 8 (mod 4 only) case.

pinterf commented 3 years ago

Theoretically, when we were reading from the very-very last block at blkwidth=4, it could happen that we were reading past the buffer. Because _mm_loadl_epi64 was used even for a four-byte read.

pinterf commented 6 months ago

Fixed in 2c30382aaea57f4224553359debe475b1b2e045c