search

pinterf / masktools

MaskTools v2 fork
Other
48 stars 11 forks source link

float clip crash in mt_edge #12

Closed realfinder closed 4 years ago

realfinder commented 4 years ago

colorbars() converttoyv12 convertbits(32) mt_edge(thY1=0,thY2=255,mode="min/max",U=1,V=1) convertbits(8)

pinterf commented 4 years ago

Could not reproduce, I have tried even with different SetMaxCPU values, and differently sized clip (ColorBarsHD)

realfinder commented 4 years ago

ok, using image

with x64 crash with image

but x86 fine! also with x64 in image

also fine

pinterf commented 4 years ago

"Illegal instruction" is important. Not the usual overaddressing C0000005. That processor knows AVX but not AVX2. I have to find an AVX2 instruction. Can you test it with SetMaxCPU("SSE4.1"), I suppose the error is gone then?

realfinder commented 4 years ago

yes with SetMaxCPU("SSE4.1") seems work fine

pinterf commented 4 years ago

Found, mt_edge module could use AVX2 instructions. Also mt_logic was suspicious. Usually 32 bit float SIMD requires only AVX but the optimizer may change some instructions into AVX2. Since the full build takes 20-30 minutes (did I say how I like masktools with thousand dll versions? :) ), I'll leave the machine work and will prepare the new pack tomorrow.

pinterf commented 4 years ago

New build, fix of illegal instruction, please try if it is O.K. https://drive.google.com/file/d/1dGh7deLE9mvUZtCYcrqxa4OTaWzpOEx8/view?usp=sharing

realfinder commented 4 years ago

yes, that work, thanks