x64 version crash

[TbtBI]

Hi.

r2728-MT x64/r2664-MT x64 - these version are crashing when I have:

cLeft=2
StackHorizontal(Crop(cLeft, 0, cLeft, 0).mt_convolution("0 1 0", "0 1 1", y=3, u=3 , v=3),Crop(cLeft, 0, 0, 0))

or

cRight=2
StackHorizontal(Crop(0, 0, width-cRight, 0),Crop(width-2*cRight, 0, cRight, 0).mt_convolution("0 1 0", "1 1 0", y=3, u=3, v=3))

When cLeft/cRight > 2, no problem. Removing mt_convolution(...) = no problem.

r2728-MT x86 doesn't crash with cLeft/cRight = 2 + mt_convolution(...).

Here both x64 and x86 doesn't crash:

cTop=2
StackVertical(Crop(0, cTop, 0, cTop).mt_convolution("0 1 1", "0 1 0", y=3, u=3, v=3),Crop(0, cTop, 0, 0))

cBottom=2
StackVertical(Crop(0, 0, 0, height-cBottom),Crop(0, height-2*cBottom, 0, cBottom).mt_convolution("1 1 0", "0 1 0", y=3, u=3 , v=3))

Thanks for everything you're doing.

[pinterf]

Thanks. Crash occurs in mt_convolution. For YV12 the chroma width is 1 in your examples and you asked for processing u and v. Convolution unfortunately does not check if the frame (for any planes) is of at least the convolution size.

Works for YV24 (all planes have width = 2)

BlankClip(120000,width=2,height=480,pixel_type="yv24")
mt_convolution("0 1 0", "1 1 0", y=3, u=3, v=3)

Crash for YV12 (Y plane width = 2, U and V plane width = 1)

BlankClip(120000,width=2,height=480,pixel_type="yv12")
mt_convolution("0 1 0", "1 1 0", y=3, u=3, v=3)

Crash for YV24 (all planes have width = 1)

BlankClip(120000,width=1,height=480,pixel_type="yv24")
mt_convolution("0 1 0", "1 1 0", y=3, u=3, v=3)

[TbtBI]

Thanks for the reply.

Yes, I should post this like masktools2 issue. You can move it (if it's possible), thanks. I'll continue here. If I should open it again in masktools2, let me know.

masktools2 2.2.16/2.2.14: x64 version crash for YV12 (Y plane width = 2, U and V plane width = 1), but x86 version doesn't crash

BlankClip(120000,width=2,height=480,pixel_type="yv12")
mt_convolution("0 1 0", "1 1 0", y=3, u=3, v=3)

x64 version crash for YV24 (all planes have width = 1), but x86 version doesn't crash

BlankClip(120000,width=1,height=480,pixel_type="yv24")
mt_convolution("0 1 0", "1 1 0", y=3, u=3, v=3)

It seems x86 is fine?

[pinterf]

Yes, it's not an Avisynth issue, I will close it here.

x86 did not crash, but it's only pure luck. You cannot do a 3x3 convolution on a width<3 clip. In masktools2 there was no check for this condition (plane dimensions are >= convolution size).