ROCm / AMDMIGraphX

AMD's graph optimization engine.
https://rocm.docs.amd.com/projects/AMDMIGraphX/en/latest/
MIT License
184 stars 82 forks source link

Conv accuracy fail #3501

Open shivadbhavsar opened 1 day ago

shivadbhavsar commented 1 day ago

This program (obtained from a torch_migraphx UT) fails driver verify:

p = migraphx.program()
m = p.get_main_module()
x_0 = m.add_literal(migraphx.generate_argument(migraphx.shape(type="float_type", lens=[16]), 0))
x_1 = m.add_literal(migraphx.generate_argument(migraphx.shape(type="float_type", lens=[16, 3, 5, 5, 5]), 1))
p_input = m.add_parameter("input", migraphx.shape(type="float_type", lens=[8, 3, 50, 50, 100]))
x_3 = m.add_instruction(migraphx.op("convolution", padding=[4,4,4,4,4,4], stride=[1,1,1], dilation=[2,2,2]), [p_input, x_1]) # migraphx.shape(type="float_type", lens=[8, 16, 50, 50, 100])
x_4 = m.add_instruction(migraphx.op("broadcast", axis=1, out_lens=[8,16,50,50,100]), [x_0]) # migraphx.shape(type="float_type", lens=[8, 16, 50, 50, 100], strides=[0, 1, 0, 0, 0])
x_5 = m.add_instruction(migraphx.op("add"), [x_3, x_4]) # migraphx.shape(type="float_type", lens=[8, 16, 50, 50, 100])
m.add_return([x_5])
shivadbhavsar commented 1 day ago

Passes verify with MIGRAPHX_DISABLE_MLIR=1, possible mlir issue? Trying to bisect problem commit

shivadbhavsar commented 1 day ago

Looks like in the past this convolution was never using mlir. It still passes when forced to use miopen. Even if i go back to the 6.2.2 release branch, I cannot get this to pass with mlir. Probably need someone from mlir to look at this one

shivadbhavsar commented 1 day ago

Ahh heres the PR that enables mlir for this: https://github.com/ROCm/AMDMIGraphX/pull/3406. This has never passed since that change using any mlir commit