search

migraphx-benchmark / AMDMIGraphX

AMD's graph optimization engine.
https://rocmsoftwareplatform.github.io/AMDMIGraphX/doc/html/
MIT License
0 stars 1 forks source link

NonMaxSuppression inaccuracies #110

Open attila-dusnoki-htec opened 1 year ago

attila-dusnoki-htec commented 1 year ago

Failing tests:

attila-dusnoki-htec commented 1 year ago
FAIL: test_nonmaxsuppression_center_point_box_format_cpu (__main__.OnnxBackendNodeModelTest) ``` ====================================================================== FAIL: test_nonmaxsuppression_center_point_box_format_cpu (__main__.OnnxBackendNodeModelTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/onnx/backend/test/runner/__init__.py", line 290, in device_test_func return test_func(*args, device=device, **kwargs) File "/usr/local/lib/python3.8/dist-packages/onnx/backend/test/runner/__init__.py", line 467, in run self.assert_similar_outputs( File "../test/py/onnx_backend_test.py", line 59, in assert_similar_outputs np.testing.assert_allclose(ref_outputs[i], File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 1530, in assert_allclose assert_array_compare(compare, actual, desired, err_msg=str(err_msg), File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 763, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=0.001, atol=1e-05 Program = module: "main" score_threshold = @param:score_threshold -> float_type, {1}, {1}, target_id=0 iou_threshold = @param:iou_threshold -> float_type, {1}, {1}, target_id=0 max_output_boxes_per_class = @param:max_output_boxes_per_class -> int64_type, {1}, {1}, target_id=0 scores = @param:scores -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 boxes = @param:boxes -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @5 = nonmaxsuppression[center_point_box=1,use_dyn_output=0](boxes,scores,max_output_boxes_per_class,iou_threshold,score_threshold) -> int64_type, {6, 3}, {3, 1}, target_id=0 @6 = @return(@5), target_id=0 Compiled program = module: "main" @0 = check_context::migraphx::gpu::context -> float_type, {}, {}, target_id=0 @1 = hip::hip_allocate_memory[shape=int8_type, {176}, {1},id=main:scratch] -> int8_type, {176}, {1}, target_id=0 score_threshold = @param:score_threshold -> float_type, {1}, {1}, target_id=0 @3 = load[offset=160,end=164](@1) -> float_type, {1}, {1}, target_id=0 @4 = hip::copy_to_gpu(score_threshold,@3) -> float_type, {1}, {1}, target_id=0 boxes = @param:boxes -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @6 = load[offset=0,end=96](@1) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @7 = hip::copy_to_gpu(boxes,@6) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 scores = @param:scores -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 @9 = load[offset=96,end=120](@1) -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 @10 = hip::copy_to_gpu(scores,@9) -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 max_output_boxes_per_class = @param:max_output_boxes_per_class -> int64_type, {1}, {1}, target_id=0 @12 = load[offset=128,end=136](@1) -> int64_type, {1}, {1}, target_id=0 @13 = hip::copy_to_gpu(max_output_boxes_per_class,@12) -> int64_type, {1}, {1}, target_id=0 @14 = load[offset=144,end=148](@1) -> float_type, {1}, {1}, target_id=0 iou_threshold = @param:iou_threshold -> float_type, {1}, {1}, target_id=0 @16 = hip::copy_to_gpu(iou_threshold,@14) -> float_type, {1}, {1}, target_id=0 @17 = hip::copy_from_gpu(@4) -> float_type, {1}, {1}, target_id=0 @18 = hip::copy_from_gpu(@7) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @19 = hip::copy_from_gpu(@10) -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 @20 = hip::copy_from_gpu(@13) -> int64_type, {1}, {1}, target_id=0 @21 = hip::copy_from_gpu(@16) -> float_type, {1}, {1}, target_id=0 @22 = hip::sync_stream(@18,@19,@20,@21,@17) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @23 = load[offset=0,end=144](@1) -> int64_type, {6, 3}, {3, 1}, target_id=0 @24 = nonmaxsuppression[center_point_box=1,use_dyn_output=0](@22,@19,@20,@21,@17) -> int64_type, {6, 3}, {3, 1}, target_id=0 @25 = hip::copy_to_gpu(@24,@23) -> int64_type, {6, 3}, {3, 1}, target_id=0 @26 = hip::copy_from_gpu(@25) -> int64_type, {6, 3}, {3, 1}, target_id=0 @27 = hip::sync_stream(@26) -> int64_type, {6, 3}, {3, 1}, target_id=0 @28 = @return(@27), target_id=0 (shapes (3, 3), (6, 3) mismatch) x: array([[0, 0, 3], [0, 0, 0], [0, 0, 5]]) y: array([[0, 0, 3], [0, 0, 0], [0, 0, 5],... ```
FAIL: test_nonmaxsuppression_flipped_coordinates_cpu (__main__.OnnxBackendNodeModelTest) ``` ====================================================================== FAIL: test_nonmaxsuppression_flipped_coordinates_cpu (__main__.OnnxBackendNodeModelTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/onnx/backend/test/runner/__init__.py", line 290, in device_test_func return test_func(*args, device=device, **kwargs) File "/usr/local/lib/python3.8/dist-packages/onnx/backend/test/runner/__init__.py", line 467, in run self.assert_similar_outputs( File "../test/py/onnx_backend_test.py", line 59, in assert_similar_outputs np.testing.assert_allclose(ref_outputs[i], File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 1530, in assert_allclose assert_array_compare(compare, actual, desired, err_msg=str(err_msg), File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 763, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=0.001, atol=1e-05 Program = module: "main" score_threshold = @param:score_threshold -> float_type, {1}, {1}, target_id=0 iou_threshold = @param:iou_threshold -> float_type, {1}, {1}, target_id=0 max_output_boxes_per_class = @param:max_output_boxes_per_class -> int64_type, {1}, {1}, target_id=0 scores = @param:scores -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 boxes = @param:boxes -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @5 = nonmaxsuppression[center_point_box=0,use_dyn_output=0](boxes,scores,max_output_boxes_per_class,iou_threshold,score_threshold) -> int64_type, {6, 3}, {3, 1}, target_id=0 @6 = @return(@5), target_id=0 Compiled program = module: "main" @0 = check_context::migraphx::gpu::context -> float_type, {}, {}, target_id=0 @1 = hip::hip_allocate_memory[shape=int8_type, {176}, {1},id=main:scratch] -> int8_type, {176}, {1}, target_id=0 @2 = load[offset=0,end=96](@1) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 boxes = @param:boxes -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @4 = hip::copy_to_gpu(boxes,@2) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @5 = load[offset=96,end=120](@1) -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 scores = @param:scores -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 @7 = hip::copy_to_gpu(scores,@5) -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 max_output_boxes_per_class = @param:max_output_boxes_per_class -> int64_type, {1}, {1}, target_id=0 @9 = load[offset=128,end=136](@1) -> int64_type, {1}, {1}, target_id=0 @10 = hip::copy_to_gpu(max_output_boxes_per_class,@9) -> int64_type, {1}, {1}, target_id=0 @11 = load[offset=160,end=164](@1) -> float_type, {1}, {1}, target_id=0 iou_threshold = @param:iou_threshold -> float_type, {1}, {1}, target_id=0 @13 = hip::copy_to_gpu(iou_threshold,@11) -> float_type, {1}, {1}, target_id=0 score_threshold = @param:score_threshold -> float_type, {1}, {1}, target_id=0 @15 = load[offset=144,end=148](@1) -> float_type, {1}, {1}, target_id=0 @16 = hip::copy_to_gpu(score_threshold,@15) -> float_type, {1}, {1}, target_id=0 @17 = hip::copy_from_gpu(@13) -> float_type, {1}, {1}, target_id=0 @18 = hip::copy_from_gpu(@10) -> int64_type, {1}, {1}, target_id=0 @19 = hip::copy_from_gpu(@4) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @20 = hip::copy_from_gpu(@7) -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 @21 = hip::copy_from_gpu(@16) -> float_type, {1}, {1}, target_id=0 @22 = hip::sync_stream(@19,@20,@18,@17,@21) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @23 = load[offset=0,end=144](@1) -> int64_type, {6, 3}, {3, 1}, target_id=0 @24 = nonmaxsuppression[center_point_box=0,use_dyn_output=0](@22,@20,@18,@17,@21) -> int64_type, {6, 3}, {3, 1}, target_id=0 @25 = hip::copy_to_gpu(@24,@23) -> int64_type, {6, 3}, {3, 1}, target_id=0 @26 = hip::copy_from_gpu(@25) -> int64_type, {6, 3}, {3, 1}, target_id=0 @27 = hip::sync_stream(@26) -> int64_type, {6, 3}, {3, 1}, target_id=0 @28 = @return(@27), target_id=0 (shapes (3, 3), (6, 3) mismatch) x: array([[0, 0, 3], [0, 0, 0], [0, 0, 5]]) y: array([[0, 0, 3], [0, 0, 0], [0, 0, 5],... ```
FAIL: test_nonmaxsuppression_identical_boxes_cpu (__main__.OnnxBackendNodeModelTest) ``` ====================================================================== FAIL: test_nonmaxsuppression_identical_boxes_cpu (__main__.OnnxBackendNodeModelTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/onnx/backend/test/runner/__init__.py", line 290, in device_test_func return test_func(*args, device=device, **kwargs) File "/usr/local/lib/python3.8/dist-packages/onnx/backend/test/runner/__init__.py", line 467, in run self.assert_similar_outputs( File "../test/py/onnx_backend_test.py", line 59, in assert_similar_outputs np.testing.assert_allclose(ref_outputs[i], File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 1530, in assert_allclose assert_array_compare(compare, actual, desired, err_msg=str(err_msg), File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 763, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=0.001, atol=1e-05 Program = module: "main" score_threshold = @param:score_threshold -> float_type, {1}, {1}, target_id=0 iou_threshold = @param:iou_threshold -> float_type, {1}, {1}, target_id=0 max_output_boxes_per_class = @param:max_output_boxes_per_class -> int64_type, {1}, {1}, target_id=0 scores = @param:scores -> float_type, {1, 1, 10}, {10, 10, 1}, target_id=0 boxes = @param:boxes -> float_type, {1, 10, 4}, {40, 4, 1}, target_id=0 @5 = nonmaxsuppression[center_point_box=0,use_dyn_output=0](boxes,scores,max_output_boxes_per_class,iou_threshold,score_threshold) -> int64_type, {10, 3}, {3, 1}, target_id=0 @6 = @return(@5), target_id=0 Compiled program = module: "main" @0 = check_context::migraphx::gpu::context -> float_type, {}, {}, target_id=0 @1 = hip::hip_allocate_memory[shape=int8_type, {256}, {1},id=main:scratch] -> int8_type, {256}, {1}, target_id=0 @2 = load[offset=240,end=244](@1) -> float_type, {1}, {1}, target_id=0 iou_threshold = @param:iou_threshold -> float_type, {1}, {1}, target_id=0 @4 = hip::copy_to_gpu(iou_threshold,@2) -> float_type, {1}, {1}, target_id=0 @5 = load[offset=224,end=228](@1) -> float_type, {1}, {1}, target_id=0 score_threshold = @param:score_threshold -> float_type, {1}, {1}, target_id=0 @7 = hip::copy_to_gpu(score_threshold,@5) -> float_type, {1}, {1}, target_id=0 @8 = load[offset=0,end=160](@1) -> float_type, {1, 10, 4}, {40, 4, 1}, target_id=0 boxes = @param:boxes -> float_type, {1, 10, 4}, {40, 4, 1}, target_id=0 @10 = hip::copy_to_gpu(boxes,@8) -> float_type, {1, 10, 4}, {40, 4, 1}, target_id=0 scores = @param:scores -> float_type, {1, 1, 10}, {10, 10, 1}, target_id=0 @12 = load[offset=160,end=200](@1) -> float_type, {1, 1, 10}, {10, 10, 1}, target_id=0 @13 = hip::copy_to_gpu(scores,@12) -> float_type, {1, 1, 10}, {10, 10, 1}, target_id=0 max_output_boxes_per_class = @param:max_output_boxes_per_class -> int64_type, {1}, {1}, target_id=0 @15 = load[offset=208,end=216](@1) -> int64_type, {1}, {1}, target_id=0 @16 = hip::copy_to_gpu(max_output_boxes_per_class,@15) -> int64_type, {1}, {1}, target_id=0 @17 = hip::copy_from_gpu(@7) -> float_type, {1}, {1}, target_id=0 @18 = hip::copy_from_gpu(@10) -> float_type, {1, 10, 4}, {40, 4, 1}, target_id=0 @19 = hip::copy_from_gpu(@16) -> int64_type, {1}, {1}, target_id=0 @20 = hip::copy_from_gpu(@13) -> float_type, {1, 1, 10}, {10, 10, 1}, target_id=0 @21 = hip::copy_from_gpu(@4) -> float_type, {1}, {1}, target_id=0 @22 = hip::sync_stream(@18,@20,@19,@21,@17) -> float_type, {1, 10, 4}, {40, 4, 1}, target_id=0 @23 = load[offset=0,end=240](@1) -> int64_type, {10, 3}, {3, 1}, target_id=0 @24 = nonmaxsuppression[center_point_box=0,use_dyn_output=0](@22,@20,@19,@21,@17) -> int64_type, {10, 3}, {3, 1}, target_id=0 @25 = hip::copy_to_gpu(@24,@23) -> int64_type, {10, 3}, {3, 1}, target_id=0 @26 = hip::copy_from_gpu(@25) -> int64_type, {10, 3}, {3, 1}, target_id=0 @27 = hip::sync_stream(@26) -> int64_type, {10, 3}, {3, 1}, target_id=0 @28 = @return(@27), target_id=0 (shapes (1, 3), (10, 3) mismatch) x: array([[0, 0, 0]]) y: array([[0, 0, 9], [0, 0, 0], [0, 0, 0],... ```
FAIL: test_nonmaxsuppression_limit_output_size_cpu (__main__.OnnxBackendNodeModelTest) ``` ====================================================================== FAIL: test_nonmaxsuppression_limit_output_size_cpu (__main__.OnnxBackendNodeModelTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/onnx/backend/test/runner/__init__.py", line 290, in device_test_func return test_func(*args, device=device, **kwargs) File "/usr/local/lib/python3.8/dist-packages/onnx/backend/test/runner/__init__.py", line 467, in run self.assert_similar_outputs( File "../test/py/onnx_backend_test.py", line 59, in assert_similar_outputs np.testing.assert_allclose(ref_outputs[i], File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 1530, in assert_allclose assert_array_compare(compare, actual, desired, err_msg=str(err_msg), File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 763, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=0.001, atol=1e-05 Program = module: "main" score_threshold = @param:score_threshold -> float_type, {1}, {1}, target_id=0 iou_threshold = @param:iou_threshold -> float_type, {1}, {1}, target_id=0 max_output_boxes_per_class = @param:max_output_boxes_per_class -> int64_type, {1}, {1}, target_id=0 scores = @param:scores -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 boxes = @param:boxes -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @5 = nonmaxsuppression[center_point_box=0,use_dyn_output=0](boxes,scores,max_output_boxes_per_class,iou_threshold,score_threshold) -> int64_type, {6, 3}, {3, 1}, target_id=0 @6 = @return(@5), target_id=0 Compiled program = module: "main" @0 = check_context::migraphx::gpu::context -> float_type, {}, {}, target_id=0 @1 = hip::hip_allocate_memory[shape=int8_type, {176}, {1},id=main:scratch] -> int8_type, {176}, {1}, target_id=0 @2 = load[offset=160,end=164](@1) -> float_type, {1}, {1}, target_id=0 score_threshold = @param:score_threshold -> float_type, {1}, {1}, target_id=0 @4 = hip::copy_to_gpu(score_threshold,@2) -> float_type, {1}, {1}, target_id=0 boxes = @param:boxes -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @6 = load[offset=0,end=96](@1) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @7 = hip::copy_to_gpu(boxes,@6) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @8 = load[offset=96,end=120](@1) -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 scores = @param:scores -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 @10 = hip::copy_to_gpu(scores,@8) -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 max_output_boxes_per_class = @param:max_output_boxes_per_class -> int64_type, {1}, {1}, target_id=0 @12 = load[offset=128,end=136](@1) -> int64_type, {1}, {1}, target_id=0 @13 = hip::copy_to_gpu(max_output_boxes_per_class,@12) -> int64_type, {1}, {1}, target_id=0 @14 = load[offset=144,end=148](@1) -> float_type, {1}, {1}, target_id=0 iou_threshold = @param:iou_threshold -> float_type, {1}, {1}, target_id=0 @16 = hip::copy_to_gpu(iou_threshold,@14) -> float_type, {1}, {1}, target_id=0 @17 = hip::copy_from_gpu(@4) -> float_type, {1}, {1}, target_id=0 @18 = hip::copy_from_gpu(@10) -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 @19 = hip::copy_from_gpu(@7) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @20 = hip::copy_from_gpu(@13) -> int64_type, {1}, {1}, target_id=0 @21 = hip::copy_from_gpu(@16) -> float_type, {1}, {1}, target_id=0 @22 = hip::sync_stream(@19,@18,@20,@21,@17) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @23 = nonmaxsuppression[center_point_box=0,use_dyn_output=0](@22,@18,@20,@21,@17) -> int64_type, {6, 3}, {3, 1}, target_id=0 @24 = load[offset=0,end=144](@1) -> int64_type, {6, 3}, {3, 1}, target_id=0 @25 = hip::copy_to_gpu(@23,@24) -> int64_type, {6, 3}, {3, 1}, target_id=0 @26 = hip::copy_from_gpu(@25) -> int64_type, {6, 3}, {3, 1}, target_id=0 @27 = hip::sync_stream(@26) -> int64_type, {6, 3}, {3, 1}, target_id=0 @28 = @return(@27), target_id=0 (shapes (2, 3), (6, 3) mismatch) x: array([[0, 0, 3], [0, 0, 0]]) y: array([[0, 0, 3], [0, 0, 0], [0, 0, 0],... ```
FAIL: test_nonmaxsuppression_suppress_by_IOU_and_scores_cpu (__main__.OnnxBackendNodeModelTest) ``` ====================================================================== FAIL: test_nonmaxsuppression_suppress_by_IOU_and_scores_cpu (__main__.OnnxBackendNodeModelTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/onnx/backend/test/runner/__init__.py", line 290, in device_test_func return test_func(*args, device=device, **kwargs) File "/usr/local/lib/python3.8/dist-packages/onnx/backend/test/runner/__init__.py", line 467, in run self.assert_similar_outputs( File "../test/py/onnx_backend_test.py", line 59, in assert_similar_outputs np.testing.assert_allclose(ref_outputs[i], File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 1530, in assert_allclose assert_array_compare(compare, actual, desired, err_msg=str(err_msg), File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 763, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=0.001, atol=1e-05 Program = module: "main" score_threshold = @param:score_threshold -> float_type, {1}, {1}, target_id=0 iou_threshold = @param:iou_threshold -> float_type, {1}, {1}, target_id=0 max_output_boxes_per_class = @param:max_output_boxes_per_class -> int64_type, {1}, {1}, target_id=0 scores = @param:scores -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 boxes = @param:boxes -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @5 = nonmaxsuppression[center_point_box=0,use_dyn_output=0](boxes,scores,max_output_boxes_per_class,iou_threshold,score_threshold) -> int64_type, {6, 3}, {3, 1}, target_id=0 @6 = @return(@5), target_id=0 Compiled program = module: "main" @0 = check_context::migraphx::gpu::context -> float_type, {}, {}, target_id=0 @1 = hip::hip_allocate_memory[shape=int8_type, {176}, {1},id=main:scratch] -> int8_type, {176}, {1}, target_id=0 boxes = @param:boxes -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @3 = load[offset=0,end=96](@1) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @4 = hip::copy_to_gpu(boxes,@3) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 scores = @param:scores -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 @6 = load[offset=96,end=120](@1) -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 @7 = hip::copy_to_gpu(scores,@6) -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 max_output_boxes_per_class = @param:max_output_boxes_per_class -> int64_type, {1}, {1}, target_id=0 @9 = load[offset=128,end=136](@1) -> int64_type, {1}, {1}, target_id=0 @10 = hip::copy_to_gpu(max_output_boxes_per_class,@9) -> int64_type, {1}, {1}, target_id=0 iou_threshold = @param:iou_threshold -> float_type, {1}, {1}, target_id=0 @12 = load[offset=160,end=164](@1) -> float_type, {1}, {1}, target_id=0 @13 = hip::copy_to_gpu(iou_threshold,@12) -> float_type, {1}, {1}, target_id=0 @14 = load[offset=144,end=148](@1) -> float_type, {1}, {1}, target_id=0 score_threshold = @param:score_threshold -> float_type, {1}, {1}, target_id=0 @16 = hip::copy_to_gpu(score_threshold,@14) -> float_type, {1}, {1}, target_id=0 @17 = hip::copy_from_gpu(@4) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @18 = hip::copy_from_gpu(@13) -> float_type, {1}, {1}, target_id=0 @19 = hip::copy_from_gpu(@16) -> float_type, {1}, {1}, target_id=0 @20 = hip::copy_from_gpu(@7) -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 @21 = hip::copy_from_gpu(@10) -> int64_type, {1}, {1}, target_id=0 @22 = hip::sync_stream(@17,@20,@21,@18,@19) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @23 = nonmaxsuppression[center_point_box=0,use_dyn_output=0](@22,@20,@21,@18,@19) -> int64_type, {6, 3}, {3, 1}, target_id=0 @24 = load[offset=0,end=144](@1) -> int64_type, {6, 3}, {3, 1}, target_id=0 @25 = hip::copy_to_gpu(@23,@24) -> int64_type, {6, 3}, {3, 1}, target_id=0 @26 = hip::copy_from_gpu(@25) -> int64_type, {6, 3}, {3, 1}, target_id=0 @27 = hip::sync_stream(@26) -> int64_type, {6, 3}, {3, 1}, target_id=0 @28 = @return(@27), target_id=0 (shapes (2, 3), (6, 3) mismatch) x: array([[0, 0, 3], [0, 0, 0]]) y: array([[0, 0, 3], [0, 0, 0], [0, 0, 0],... ```
FAIL: test_nonmaxsuppression_suppress_by_IOU_cpu (__main__.OnnxBackendNodeModelTest) ``` ====================================================================== FAIL: test_nonmaxsuppression_suppress_by_IOU_cpu (__main__.OnnxBackendNodeModelTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/onnx/backend/test/runner/__init__.py", line 290, in device_test_func return test_func(*args, device=device, **kwargs) File "/usr/local/lib/python3.8/dist-packages/onnx/backend/test/runner/__init__.py", line 467, in run self.assert_similar_outputs( File "../test/py/onnx_backend_test.py", line 59, in assert_similar_outputs np.testing.assert_allclose(ref_outputs[i], File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 1530, in assert_allclose assert_array_compare(compare, actual, desired, err_msg=str(err_msg), File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 763, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=0.001, atol=1e-05 Program = module: "main" score_threshold = @param:score_threshold -> float_type, {1}, {1}, target_id=0 iou_threshold = @param:iou_threshold -> float_type, {1}, {1}, target_id=0 max_output_boxes_per_class = @param:max_output_boxes_per_class -> int64_type, {1}, {1}, target_id=0 scores = @param:scores -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 boxes = @param:boxes -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @5 = nonmaxsuppression[center_point_box=0,use_dyn_output=0](boxes,scores,max_output_boxes_per_class,iou_threshold,score_threshold) -> int64_type, {6, 3}, {3, 1}, target_id=0 @6 = @return(@5), target_id=0 Compiled program = module: "main" @0 = check_context::migraphx::gpu::context -> float_type, {}, {}, target_id=0 @1 = hip::hip_allocate_memory[shape=int8_type, {176}, {1},id=main:scratch] -> int8_type, {176}, {1}, target_id=0 @2 = load[offset=0,end=96](@1) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 boxes = @param:boxes -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @4 = hip::copy_to_gpu(boxes,@2) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @5 = load[offset=96,end=120](@1) -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 scores = @param:scores -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 @7 = hip::copy_to_gpu(scores,@5) -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 @8 = load[offset=128,end=136](@1) -> int64_type, {1}, {1}, target_id=0 max_output_boxes_per_class = @param:max_output_boxes_per_class -> int64_type, {1}, {1}, target_id=0 @10 = hip::copy_to_gpu(max_output_boxes_per_class,@8) -> int64_type, {1}, {1}, target_id=0 @11 = load[offset=160,end=164](@1) -> float_type, {1}, {1}, target_id=0 iou_threshold = @param:iou_threshold -> float_type, {1}, {1}, target_id=0 @13 = hip::copy_to_gpu(iou_threshold,@11) -> float_type, {1}, {1}, target_id=0 @14 = load[offset=144,end=148](@1) -> float_type, {1}, {1}, target_id=0 score_threshold = @param:score_threshold -> float_type, {1}, {1}, target_id=0 @16 = hip::copy_to_gpu(score_threshold,@14) -> float_type, {1}, {1}, target_id=0 @17 = hip::copy_from_gpu(@13) -> float_type, {1}, {1}, target_id=0 @18 = hip::copy_from_gpu(@4) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @19 = hip::copy_from_gpu(@10) -> int64_type, {1}, {1}, target_id=0 @20 = hip::copy_from_gpu(@16) -> float_type, {1}, {1}, target_id=0 @21 = hip::copy_from_gpu(@7) -> float_type, {1, 1, 6}, {6, 6, 1}, target_id=0 @22 = hip::sync_stream(@18,@21,@19,@17,@20) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @23 = load[offset=0,end=144](@1) -> int64_type, {6, 3}, {3, 1}, target_id=0 @24 = nonmaxsuppression[center_point_box=0,use_dyn_output=0](@22,@21,@19,@17,@20) -> int64_type, {6, 3}, {3, 1}, target_id=0 @25 = hip::copy_to_gpu(@24,@23) -> int64_type, {6, 3}, {3, 1}, target_id=0 @26 = hip::copy_from_gpu(@25) -> int64_type, {6, 3}, {3, 1}, target_id=0 @27 = hip::sync_stream(@26) -> int64_type, {6, 3}, {3, 1}, target_id=0 @28 = @return(@27), target_id=0 (shapes (3, 3), (6, 3) mismatch) x: array([[0, 0, 3], [0, 0, 0], [0, 0, 5]]) y: array([[0, 0, 3], [0, 0, 0], [0, 0, 5],... ```
FAIL: test_nonmaxsuppression_two_batches_cpu (__main__.OnnxBackendNodeModelTest) ``` ====================================================================== FAIL: test_nonmaxsuppression_two_batches_cpu (__main__.OnnxBackendNodeModelTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/onnx/backend/test/runner/__init__.py", line 290, in device_test_func return test_func(*args, device=device, **kwargs) File "/usr/local/lib/python3.8/dist-packages/onnx/backend/test/runner/__init__.py", line 467, in run self.assert_similar_outputs( File "../test/py/onnx_backend_test.py", line 59, in assert_similar_outputs np.testing.assert_allclose(ref_outputs[i], File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 1530, in assert_allclose assert_array_compare(compare, actual, desired, err_msg=str(err_msg), File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 763, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=0.001, atol=1e-05 Program = module: "main" score_threshold = @param:score_threshold -> float_type, {1}, {1}, target_id=0 iou_threshold = @param:iou_threshold -> float_type, {1}, {1}, target_id=0 max_output_boxes_per_class = @param:max_output_boxes_per_class -> int64_type, {1}, {1}, target_id=0 scores = @param:scores -> float_type, {2, 1, 6}, {6, 6, 1}, target_id=0 boxes = @param:boxes -> float_type, {2, 6, 4}, {24, 4, 1}, target_id=0 @5 = nonmaxsuppression[center_point_box=0,use_dyn_output=0](boxes,scores,max_output_boxes_per_class,iou_threshold,score_threshold) -> int64_type, {12, 3}, {3, 1}, target_id=0 @6 = @return(@5), target_id=0 Compiled program = module: "main" @0 = check_context::migraphx::gpu::context -> float_type, {}, {}, target_id=0 @1 = hip::hip_allocate_memory[shape=int8_type, {352}, {1},id=main:scratch] -> int8_type, {352}, {1}, target_id=0 boxes = @param:boxes -> float_type, {2, 6, 4}, {24, 4, 1}, target_id=0 @3 = load[offset=0,end=192](@1) -> float_type, {2, 6, 4}, {24, 4, 1}, target_id=0 @4 = hip::copy_to_gpu(boxes,@3) -> float_type, {2, 6, 4}, {24, 4, 1}, target_id=0 scores = @param:scores -> float_type, {2, 1, 6}, {6, 6, 1}, target_id=0 @6 = load[offset=192,end=240](@1) -> float_type, {2, 1, 6}, {6, 6, 1}, target_id=0 @7 = hip::copy_to_gpu(scores,@6) -> float_type, {2, 1, 6}, {6, 6, 1}, target_id=0 max_output_boxes_per_class = @param:max_output_boxes_per_class -> int64_type, {1}, {1}, target_id=0 @9 = load[offset=256,end=264](@1) -> int64_type, {1}, {1}, target_id=0 @10 = hip::copy_to_gpu(max_output_boxes_per_class,@9) -> int64_type, {1}, {1}, target_id=0 iou_threshold = @param:iou_threshold -> float_type, {1}, {1}, target_id=0 @12 = load[offset=320,end=324](@1) -> float_type, {1}, {1}, target_id=0 @13 = hip::copy_to_gpu(iou_threshold,@12) -> float_type, {1}, {1}, target_id=0 @14 = load[offset=288,end=292](@1) -> float_type, {1}, {1}, target_id=0 score_threshold = @param:score_threshold -> float_type, {1}, {1}, target_id=0 @16 = hip::copy_to_gpu(score_threshold,@14) -> float_type, {1}, {1}, target_id=0 @17 = hip::copy_from_gpu(@7) -> float_type, {2, 1, 6}, {6, 6, 1}, target_id=0 @18 = hip::copy_from_gpu(@13) -> float_type, {1}, {1}, target_id=0 @19 = hip::copy_from_gpu(@16) -> float_type, {1}, {1}, target_id=0 @20 = hip::copy_from_gpu(@4) -> float_type, {2, 6, 4}, {24, 4, 1}, target_id=0 @21 = hip::copy_from_gpu(@10) -> int64_type, {1}, {1}, target_id=0 @22 = hip::sync_stream(@20,@17,@21,@18,@19) -> float_type, {2, 6, 4}, {24, 4, 1}, target_id=0 @23 = nonmaxsuppression[center_point_box=0,use_dyn_output=0](@22,@17,@21,@18,@19) -> int64_type, {12, 3}, {3, 1}, target_id=0 @24 = load[offset=0,end=288](@1) -> int64_type, {12, 3}, {3, 1}, target_id=0 @25 = hip::copy_to_gpu(@23,@24) -> int64_type, {12, 3}, {3, 1}, target_id=0 @26 = hip::copy_from_gpu(@25) -> int64_type, {12, 3}, {3, 1}, target_id=0 @27 = hip::sync_stream(@26) -> int64_type, {12, 3}, {3, 1}, target_id=0 @28 = @return(@27), target_id=0 (shapes (4, 3), (12, 3) mismatch) x: array([[0, 0, 3], [0, 0, 0], [1, 0, 3], [1, 0, 0]]) y: array([[0, 0, 3], [0, 0, 0], [1, 0, 3],... ```
FAIL: test_nonmaxsuppression_two_classes_cpu (__main__.OnnxBackendNodeModelTest) ``` ====================================================================== FAIL: test_nonmaxsuppression_two_classes_cpu (__main__.OnnxBackendNodeModelTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/onnx/backend/test/runner/__init__.py", line 290, in device_test_func return test_func(*args, device=device, **kwargs) File "/usr/local/lib/python3.8/dist-packages/onnx/backend/test/runner/__init__.py", line 467, in run self.assert_similar_outputs( File "../test/py/onnx_backend_test.py", line 59, in assert_similar_outputs np.testing.assert_allclose(ref_outputs[i], File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 1530, in assert_allclose assert_array_compare(compare, actual, desired, err_msg=str(err_msg), File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 763, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=0.001, atol=1e-05 Program = module: "main" score_threshold = @param:score_threshold -> float_type, {1}, {1}, target_id=0 iou_threshold = @param:iou_threshold -> float_type, {1}, {1}, target_id=0 max_output_boxes_per_class = @param:max_output_boxes_per_class -> int64_type, {1}, {1}, target_id=0 scores = @param:scores -> float_type, {1, 2, 6}, {12, 6, 1}, target_id=0 boxes = @param:boxes -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @5 = nonmaxsuppression[center_point_box=0,use_dyn_output=0](boxes,scores,max_output_boxes_per_class,iou_threshold,score_threshold) -> int64_type, {6, 3}, {3, 1}, target_id=0 @6 = @return(@5), target_id=0 Compiled program = module: "main" @0 = check_context::migraphx::gpu::context -> float_type, {}, {}, target_id=0 @1 = hip::hip_allocate_memory[shape=int8_type, {192}, {1},id=main:scratch] -> int8_type, {192}, {1}, target_id=0 @2 = load[offset=144,end=152](@1) -> int64_type, {1}, {1}, target_id=0 max_output_boxes_per_class = @param:max_output_boxes_per_class -> int64_type, {1}, {1}, target_id=0 @4 = hip::copy_to_gpu(max_output_boxes_per_class,@2) -> int64_type, {1}, {1}, target_id=0 iou_threshold = @param:iou_threshold -> float_type, {1}, {1}, target_id=0 @6 = load[offset=176,end=180](@1) -> float_type, {1}, {1}, target_id=0 @7 = hip::copy_to_gpu(iou_threshold,@6) -> float_type, {1}, {1}, target_id=0 score_threshold = @param:score_threshold -> float_type, {1}, {1}, target_id=0 @9 = load[offset=160,end=164](@1) -> float_type, {1}, {1}, target_id=0 @10 = hip::copy_to_gpu(score_threshold,@9) -> float_type, {1}, {1}, target_id=0 @11 = load[offset=0,end=96](@1) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 boxes = @param:boxes -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @13 = hip::copy_to_gpu(boxes,@11) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 scores = @param:scores -> float_type, {1, 2, 6}, {12, 6, 1}, target_id=0 @15 = load[offset=96,end=144](@1) -> float_type, {1, 2, 6}, {12, 6, 1}, target_id=0 @16 = hip::copy_to_gpu(scores,@15) -> float_type, {1, 2, 6}, {12, 6, 1}, target_id=0 @17 = hip::copy_from_gpu(@16) -> float_type, {1, 2, 6}, {12, 6, 1}, target_id=0 @18 = hip::copy_from_gpu(@10) -> float_type, {1}, {1}, target_id=0 @19 = hip::copy_from_gpu(@7) -> float_type, {1}, {1}, target_id=0 @20 = hip::copy_from_gpu(@4) -> int64_type, {1}, {1}, target_id=0 @21 = hip::copy_from_gpu(@13) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @22 = hip::sync_stream(@21,@17,@20,@19,@18) -> float_type, {1, 6, 4}, {24, 4, 1}, target_id=0 @23 = load[offset=0,end=144](@1) -> int64_type, {6, 3}, {3, 1}, target_id=0 @24 = nonmaxsuppression[center_point_box=0,use_dyn_output=0](@22,@17,@20,@19,@18) -> int64_type, {6, 3}, {3, 1}, target_id=0 @25 = hip::copy_to_gpu(@24,@23) -> int64_type, {6, 3}, {3, 1}, target_id=0 @26 = hip::copy_from_gpu(@25) -> int64_type, {6, 3}, {3, 1}, target_id=0 @27 = hip::sync_stream(@26) -> int64_type, {6, 3}, {3, 1}, target_id=0 @28 = @return(@27), target_id=0 (shapes (4, 3), (6, 3) mismatch) x: array([[0, 0, 3], [0, 0, 0], [0, 1, 3], [0, 1, 0]]) y: array([[0, 0, 3], [0, 0, 0], [0, 1, 3],... ```
attila-dusnoki-htec commented 11 months ago

The tests are failing due to the output shape. MIGraphX allocates it for all possible boxes. There is an option to be dynamic with use_dyn_output = true, wich will make the tests passes.

The only fail will be test_nonmaxsuppression_identical_boxes_cpu, because the first box is selected, and the test expects the last. Note: the spec does not specify this.