LayerNormalization operator is unsupported - Githubissues

migraphx-benchmark / AMDMIGraphX

AMD's graph optimization engine.

https://rocmsoftwareplatform.github.io/AMDMIGraphX/doc/html/

MIT License

0 stars 1 forks source link

LayerNormalization operator is unsupported #100

Closed attila-dusnoki-htec closed 11 months ago

attila-dusnoki-htec commented 1 year ago

Failing tests:

test_layer_normalization_2d_axis0_cpu
test_layer_normalization_2d_axis1_cpu
test_layer_normalization_2d_axis_negative_1_cpu
test_layer_normalization_2d_axis_negative_2_cpu
test_layer_normalization_3d_axis0_epsilon_cpu
test_layer_normalization_3d_axis1_epsilon_cpu
test_layer_normalization_3d_axis2_epsilon_cpu
test_layer_normalization_3d_axis_negative_1_epsilon_cpu
test_layer_normalization_3d_axis_negative_2_epsilon_cpu
test_layer_normalization_3d_axis_negative_3_epsilon_cpu
test_layer_normalization_4d_axis0_cpu
test_layer_normalization_4d_axis1_cpu
test_layer_normalization_4d_axis2_cpu
test_layer_normalization_4d_axis3_cpu
test_layer_normalization_4d_axis_negative_1_cpu
test_layer_normalization_4d_axis_negative_2_cpu
test_layer_normalization_4d_axis_negative_3_cpu
test_layer_normalization_4d_axis_negative_4_cpu
test_layer_normalization_default_axis_cpu

attila-dusnoki-htec commented 1 year ago

Operator description: https://github.com/onnx/onnx/blob/main/docs/Operators.md#LayerNormalization

Tests: https://github.com/onnx/onnx/blob/main/docs/TestCoverage.md#layernormalization

oneDNN support: https://oneapi-src.github.io/oneDNN/group_dnnl_api_layer_normalization.html

attila-dusnoki-htec commented 1 year ago

Reference implementation: https://github.com/onnx/onnx/blob/main/onnx/backend/test/case/node/layernormalization.py#L13

MIGraphX implementation of BN and IN

attila-dusnoki-htec commented 11 months ago

Resolved in https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/pull/2242