[Good First Issue] [ARM]: Implement CPU plugin just-in-time emitter for Mod operation

Context

JIT Emitters are part of code generation feature (a.k.a. tensor compiler) that automatically produces highly-efficient optimized fused subgraph binary code. Each emitter implements specific operation from low level OpenVINO dialect.

Prerequisites

Recommended to use ARM CPU based platform for development (e.g. Mac, Raspberry Pi etc). Emulators (e.g. QEMU) is still an option, but not that convenient, especially for final performance evaluation.

What needs to be done?

Create fp32 Mod operation JIT emitter for ARM64 SIMD platform. Use OpenVINO CPU plugin JIT emitters documentation and Arm A-profile A64 Instruction Set Architecture documentation for details.
Modify ARM64 executor and ARM64 kernel (in both places: create_eltwise_emitter and get_supported_precisions) to apply developed JIT emitter.

Before emitter implementation, please, modify tests to be sure that deveoped functionality is covered by test:

Add jit kernel check in element-wise or activation test, the test must fail.
If test doesn't fail then add supported operation in element-wise test instantiation or activation test instantiation if it's absent.

Tests

Tests are disabled in default build, so ensure to add -DENABLE_TESTS=ON into cmake command.

GoogleTest is used for testing. CPU functional test target is ov_cpu_func_tests. You can use two GoogleTest filters for element-wise and activation operations:

for element-wise operations:

./bin/[platform]/[build_type]/ov_cpu_func_tests --gtest_filter="*smoke*Eltwise*"

for activations:

./bin/[platform]/[build_type]/ov_cpu_func_tests --gtest_filter="*smoke*Activation*"

Example Pull Requests

Abs operation: https://github.com/openvinotoolkit/openvino/pull/23692
Sigmoid operation, which reuses Exp emitter: https://github.com/openvinotoolkit/openvino/pull/23065

Resources

Contact points

@eshoguli, @dmitry-gorokhov

Ticket

CVS-137689

Thank you for looking into this issue! Please let us know if you have any questions or require any help.

@eshoguli Hi,I rent a arm cpu server, but could not build openvino successful, is my environment suitable for development?

ubuntu@ecs-897f:~/openvino/build$ cat /proc/version
Linux version 4.15.0-70-generic (buildd@bos02-arm64-021) (gcc version 7.4.0 (Ubuntu/Linaro 7.4.0-1ubuntu1~18.04.1)) #79-Ubuntu SMP Tue Nov 12 10:36:10 UTC 2019
ubuntu@ecs-897f:~/openvino/build$ cat /proc/cpuinfo
processor   : 0
BogoMIPS    : 200.00
Features    : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma dcpop asimddp
CPU implementer : 0x48
CPU architecture: 8
CPU variant : 0x1
CPU part    : 0xd01
CPU revision    : 0

processor   : 1
BogoMIPS    : 200.00
Features    : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma dcpop asimddp
CPU implementer : 0x48
CPU architecture: 8
CPU variant : 0x1
CPU part    : 0xd01
CPU revision    : 0

ubuntu@ecs-897f:~/openvino/build$ cmake -DCMAKE_BUILD_TYPE=Release -DENABLE_TESTS=OFF -DENABLE_CLANG_FORMAT=ON -DENABLE_NCC_STYLE=OFF -DENABLE_INTEL_CPU=ON -DENABLE_INTEL_NPU=OFF -DENABLE_OV_PADDLE_FRONTEND=OFF cmake -DENABLE_OV_ONNX_FRONTEND=OFF -DENABLE_OV_PYTORCH_FRONTEND=OFF -DENABLE_OV_TF_FRONTEND=OFF -DENABLE_OV_TF_LITE_FRONTEND=OFF -DENABLE_INTEL_GPU=OFF  -DENABLE_TESTS=ON -DENABLE_FUNCTIONAL_TESTS=ON ..
-- OpenVINO version is 2024.2.0 (Build 15123)
-- OpenVINO Runtime enabled features:
--
--     CI_BUILD_NUMBER: 2024.2.0-15123-007ed84886c
--     CPACK_GENERATOR = TGZ
--     ENABLE_LTO = OFF
--     OS_FOLDER = OFF
--     USE_BUILD_TYPE_SUBFOLDER = ON
--     CMAKE_COMPILE_WARNING_AS_ERROR = OFF
--     ENABLE_QSPECTRE = OFF
--     ENABLE_INTEGRITYCHECK = OFF
--     ENABLE_SANITIZER = OFF
--     ENABLE_UB_SANITIZER = OFF
--     ENABLE_THREAD_SANITIZER = OFF
--     ENABLE_COVERAGE = OFF
--     ENABLE_SSE42 = OFF
--     ENABLE_AVX2 = OFF
--     ENABLE_AVX512F = OFF
--     BUILD_SHARED_LIBS = ON
--     ENABLE_LIBRARY_VERSIONING = ON
--     ENABLE_FASTER_BUILD = OFF
--     ENABLE_CPPLINT = ON
--     ENABLE_CPPLINT_REPORT = OFF
--     ENABLE_CLANG_FORMAT = ON
--     ENABLE_NCC_STYLE = OFF
--     ENABLE_UNSAFE_LOCATIONS = OFF
--     ENABLE_FUZZING = OFF
--     ENABLE_PROXY = ON
--     ENABLE_INTEL_CPU = ON
--     ENABLE_ARM_COMPUTE_CMAKE = OFF
--     ENABLE_TESTS = ON
--     ENABLE_INTEL_GPU = OFF
--     ENABLE_ONEDNN_FOR_GPU = OFF
--     ENABLE_INTEL_NPU = OFF
--     ENABLE_DEBUG_CAPS = OFF
--     ENABLE_NPU_DEBUG_CAPS = OFF
--     ENABLE_GPU_DEBUG_CAPS = OFF
--     ENABLE_CPU_DEBUG_CAPS = OFF
--     ENABLE_SNIPPETS_DEBUG_CAPS = OFF
--     ENABLE_PROFILING_ITT = OFF
--     ENABLE_PROFILING_FILTER = ALL
--     ENABLE_PROFILING_FIRST_INFERENCE = ON
--     SELECTIVE_BUILD = OFF
--     ENABLE_DOCS = OFF
--     ENABLE_PKGCONFIG_GEN = ON
--     THREADING = TBB
--     ENABLE_TBBBIND_2_5 = ON
--     ENABLE_TBB_RELEASE_ONLY = ON
--     ENABLE_MULTI = ON
--     ENABLE_AUTO = ON
--     ENABLE_AUTO_BATCH = ON
--     ENABLE_HETERO = ON
--     ENABLE_TEMPLATE = ON
--     ENABLE_PLUGINS_XML = OFF
--     ENABLE_FUNCTIONAL_TESTS = ON
--     ENABLE_SAMPLES = ON
--     ENABLE_OV_ONNX_FRONTEND = OFF
--     ENABLE_OV_PADDLE_FRONTEND = OFF
--     ENABLE_OV_IR_FRONTEND = ON
--     ENABLE_OV_PYTORCH_FRONTEND = OFF
--     ENABLE_OV_IR_FRONTEND = ON
--     ENABLE_OV_TF_FRONTEND = OFF
--     ENABLE_OV_TF_LITE_FRONTEND = OFF
--     ENABLE_SNAPPY_COMPRESSION = OFF
--     ENABLE_STRICT_DEPENDENCIES = OFF
--     ENABLE_SYSTEM_TBB = OFF
--     ENABLE_SYSTEM_PUGIXML = OFF
--     ENABLE_SYSTEM_FLATBUFFERS = OFF
--     ENABLE_SYSTEM_OPENCL = OFF
--     ENABLE_SYSTEM_PROTOBUF = OFF
--     ENABLE_SYSTEM_SNAPPY = OFF
--     ENABLE_PYTHON_PACKAGING = OFF
--     ENABLE_JS = ON
--     ENABLE_OPENVINO_DEBUG = OFF
--
-- CMAKE_VERSION ......................... 3.26.0
-- CMAKE_CROSSCOMPILING .................. FALSE
-- OpenVINO_SOURCE_DIR ................... /home/ubuntu/openvino
-- OpenVINO_BINARY_DIR ................... /home/ubuntu/openvino
-- CMAKE_GENERATOR ....................... Unix Makefiles
-- CPACK_GENERATOR ....................... TGZ
-- CMAKE_C_COMPILER_ID ................... GNU
-- CMAKE_CXX_COMPILER_ID ................. GNU
-- CMAKE_CXX_STANDARD .................... 11
-- CMAKE_BUILD_TYPE ...................... Release
-- GLIBC_VERSION ......................... 2.27
-- Using the multi-header code from /home/ubuntu/openvino/thirdparty/json/nlohmann_json/include/
-- TBB (2021.2.1) is found at /home/ubuntu/openvino/temp/tbb/lib/cmake/TBB
-- Static tbbbind_2_5 package usage is disabled, since oneTBB (ver. 2021.2.1) provides dynamic TBBBind 2.5+
-- DNNL_TARGET_ARCH: AARCH64
-- DNNL_LIBRARY_NAME: openvino_onednn_cpu
-- Configure to build /home/ubuntu/openvino/src/plugins/intel_cpu/thirdparty/ComputeLibrary
CMake Warning at src/plugins/intel_cpu/thirdparty/onednn/cmake/ACL.cmake:39 (message):
  Build may fail: Could not determine ACL version (minimum required is
  23.02.1)
Call Stack (most recent call first):
  src/plugins/intel_cpu/thirdparty/onednn/CMakeLists.txt:129 (include)

-- Arm Compute Library: arm_compute::arm_compute;arm_compute::half
-- Arm Compute Library headers: $<TARGET_PROPERTY:arm_compute::arm_compute,INTERFACE_INCLUDE_DIRECTORIES>;$<TARGET_PROPERTY:arm_compute::half,INTERFACE_INCLUDE_DIRECTORIES>
-- Could NOT find Doxyrest (missing: DOXYREST_EXECUTABLE)
-- Could NOT find Sphinx (missing: SPHINX_EXECUTABLE)
-- Enabled workload: INFERENCE
-- Enabled primitives: CONVOLUTION;DECONVOLUTION;CONCAT;LRN;INNER_PRODUCT;MATMUL;POOLING;REDUCTION;REORDER;RNN;SOFTMAX
-- Enabled primitive CPU ISA: ALL
-- Enabled primitive GPU ISA: ALL
-- Enabled GeMM kernels ISA: ALL
-- Primitive cache is enabled
-- Python3 executable: /usr/bin/python3.6
-- Python3 version: 3.6.9
-- Python3 interpreter ID: Python
-- Python3 SOABI: cpython-36m-aarch64-linux-gnu
-- PYTHON_MODULE_EXTENSION: .cpython-36m-aarch64-linux-gnu.so
-- Python3 include dirs: /usr/include/python3.6m
-- Python3 libraries:
CMake Warning at cmake/developer_package/python_requirements.cmake:118 (message):
  Python requirement file
  /home/ubuntu/openvino/src/bindings/python/wheel/requirements-dev.txt is not
  installed,
Call Stack (most recent call first):
  src/bindings/python/CMakeLists.txt:155 (ov_check_pip_packages)

-- pybind11 v2.12.0
CMake Warning at samples/cpp/common/format_reader/CMakeLists.txt:22 (message):
  OpenCV ver.  3.0+ is not found, format_reader will be built without OpenCV
  support

CMake Warning at samples/cpp/benchmark_app/CMakeLists.txt:112 (message):
  OpenCV ver.  3.0+ is not found, benchmark_app will be built without OpenCV
  support.  Set OpenCV_DIR

CMake Warning at samples/c/common/opencv_c_wrapper/CMakeLists.txt:20 (message):
  OpenCV ver.  3.0+ is not found, opencv_c_wrapper is built without OPENCV
  support

-- Register template to be built in build-modules/template
CMake Warning at src/plugins/template/tests/functional/CMakeLists.txt:35 (message):
  Reference preprocessing: OpenCV tests are disabled, because OpenCV ver.
  3.4+ is not found

-- Register template_extension to be built in build-modules/template_extension
-- /usr/bin/pkg-config: libva (1.1.0) is found at /usr
-- pybind11 v2.12.0
-- Configuring done (3.0s)
-- Generating done (1.8s)
-- Build files have been written to: /home/ubuntu/openvino
ubuntu@ecs-897f:~/openvino/build$ cmake --build . --parallel 2
Error: could not load cache
ubuntu@ecs-897f:~/openvino/build$ ls
ubuntu@ecs-897f:~/openvino/build$

Build files have been written to: /home/ubuntu/openvino

So, you should run build command there

Hello @awayzjj, are you still working on that issue? Do you need any help?

@eshoguli Hi, I tried to implemented a JIT emitter for Mod operation as the linked PR shows , but the test failed. For example, one failed test is as follows:

# ./bin/aarch64/Debug/ov_cpu_func_tests --gtest_filter="smoke_CompareWithRefs_static/EltwiseLayerTest.Inference/IS=([]_)_TS=((1.1.1.3)_)_eltwise_op_type=Mod_secondary_input_type=CONSTANT_opType=VECTOR_model_type=i32_InType=undefined_OutType=undefined_trgDev=CPU"
Note: Google Test filter = smoke_CompareWithRefs_static/EltwiseLayerTest.Inference/IS=([]_)_TS=((1.1.1.3)_)_eltwise_op_type=Mod_secondary_input_type=CONSTANT_opType=VECTOR_model_type=i32_InType=undefined_OutType=undefined_trgDev=CPU
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from smoke_CompareWithRefs_static/EltwiseLayerTest
[ RUN      ] smoke_CompareWithRefs_static/EltwiseLayerTest.Inference/IS=([]_)_TS=((1.1.1.3)_)_eltwise_op_type=Mod_secondary_input_type=CONSTANT_opType=VECTOR_model_type=i32_InType=undefined_OutType=undefined_trgDev=CPU

MEM_USAGE=176676KB

Expected: 5 Actual: 0 Diff: 5 calculated_abs_threshold: 0.00573242 abs_threshold: 0.000732422 rel_threshold: 0.001

Expected: 2 Actual: 0 Diff: 2 calculated_abs_threshold: 0.00273242 abs_threshold: 0.000732422 rel_threshold: 0.001

Expected: 3 Actual: 0 Diff: 3 calculated_abs_threshold: 0.00373242 abs_threshold: 0.000732422 rel_threshold: 0.001
/mnt/data/openvino/src/tests/functional/shared_test_classes/src/base/ov_subgraph.cpp:95: Failure
[ COMPARATION ] COMPARATION IS FAILED! incorrect elem counter: 3 among 3 shapes.
[  FAILED  ] smoke_CompareWithRefs_static/EltwiseLayerTest.Inference/IS=([]_)_TS=((1.1.1.3)_)_eltwise_op_type=Mod_secondary_input_type=CONSTANT_opType=VECTOR_model_type=i32_InType=undefined_OutType=undefined_trgDev=CPU, where GetParam() = ({ ({}, { { 1, 1, 1, 3 } }) }, Mod, CONSTANT, VECTOR, 4-byte object <0A-00 00-00>, 4-byte object <00-00 00-00>, 4-byte object <00-00 00-00>, "CPU", {}) (595 ms)
[----------] 1 test from smoke_CompareWithRefs_static/EltwiseLayerTest (598 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (640 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] smoke_CompareWithRefs_static/EltwiseLayerTest.Inference/IS=([]_)_TS=((1.1.1.3)_)_eltwise_op_type=Mod_secondary_input_type=CONSTANT_opType=VECTOR_model_type=i32_InType=undefined_OutType=undefined_trgDev=CPU, where GetParam() = ({ ({}, { { 1, 1, 1, 3 } }) }, Mod, CONSTANT, VECTOR, 4-byte object <0A-00 00-00>, 4-byte object <00-00 00-00>, 4-byte object <00-00 00-00>, "CPU", {})

 1 FAILED TEST

I did not know where I can print the input and output of my implementation operation and the reference implementation operation. Could you please give me some suggestions on debugging?

openvinotoolkit / openvino