search

ROCm / rpp

AMD ROCm Performance Primitives (RPP) library is a comprehensive high-performance computer vision library for AMD processors with HIP/OpenCL/CPU back-ends.
https://rocm.docs.amd.com/projects/rpp/en/latest/
MIT License
53 stars 38 forks source link

RPP Gaussian Noise Voxel Tensor on HOST and HIP #323

Closed r-abishek closed 5 months ago

r-abishek commented 6 months ago
r-abishek commented 6 months ago

@kiritigowda SLES and Ubuntu passing. RHEL says: [Configuration error: Could not find any CPUs in the range 384 to 447.]

kiritigowda commented 6 months ago

@r-abishek -- failing CI

kiritigowda commented 5 months ago

@r-abishek - needs conflicts resolved

LakshmiKumar23 commented 5 months ago

@r-abishek Seeing this warning with convert:not found, but the tests pass with both HIP & HOST. Please check once.

svcbuild@Legolas:~/work/rpp/utilities/test_suite/HOST$ python3 runTests_voxel.py  --case_list 6
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is Clang 14.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using Turbo JPEG --
        Libraries:/lib/libturbojpeg.so
        Includes:/usr/include
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp=libomp (found version "5.0")
-- Found OpenMP: TRUE (found version "4.5")
-- test_suite/HOST set to build with rpp and TurboJpeg
-- test_suite/HOST set to build with rpp and OpenCV
-- test_suite/HOST set to build with rpp and libniftiio and OpenCV
-- test_suite/HOST set to build with rpp and libsndfile
-- Configuring done
-- Generating done
-- Build files have been written to: /home/svcbuild/work/rpp/utilities/test_suite/HOST/build
[  8%] Building CXX object CMakeFiles/BatchPD_host_pln3.dir/opt/rocm/share/rpp/test/rpp-performancetests/HOST_NEW/BatchPD_host_pln3.cpp.o
[ 16%] Building CXX object CMakeFiles/Tensor_host.dir/Tensor_host.cpp.o
[ 25%] Building CXX object CMakeFiles/BatchPD_host_pln1.dir/opt/rocm/share/rpp/test/rpp-performancetests/HOST_NEW/BatchPD_host_pln1.cpp.o
[ 33%] Building CXX object CMakeFiles/Tensor_host_audio.dir/Tensor_host_audio.cpp.o
[ 41%] Building CXX object CMakeFiles/BatchPD_host_pkd3.dir/opt/rocm/share/rpp/test/rpp-performancetests/HOST_NEW/BatchPD_host_pkd3.cpp.o
[ 50%] Building CXX object CMakeFiles/Tensor_voxel_host.dir/Tensor_voxel_host.cpp.o
[ 58%] Linking CXX executable BatchPD_host_pkd3
[ 66%] Linking CXX executable BatchPD_host_pln1
[ 75%] Linking CXX executable BatchPD_host_pln3
[ 75%] Built target BatchPD_host_pkd3
[ 75%] Built target BatchPD_host_pln1
[ 83%] Linking CXX executable Tensor_voxel_host
[ 83%] Built target BatchPD_host_pln3
[ 83%] Built target Tensor_voxel_host
[ 91%] Linking CXX executable Tensor_host_audio
[100%] Linking CXX executable Tensor_host
[100%] Built target Tensor_host_audio
[100%] Built target Tensor_host

##########################################################################################
Running all layout Inputs...
##########################################################################################

--------------------------------
Running a New Functionality...
--------------------------------

Running New Bit Depth...
-------------------------

./Tensor_voxel_host /home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL /home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL /home/svcbuild/work/rpp/utilities/test_suite/HOST/OUTPUT_VOXEL_HOST_2024-04-12_09-38-55/rpp_host_pkd3_miscellaneous 0 6 1 0 0 1 0

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/1_nifti_input_50x50x4.nii header information:
NIFTI1 XYZT dimensions: 50 50 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/2_nifti_input_100x100x4.nii header information:
NIFTI1 XYZT dimensions: 100 100 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/3_nifti_input_150x150x4.nii header information:
NIFTI1 XYZT dimensions: 150 150 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

Running gaussian_noise_voxel_u8_Tensor_HOST_PKD3_toPKD3 1 times (each time with a batch size of 1 images) and computing mean statistics...
The functionality doesn't yet exist in RPP

------------------------------------------------------------------------------------------

Running New Bit Depth...
-------------------------

./Tensor_voxel_host /home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL /home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL /home/svcbuild/work/rpp/utilities/test_suite/HOST/OUTPUT_VOXEL_HOST_2024-04-12_09-38-55/rpp_host_pkd3_miscellaneous 0 6 1 0 0 1 2
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/1_nifti_input_50x50x4.nii header information:
NIFTI1 XYZT dimensions: 50 50 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/2_nifti_input_100x100x4.nii header information:
NIFTI1 XYZT dimensions: 100 100 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/3_nifti_input_150x150x4.nii header information:
NIFTI1 XYZT dimensions: 150 150 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

Running gaussian_noise_voxel_f32_Tensor_HOST_PKD3_toPKD3 1 times (each time with a batch size of 1 images) and computing mean statistics...

CPU Backend Wall Time: 3.10993 ms per batch

CPU Backend Wall Time: 0.693083 ms per batch

CPU Backend Wall Time: 1.22786 ms per batch

------------------------------------------------------------------------------------------

--------------------------------
Running a New Functionality...
--------------------------------

Running New Bit Depth...
-------------------------

./Tensor_voxel_host /home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL /home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL /home/svcbuild/work/rpp/utilities/test_suite/HOST/OUTPUT_VOXEL_HOST_2024-04-12_09-38-55/rpp_host_pln3_miscellaneous 1 6 1 0 0 1 0

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/1_nifti_input_50x50x4.nii header information:
NIFTI1 XYZT dimensions: 50 50 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/2_nifti_input_100x100x4.nii header information:
NIFTI1 XYZT dimensions: 100 100 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/3_nifti_input_150x150x4.nii header information:
NIFTI1 XYZT dimensions: 150 150 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

Running gaussian_noise_voxel_u8_Tensor_HOST_PLN3_toPLN3 1 times (each time with a batch size of 1 images) and computing mean statistics...
The functionality doesn't yet exist in RPP

------------------------------------------------------------------------------------------

Running New Bit Depth...
-------------------------

./Tensor_voxel_host /home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL /home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL /home/svcbuild/work/rpp/utilities/test_suite/HOST/OUTPUT_VOXEL_HOST_2024-04-12_09-38-55/rpp_host_pln3_miscellaneous 1 6 1 0 0 1 2
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/1_nifti_input_50x50x4.nii header information:
NIFTI1 XYZT dimensions: 50 50 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/2_nifti_input_100x100x4.nii header information:
NIFTI1 XYZT dimensions: 100 100 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/3_nifti_input_150x150x4.nii header information:
NIFTI1 XYZT dimensions: 150 150 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

Running gaussian_noise_voxel_f32_Tensor_HOST_PLN3_toPLN3 1 times (each time with a batch size of 1 images) and computing mean statistics...

CPU Backend Wall Time: 2.85411 ms per batch

CPU Backend Wall Time: 0.608921 ms per batch

CPU Backend Wall Time: 1.00803 ms per batch

------------------------------------------------------------------------------------------

--------------------------------
Running a New Functionality...
--------------------------------

Running New Bit Depth...
-------------------------

./Tensor_voxel_host /home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL /home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL /home/svcbuild/work/rpp/utilities/test_suite/HOST/OUTPUT_VOXEL_HOST_2024-04-12_09-38-55/rpp_host_pln1_miscellaneous 2 6 1 0 0 1 0

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/1_nifti_input_50x50x4.nii header information:
NIFTI1 XYZT dimensions: 50 50 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/2_nifti_input_100x100x4.nii header information:
NIFTI1 XYZT dimensions: 100 100 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/3_nifti_input_150x150x4.nii header information:
NIFTI1 XYZT dimensions: 150 150 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

Running gaussian_noise_voxel_u8_Tensor_HOST_PLN1_toPLN1 1 times (each time with a batch size of 1 images) and computing mean statistics...
The functionality doesn't yet exist in RPP

------------------------------------------------------------------------------------------

Running New Bit Depth...
-------------------------

./Tensor_voxel_host /home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL /home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL /home/svcbuild/work/rpp/utilities/test_suite/HOST/OUTPUT_VOXEL_HOST_2024-04-12_09-38-55/rpp_host_pln1_miscellaneous 2 6 1 0 0 1 2
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/1_nifti_input_50x50x4.nii header information:
NIFTI1 XYZT dimensions: 50 50 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/2_nifti_input_100x100x4.nii header information:
NIFTI1 XYZT dimensions: 100 100 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/3_nifti_input_150x150x4.nii header information:
NIFTI1 XYZT dimensions: 150 150 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

Running gaussian_noise_voxel_f32_Tensor_HOST_PLN1_toPLN1 1 times (each time with a batch size of 1 images) and computing mean statistics...

CPU Backend Wall Time: 2.75517 ms per batch

CPU Backend Wall Time: 0.329971 ms per batch

CPU Backend Wall Time: 0.323057 ms per batch

------------------------------------------------------------------------------------------
r-abishek commented 5 months ago

@kiritigowda @LakshmiKumar23 Merge conflicts resolved and CI passing

sampath1117 commented 5 months ago

@r-abishek Seeing this warning with convert:not found, but the tests pass with both HIP & HOST. Please check once.

svcbuild@Legolas:~/work/rpp/utilities/test_suite/HOST$ python3 runTests_voxel.py  --case_list 6
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is Clang 14.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using Turbo JPEG --
        Libraries:/lib/libturbojpeg.so
        Includes:/usr/include
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp=libomp (found version "5.0")
-- Found OpenMP: TRUE (found version "4.5")
-- test_suite/HOST set to build with rpp and TurboJpeg
-- test_suite/HOST set to build with rpp and OpenCV
-- test_suite/HOST set to build with rpp and libniftiio and OpenCV
-- test_suite/HOST set to build with rpp and libsndfile
-- Configuring done
-- Generating done
-- Build files have been written to: /home/svcbuild/work/rpp/utilities/test_suite/HOST/build
[  8%] Building CXX object CMakeFiles/BatchPD_host_pln3.dir/opt/rocm/share/rpp/test/rpp-performancetests/HOST_NEW/BatchPD_host_pln3.cpp.o
[ 16%] Building CXX object CMakeFiles/Tensor_host.dir/Tensor_host.cpp.o
[ 25%] Building CXX object CMakeFiles/BatchPD_host_pln1.dir/opt/rocm/share/rpp/test/rpp-performancetests/HOST_NEW/BatchPD_host_pln1.cpp.o
[ 33%] Building CXX object CMakeFiles/Tensor_host_audio.dir/Tensor_host_audio.cpp.o
[ 41%] Building CXX object CMakeFiles/BatchPD_host_pkd3.dir/opt/rocm/share/rpp/test/rpp-performancetests/HOST_NEW/BatchPD_host_pkd3.cpp.o
[ 50%] Building CXX object CMakeFiles/Tensor_voxel_host.dir/Tensor_voxel_host.cpp.o
[ 58%] Linking CXX executable BatchPD_host_pkd3
[ 66%] Linking CXX executable BatchPD_host_pln1
[ 75%] Linking CXX executable BatchPD_host_pln3
[ 75%] Built target BatchPD_host_pkd3
[ 75%] Built target BatchPD_host_pln1
[ 83%] Linking CXX executable Tensor_voxel_host
[ 83%] Built target BatchPD_host_pln3
[ 83%] Built target Tensor_voxel_host
[ 91%] Linking CXX executable Tensor_host_audio
[100%] Linking CXX executable Tensor_host
[100%] Built target Tensor_host_audio
[100%] Built target Tensor_host

##########################################################################################
Running all layout Inputs...
##########################################################################################

--------------------------------
Running a New Functionality...
--------------------------------

Running New Bit Depth...
-------------------------

./Tensor_voxel_host /home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL /home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL /home/svcbuild/work/rpp/utilities/test_suite/HOST/OUTPUT_VOXEL_HOST_2024-04-12_09-38-55/rpp_host_pkd3_miscellaneous 0 6 1 0 0 1 0

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/1_nifti_input_50x50x4.nii header information:
NIFTI1 XYZT dimensions: 50 50 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/2_nifti_input_100x100x4.nii header information:
NIFTI1 XYZT dimensions: 100 100 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

/home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL/3_nifti_input_150x150x4.nii header information:
NIFTI1 XYZT dimensions: 150 150 4 1
NIFTI1 Datatype code and bits/pixel: 4 16
NIFTI1 Scaling slope and intercept: 0.000000 0.000000
NIFTI1 Byte offset to data in datafile: 2896

Running gaussian_noise_voxel_u8_Tensor_HOST_PKD3_toPKD3 1 times (each time with a batch size of 1 images) and computing mean statistics...
The functionality doesn't yet exist in RPP

------------------------------------------------------------------------------------------

Running New Bit Depth...
-------------------------

./Tensor_voxel_host /home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL /home/svcbuild/work/rpp/utilities/test_suite/HOST/../TEST_QA_IMAGES_VOXEL /home/svcbuild/work/rpp/utilities/test_suite/HOST/OUTPUT_VOXEL_HOST_2024-04-12_09-38-55/rpp_host_pkd3_miscellaneous 0 6 1 0 0 1 2
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found
sh: 1: convert: not found

Hi @LakshmiKumar23 For the voxel test suite, In case of unit test mode (where outputs will be displayed to user)

we are also converting the outputs to .gif to show how the outputs are changing along depth dimension. For converting to gif we need to install the image-magick library which is a part of test suite prerequisites image

If that library is not installed, this warning will be shown to user sh: 1: convert: not found

Also please note that for gaussian noise augmentation QAmode is not supported since the outputs generated will be random for each run and the random number generation also varies between HOST and HIP. So it wont show any PASS/FAIL for gaussian noise, Only Unit test and performance test is supported for this case

rrawther commented 5 months ago

please address review comments before merge