Legion: Add Complex/Half Reductions for HIP

[syamajala]

Building complex reductions using HIP currently fails with the following error:

[ 59%] Building HIPCC object runtime/CMakeFiles/hip_compile.dir/hip_compile_generated_legion_redop.cpp.o               
In file included from /gpfs/alpine/cmb103/scratch/seshuy/legion_s3d_nscbc_spock_gasnet1/legion/language/build/runtime/legion_redop.cpp:23:                                                                                                    
In file included from /gpfs/alpine/cmb103/scratch/seshuy/legion_s3d_nscbc_spock_gasnet1/legion/runtime/legion.h:59:    
In file included from /gpfs/alpine/cmb103/scratch/seshuy/legion_s3d_nscbc_spock_gasnet1/legion/runtime/legion/legion_redop.h:1904:                                                                                                            
/gpfs/alpine/cmb103/scratch/seshuy/legion_s3d_nscbc_spock_gasnet1/legion/runtime/legion/legion_redop.inl:440:34: error:
 invalid output constraint '=l' in asm                                                                                     asm("mov.b64 %0, {%1,%2};" : "=l"(result) : "f"(value.real()), "f"(value.imag()));        
                                 ^                                                                                     
/gpfs/alpine/cmb103/scratch/seshuy/legion_s3d_nscbc_spock_gasnet1/legion/runtime/legion/legion_redop.inl:448:34: error: invalid output constraint '=f' in asm                                                                                 
    asm("mov.b64 {%0,%1}, %2;" : "=f"(real), "=f"(imag) : "l"(value));                                                 
                                 ^                                                                                     
2 errors generated when compiling for gfx908.                                                                          
CMake Error at hip_compile_generated_legion_redop.cpp.o.cmake:200 (message):                                           
  Error generating file                                                                                                
  /gpfs/alpine/cmb103/scratch/seshuy/legion_s3d_nscbc_spock_gasnet1/legion/language/build/runtime/CMakeFiles/hip_compile.dir//./hip_compile_generated_legion_redop.cpp.o 

[eddy16112]

I added Half to the title as complex and half reductions are similar.

CUDA use inline PTX to reinterpret data type. PTX is not supported in HIP.

For complex I fall back to use union, which is fine in C. Here is the commit, however, there is an error inthrust/complex.h, looks like the HIP version of thrust is copied from CUDA, but missing files. Here are the errors:

In file included from /opt/rocm-4.3.1/hip/../include/thrust/system/cuda/detail/execution_policy.h:33,
                 from /opt/rocm-4.3.1/hip/../include/thrust/iterator/detail/device_system_tag.h:23,
                 from /opt/rocm-4.3.1/hip/../include/thrust/iterator/iterator_traits.h:111,
                 from /opt/rocm-4.3.1/hip/../include/thrust/detail/type_traits/pointer_traits.h:23,
                 from /opt/rocm-4.3.1/hip/../include/thrust/type_traits/is_contiguous_iterator.h:27,
                 from /opt/rocm-4.3.1/hip/../include/thrust/type_traits/is_trivially_relocatable.h:19,
                 from /opt/rocm-4.3.1/hip/../include/thrust/detail/complex/complex.inl:20,
                 from /opt/rocm-4.3.1/hip/../include/thrust/complex.h:1032,
                 from /home/wwu/legion-hip3-noctx/runtime/mathtypes/complex.h:38,
                 from /home/wwu/legion-hip3-noctx/runtime/legion/legion_redop.h:25,
                 from /home/wwu/legion-hip3-noctx/runtime/legion.h:59,
                 from /home/wwu/legion-hip3-noctx/runtime/legion/legion_trace.cc:17:
/opt/rocm-4.3.1/hip/../include/thrust/system/cuda/config.h:76:10: fatal error: cub/util_namespace.cuh: No such file or directory
 #include <cub/util_namespace.cuh> // This includes <cub/version.cuh> in newer releases.

For half __short_as_half and other related conversion functions are missing in HIP, will keep an eye on the future releases of HIP.

[eddy16112]

I forgot to include the hip_fp16.h, now the half reduction support has been merged into master.

[eddy16112]

I have added the complex support for HIP. Please download the old version of thrust from https://github.com/ROCmSoftwarePlatform/Thrust Somehow the latest thrust (https://github.com/ROCmSoftwarePlatform/rocThrust) does not work, I have submitted an issue to let AMD take a look at it.

[elliottslaughter]

@syamajala Can you try again? I've merged the CMake patches, and Regent CI on Crusher is currently passing.