building hip-python from sources fails

[trz42]

Attempted to build hip-python from sources following https://github.com/ROCmSoftwarePlatform/hip-python#build-from-source with

./build_hip_python_pkgs.sh --pre-clean --no-cuda --no-docs --no-api-docs --no-clean-docs --no-archive

This resulted in some errors such as

./hip/chip.c:25019:123: error: parameter 2 (‘__pyx_v_attr’) has incomplete type
 static enum hipError_t __pyx_f_3hip_4chip_hipDeviceGetGraphMemAttribute(int __pyx_v_device, enum hipGraphMemAttributeType __pyx_v_attr, void *__pyx_v_value) {
                                                                                                                           ^~~~~~~~~~~~
./hip/chip.c: In function ‘__pyx_f_3hip_4chip_hipDeviceGetGraphMemAttribute’:
./hip/chip.c:25047:156: error: type of formal parameter 2 is incomplete
   __pyx_t_1 = ((enum hipError_t (*)(int, enum hipGraphMemAttributeType, void *))__pyx_v_3hip_4chip__hipDeviceGetGraphMemAttribute__funptr)(__pyx_v_device, __pyx_v_attr, __pyx_v_value); if (unlikely(__Pyx_ErrOccurredWithGIL())) __PYX_ERR(0, 6931, __pyx_L1_error)

and

./hip/chip.c: At top level:
./hip/chip.c:25208:63: error: unknown type name ‘hipUserObject_t’; did you mean ‘hipTextureObject_t’?
 static enum hipError_t __pyx_f_3hip_4chip_hipUserObjectCreate(hipUserObject_t *__pyx_v_object_out, void *__pyx_v_ptr, hipHostFn_t __pyx_v_destroy, unsigned int __pyx_v_initialRefcount, unsigned int __pyx_v_flags) {
                                                               ^~~~~~~~~~~~~~~
                                                               hipTextureObject_t

What could cause this? Maybe the installed rocm is too old? Is there a minimum version of rocm required such that hip-python can be built?

[domcharrier]

Hi,

thanks for submitting this issue. What ROCm version do you have installed on your system right now?

• main is currently based on release branch release/rocm-rel-5.6.1, which in turn assumes ROCm 5.6.1 header files when you built from source.
  • In the linked PR, I try to describe this a little better in the README.md.
• Generally, it is recommended to use the respective release branch for your particular ROCm installation (the PR also adds this part to the README.md).

[trz42]

Thanks for the clarification. Currently, we have ROCm 5.2.3 on our system. It seems, we may not upgrade very soon (not before end of this year for sure) due to other components relying on that particular version.

Would it make sense to just try building hip-python in a container which provides ROCm 5.6.1, gcc and Python? Here we would be interested if it builds or not.

[domcharrier]

Technically this would work but might be overkill for the reasons listed below.

On building HIP Python packages

• Technically, you would only need the ROCm 5.6.1. header files for BUILDING(!) (You can supply the ROCm path to the install script).
• It is possible to build the packages without any AMD GPU driver installed or any AMD GPU installed!

On using HIP Python packages in a Python program

• The Python packages all perform "runtime linking", i.e. a HIP C function symbol is only looked up within the respective ROCm HIP SDK library objects when actually needed!

• Implication: You might mix and match HIP Python packages with various ROCm HIP SDK versions as long as you use a subset of functions where the ABI has not changed between releases.

  • This might already work for you if you use a stable subset of the ROCm HIP SDK API and might be worth trying out.

  Checking if there is a prebuilt package for my Python installation

  If you click on the below link, you will see a list of prebuilt binary packages https://test.pypi.org/simple/hip-python/

• The manylinux_2_17 tag constrains the compatible Linux OS, 2_17 refers to the GLIBC version 2.17; more details: https://github.com/pypa/manylinux/blob/main/README.rst

• The -cp3X indicates the Python version that the package requires to run.

Final thoughts

The conda Python environment manager (installed via miniconda distribution) allows to create environment that use particular Python versions (and other dependencies). This might be one way to get around the constraint on the Python version that is installed on your system.