ROCm / apex

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch
BSD 3-Clause "New" or "Revised" License
17 stars 14 forks source link

change compute type for F16 wrapper around cublas GEMMEx #133

Closed suachong closed 2 months ago

suachong commented 2 months ago

Change the compute type for the F16 wrapper around cublas GEMMEx.

The original code had CUDA_R_32F but was changed to CUBLAS_16F inadvertently during the rocblas->hipblas changes in PyTorch. Hence, the bug was introduced.

This change will allow the unit test for test_fused_dense.py to pass sucessfully.

/opt/rocm/apex/apex/contrib/test/fused_dense# pytest -v 
============================================================================== test session starts ===============================================================================
platform linux -- Python 3.9.19, pytest-7.3.2, pluggy-1.4.0 -- /opt/conda/envs/py_3.9/bin/python
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/opt/rocm-6.1.0/apex/apex/contrib/test/fused_dense/.hypothesis/examples')
rootdir: /opt/rocm-6.1.0/apex
plugins: xdist-3.3.1, xdoctest-1.1.0, hypothesis-5.35.1, flakefinder-1.1.0, rerunfailures-14.0, shard-0.1.2, cpp-2.3.0
collected 1 item                                                                                                                                                                 
Running 1 items in this shard: apex/contrib/test/fused_dense/test_fused_dense.py::FusedDenseTest::test_fused_dense

test_fused_dense.py::FusedDenseTest::test_fused_dense PASSED                                                                                                               [100%]
=============================================================================== 1 passed in 6.83s ================================================================================
pruthvistony commented 2 months ago

@ramcherukuri , I believe this change may need to be cherry-picked to other release branches too. Not sure when it was introduced.