All tests passed in dask-cuda, full logs.
All tests passed in ucx-py-ib-rdmacm-debug-test, full logs.
All tests passed in ucx-py-ib-test, full logs.
All tests passed in ucx-py-libs-ib-test, full logs.
All tests passed in ucx-py-libs-nvlink-test, full logs.
All tests passed in ucx-py-libs-tcp-test, full logs.
All tests passed in ucx-py-nvlink-test, full logs.
All tests passed in ucx-py-tcp-test, full logs.
Performance results
Failures
Scenario numpy-TAG-Core-TCP SM expected_bw=4.0GB/s failed with bw 3.3 GB/s; has never passed
Scenario numpy-TAG-Core-RC expected_bw=16.0GB/s failed with bw 8.22 GB/s; has never passed
Scenario numpy-AM-Core-TCP SM expected_bw=2.8GB/s failed with bw 2.62 GB/s; has never passed
Scenario numpy-AM-Core-RC expected_bw=3.0GB/s failed with bw 2.09 GB/s; has never passed
Scenario numpy-TAG-Async-TCP SM expected_bw=4.0GB/s failed with bw 3.64 GB/s; has never passed
Scenario numpy-TAG-Async-RC expected_bw=16.0GB/s failed with bw 8.11 GB/s; has never passed
Scenario numpy-AM-Async-TCP SM expected_bw=2.8GB/s failed with bw 2.6 GB/s; has never passed
Scenario numpy-AM-Async-RC expected_bw=3.0GB/s failed with bw 2.09 GB/s; has never passed
Scenario cupy-TAG-Core-TCP SM expected_bw=3.0GB/s failed with bw 2.44 GB/s; has never passed
Scenario cupy-TAG-Core-TCP expected_bw=3.0GB/s failed with bw 2.43 GB/s; has never passed
Scenario cupy-TAG-Core-RC, expected_bw=12.0GB/s failed with bw 11.36GB/s; last pass on 2024-09-13T22:00:00 (UCX-Py version 0.40.0; UCX commit 27ab197)
Scenario cupy-AM-Core-TCP SM expected_bw=3.0GB/s failed with bw 2.43 GB/s; has never passed
Scenario cupy-AM-Core-TCP expected_bw=3.0GB/s failed with bw 2.43 GB/s; has never passed
Scenario cupy-AM-Core-RC, expected_bw=12.0GB/s failed with bw 11.26GB/s; last pass on 2024-09-05T22:00:00 (UCX-Py version 0.40.0; UCX commit 2a78fca)
Scenario cupy-TAG-Async-TCP SM expected_bw=3.0GB/s failed with bw 2.41 GB/s; has never passed
Scenario cupy-TAG-Async-TCP expected_bw=3.0GB/s failed with bw 2.41 GB/s; has never passed
Scenario cupy-TAG-Async-RC, expected_bw=12.0GB/s failed with bw 11.3GB/s; last pass on 2024-05-08T05:00:00 (UCX-Py version 0.38.0; UCX commit 63f85a9)
Scenario cupy-AM-Async-TCP SM expected_bw=3.0GB/s failed with bw 2.42 GB/s; has never passed
Scenario cupy-AM-Async-TCP expected_bw=3.0GB/s failed with bw 2.39 GB/s; has never passed
Scenario cupy-AM-Async-RC, expected_bw=12.0GB/s failed with bw 11.3GB/s; last pass on 2024-05-09T05:00:00 (UCX-Py version 0.38.0; UCX commit f02bc54)
Passes
Scenario numpy-TAG-Core-TCP, expected_bw=2.8GB/s passed with bw 3.33GB/s
Scenario numpy-AM-Core-TCP, expected_bw=2.3GB/s passed with bw 2.62GB/s
Scenario numpy-TAG-Async-TCP, expected_bw=2.8GB/s passed with bw 3.64GB/s
Scenario numpy-AM-Async-TCP, expected_bw=2.3GB/s passed with bw 2.6GB/s
Scenario cupy-TAG-Core-CUDA_IPC_SELF, expected_bw=370.0GB/s passed with bw 373.03GB/s
Scenario cupy-TAG-Core-CUDA_IPC_NV2, expected_bw=48.0GB/s passed with bw 48.44GB/s
Scenario cupy-TAG-Core-CUDA_IPC_NV1, expected_bw=24.0GB/s passed with bw 24.25GB/s
Scenario cupy-AM-Core-CUDA_IPC_SELF, expected_bw=370.0GB/s passed with bw 369.78GB/s
Scenario cupy-AM-Core-CUDA_IPC_NV2, expected_bw=48.0GB/s passed with bw 48.37GB/s
Scenario cupy-AM-Core-CUDA_IPC_NV1, expected_bw=24.0GB/s passed with bw 24.22GB/s
Scenario cupy-TAG-Async-CUDA_IPC_SELF, expected_bw=325.0GB/s passed with bw 326.06GB/s
Scenario cupy-TAG-Async-CUDA_IPC_NV2, expected_bw=48.0GB/s passed with bw 47.3GB/s
Scenario cupy-TAG-Async-CUDA_IPC_NV1, expected_bw=24.0GB/s passed with bw 23.94GB/s
Scenario cupy-AM-Async-CUDA_IPC_SELF, expected_bw=325.0GB/s passed with bw 325.71GB/s
Scenario cupy-AM-Async-CUDA_IPC_NV2, expected_bw=48.0GB/s passed with bw 47.24GB/s
Scenario cupy-AM-Async-CUDA_IPC_NV1, expected_bw=24.0GB/s passed with bw 23.93GB/s
Test results
Passes
All tests passed in
dask-cuda
, full logs. All tests passed inucx-py-ib-rdmacm-debug-test
, full logs. All tests passed inucx-py-ib-test
, full logs. All tests passed inucx-py-libs-ib-test
, full logs. All tests passed inucx-py-libs-nvlink-test
, full logs. All tests passed inucx-py-libs-tcp-test
, full logs. All tests passed inucx-py-nvlink-test
, full logs. All tests passed inucx-py-tcp-test
, full logs.Performance results
Failures
Scenario numpy-TAG-Core-TCP SM expected_bw=4.0GB/s failed with bw 3.3 GB/s; has never passed
Scenario numpy-TAG-Core-RC expected_bw=16.0GB/s failed with bw 8.22 GB/s; has never passed
Scenario numpy-AM-Core-TCP SM expected_bw=2.8GB/s failed with bw 2.62 GB/s; has never passed
Scenario numpy-AM-Core-RC expected_bw=3.0GB/s failed with bw 2.09 GB/s; has never passed
Scenario numpy-TAG-Async-TCP SM expected_bw=4.0GB/s failed with bw 3.64 GB/s; has never passed
Scenario numpy-TAG-Async-RC expected_bw=16.0GB/s failed with bw 8.11 GB/s; has never passed
Scenario numpy-AM-Async-TCP SM expected_bw=2.8GB/s failed with bw 2.6 GB/s; has never passed
Scenario numpy-AM-Async-RC expected_bw=3.0GB/s failed with bw 2.09 GB/s; has never passed
Scenario cupy-TAG-Core-TCP SM expected_bw=3.0GB/s failed with bw 2.44 GB/s; has never passed
Scenario cupy-TAG-Core-TCP expected_bw=3.0GB/s failed with bw 2.43 GB/s; has never passed
Scenario cupy-TAG-Core-RC, expected_bw=12.0GB/s failed with bw 11.36GB/s; last pass on 2024-09-13T22:00:00 (UCX-Py version 0.40.0; UCX commit 27ab197)
Scenario cupy-AM-Core-TCP SM expected_bw=3.0GB/s failed with bw 2.43 GB/s; has never passed
Scenario cupy-AM-Core-TCP expected_bw=3.0GB/s failed with bw 2.43 GB/s; has never passed
Scenario cupy-AM-Core-RC, expected_bw=12.0GB/s failed with bw 11.26GB/s; last pass on 2024-09-05T22:00:00 (UCX-Py version 0.40.0; UCX commit 2a78fca)
Scenario cupy-TAG-Async-TCP SM expected_bw=3.0GB/s failed with bw 2.41 GB/s; has never passed
Scenario cupy-TAG-Async-TCP expected_bw=3.0GB/s failed with bw 2.41 GB/s; has never passed
Scenario cupy-TAG-Async-RC, expected_bw=12.0GB/s failed with bw 11.3GB/s; last pass on 2024-05-08T05:00:00 (UCX-Py version 0.38.0; UCX commit 63f85a9)
Scenario cupy-AM-Async-TCP SM expected_bw=3.0GB/s failed with bw 2.42 GB/s; has never passed
Scenario cupy-AM-Async-TCP expected_bw=3.0GB/s failed with bw 2.39 GB/s; has never passed
Scenario cupy-AM-Async-RC, expected_bw=12.0GB/s failed with bw 11.3GB/s; last pass on 2024-05-09T05:00:00 (UCX-Py version 0.38.0; UCX commit f02bc54)
Passes
Scenario numpy-TAG-Core-TCP, expected_bw=2.8GB/s passed with bw 3.33GB/s
Scenario numpy-AM-Core-TCP, expected_bw=2.3GB/s passed with bw 2.62GB/s
Scenario numpy-TAG-Async-TCP, expected_bw=2.8GB/s passed with bw 3.64GB/s
Scenario numpy-AM-Async-TCP, expected_bw=2.3GB/s passed with bw 2.6GB/s
Scenario cupy-TAG-Core-CUDA_IPC_SELF, expected_bw=370.0GB/s passed with bw 373.03GB/s
Scenario cupy-TAG-Core-CUDA_IPC_NV2, expected_bw=48.0GB/s passed with bw 48.44GB/s
Scenario cupy-TAG-Core-CUDA_IPC_NV1, expected_bw=24.0GB/s passed with bw 24.25GB/s
Scenario cupy-AM-Core-CUDA_IPC_SELF, expected_bw=370.0GB/s passed with bw 369.78GB/s
Scenario cupy-AM-Core-CUDA_IPC_NV2, expected_bw=48.0GB/s passed with bw 48.37GB/s
Scenario cupy-AM-Core-CUDA_IPC_NV1, expected_bw=24.0GB/s passed with bw 24.22GB/s
Scenario cupy-TAG-Async-CUDA_IPC_SELF, expected_bw=325.0GB/s passed with bw 326.06GB/s
Scenario cupy-TAG-Async-CUDA_IPC_NV2, expected_bw=48.0GB/s passed with bw 47.3GB/s
Scenario cupy-TAG-Async-CUDA_IPC_NV1, expected_bw=24.0GB/s passed with bw 23.94GB/s
Scenario cupy-AM-Async-CUDA_IPC_SELF, expected_bw=325.0GB/s passed with bw 325.71GB/s
Scenario cupy-AM-Async-CUDA_IPC_NV2, expected_bw=48.0GB/s passed with bw 47.24GB/s
Scenario cupy-AM-Async-CUDA_IPC_NV1, expected_bw=24.0GB/s passed with bw 23.93GB/s