Open luweizheng opened 1 month ago
Attention: Patch coverage is 19.35484%
with 75 lines
in your changes missing coverage. Please review.
Project coverage is 52.76%. Comparing base (
a3af71f
) to head (77cf425
).
Files | Patch % | Lines |
---|---|---|
python/xorbits/_mars/services/storage/handler.py | 15.27% | 61 Missing :warning: |
python/xorbits/_mars/services/storage/transfer.py | 22.22% | 14 Missing :warning: |
:exclamation: There is a different number of reports uploaded between BASE (a3af71f) and HEAD (77cf425). Click for more details.
HEAD has 10 uploads less than BASE
| Flag | BASE (a3af71f) | HEAD (77cf425) | |------|------|------| |unittests|11|1|
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Fix storage transfer deadlock of CUDA storage on multiple GPUs. I do this work based on https://github.com/xorbitsai/xorbits/pull/488
The current implementation of the transfer function leads to a deadlock when executing Xorbits on multiple GPUs. The issue arises from the
StorageHandlerActor.fetch_batch
function, which invokesSenderManagerActor.send_batch_data
and subsequently callsStorageHandlerActor.request_quota_with_spill
. Due to the locking mechanism within the StorageHandlerActor method call, a deadlock arises.NOTEs:
test_transfer_gpu.py
of ucx channel has bugs while socket channel works.Check code requirements