Can the sparkucx can be used when UCX_TLS=tcp ?

openucx / sparkucx

A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer

https://www.sparkucx.org/

BSD 3-Clause "New" or "Revised" License

44 stars 31 forks source link

Can the sparkucx can be used when UCX_TLS=tcp ? #24

Open X-zhazhaxi opened 4 years ago

X-zhazhaxi commented 4 years ago

There is only TCP support at my machine , and there will be [1597129379.385163] [server:20110:1] select.c:434 UCX ERROR no remote registered memory access transport to client:6868: tcp/bond0 - no put short, tcp/eth0 - no put short, sockcm/sockaddr - no pending [1597129379.385171] [server:20110:1] ucp_listener.c:121 UCX ERROR connection request failed on listener 0x7fedbd6f8480 with status Destination is unreachable

petro-rudenko commented 4 years ago

The current version of SparkUCX utilizes RDMA to accelerate shuffle block transfer. We're working on a version, that would have additional protocols, so it should work with TCP transport.

X-zhazhaxi commented 4 years ago

Thanks And when the version with TCP transport will be release?

petro-rudenko commented 4 years ago

We expect to have it in a few months.

X-zhazhaxi commented 4 years ago

Thanks