issues
search
openucx
/
sparkucx
A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer
https://www.sparkucx.org/
BSD 3-Clause "New" or "Revised" License
44
stars
31
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
bug fix: metadata buffer resident in Spark driver may go out of range
#41
endersuu
opened
1 year ago
0
will ucx release its support for spark >= 3.1
#40
mtsol
opened
1 year ago
0
java.lang.VerifyError: class org.apache.spark.shuffle.UcxShuffleManager overrides final method getReader.(Lorg/apache/spark/shuffle/ShuffleHandle;IILorg/apache/spark/TaskContext;Lorg/apache/spark/shuffle/ShuffleReadMetricsReporter;)Lorg/apache/spark/shuffle/ShuffleReader;
#39
mtsol
opened
1 year ago
0
Didn't get worker address for BlockManagerId(2, xxx, xxx, None) during 3600
#38
eric-doug
opened
1 year ago
0
Is UCX also supported in spark 3.1.2 with java 11
#37
KunwarAkanksha
opened
2 years ago
1
feat: Support Spark-3.1 version.
#36
trietopsoft
closed
2 years ago
1
Update LICENSE
#35
petro-rudenko
closed
3 years ago
0
Fix errors on UcxNode close.
#34
petro-rudenko
closed
3 years ago
0
Support Spark-2.1 version.
#33
petro-rudenko
closed
3 years ago
0
Update master version with JUCX 1.11-rc3 and update README.
#32
petro-rudenko
closed
3 years ago
0
Latest work done for unified API
#31
petro-rudenko
opened
3 years ago
0
SIGSEGV in JVM Runtime
#30
stevenybw
opened
3 years ago
9
Fix java path in AZP + update JUCX version.
#29
petro-rudenko
closed
3 years ago
1
Avoid get lock from synchronized in every shuffle
#28
wenxuanguan
closed
3 years ago
5
Performance benchmark
#27
petro-rudenko
opened
3 years ago
1
[WIP] Spark Shuffle manager implementation.
#26
petro-rudenko
opened
3 years ago
2
Public transport api implementation
#25
petro-rudenko
closed
4 years ago
5
Can the sparkucx can be used when UCX_TLS=tcp ?
#24
X-zhazhaxi
opened
4 years ago
4
Public Shuffle Transport API.
#23
petro-rudenko
closed
4 years ago
5
Update README and prepare release action.
#22
petro-rudenko
closed
4 years ago
1
[CORE] Spark-3.0 reducer protocol.
#21
petro-rudenko
closed
4 years ago
1
[CORE] Spark-3.0 mapper writer plugin implementation.
#20
petro-rudenko
closed
4 years ago
1
[CORE] use ucx from hpcx module, disable mt workers.
#19
petro-rudenko
closed
4 years ago
1
[CORE] Split common / version dependent code.
#18
petro-rudenko
closed
4 years ago
1
[CORE] Update to jucx-1.8.
#17
petro-rudenko
closed
4 years ago
1
[CORE] Reducer protocol.
#16
petro-rudenko
closed
4 years ago
2
[CORE] Mapper protocol implementation.
#15
petro-rudenko
closed
4 years ago
0
[CORE] UcxShuffleManager implementation.
#14
petro-rudenko
closed
4 years ago
0
[DNM] POC branch
#13
petro-rudenko
closed
4 years ago
0
[CI] Testing script.
#12
petro-rudenko
closed
4 years ago
13
Benchmark on performance
#11
TheForgotten69
opened
4 years ago
0
AZP: implement Azure based CI pipeline
#10
amaslenn
closed
4 years ago
10
[CORE] UcxNode implementation - coordination point for ucp operations.
#9
petro-rudenko
closed
4 years ago
0
CORE: SparkUCX memory pool + common RPC classes.
#8
petro-rudenko
closed
4 years ago
0
[DOC][Minor] Update typos in readme.
#7
petro-rudenko
closed
4 years ago
0
CORE: SparkUCX - initial implementation.
#6
petro-rudenko
closed
4 years ago
0
README: update readme information.
#5
petro-rudenko
closed
4 years ago
6
CI: add github build action.
#4
petro-rudenko
closed
5 years ago
0
BUILD: Add initial files for build and CI
#3
petro-rudenko
closed
4 years ago
3
Add build and styles scripts.
#2
petro-rudenko
closed
5 years ago
0
Add LICENCE and README.
#1
petro-rudenko
closed
5 years ago
0