How to close BlazeShuffleManager?

kwai / blaze

Blazing-fast query execution engine speaks Apache Spark language and has Arrow-DataFusion at its core.

Apache License 2.0

937 stars 88 forks source link

How to close BlazeShuffleManager? #499

Open dpengpeng opened 1 week ago

dpengpeng commented 1 week ago

How to close BlazeShuffleManager?I want to test performance without BlazeShuffleManager. If i delete "--conf spark.shuffle.manager=org.apache.spark.sql.execution.blaze.shuffle.BlazeShuffleManager", Spark Job will fail.

richox commented 5 days ago

there is no way to disable BlazeShuffleManager when using BlazeSessionExtension at the moment.