search

rajasekarv / vega

A new arguably faster implementation of Apache Spark from scratch in Rust
Apache License 2.0
2.23k stars 207 forks source link

Shutdown remote executors on ctrl-c on driver #85

Closed iduartgomez closed 4 years ago

iduartgomez commented 4 years ago

This handles shut down of remote workers, main change in file context.rs. Tested in distributed mode and works correctly.

iduartgomez commented 4 years ago

Removed dependency (tokio already includes the functionality to handle this) and added shuffle data clean up too