search

jakipatryk / spark-persistent-homology

(WIP, not fast enough for any production usage yet) Library for persistent homology computations in Apache Spark.
Apache License 2.0
0 stars 0 forks source link

Avoid global sort by removing `indexInMatrix` #32

Open jakipatryk opened 1 year ago

jakipatryk commented 1 year ago

Subtask of #11.

Global sort is an expensive operation, that doesn't seem to be unavoidable. Try to remove indexInMatrix and all dependencies of it by utylizing threshold instead to bucket the matrix on horizontal axis, and threshold+dim to sort within partition.

jakipatryk commented 1 year ago

Doesn't seem to be (easly) doable. Removing from 0.1.0 release.