search

quinngroup / dr1dl-pyspark

Dictionary Learning in PySpark
Apache License 2.0
1 stars 1 forks source link

LinAlg operations in PySpark #38

Closed magsol closed 8 years ago

magsol commented 8 years ago

The PySpark API has improved considerably in the last several months--there are now several data structures and distributed methods that can be used in native PySpark.

For generating random vectors / matrices:

Distributed data structures and primitives:

However, the thunder-project also has very mature Python-based distributed linear algebra structures and methods built on top of Spark that we can use.

magsol commented 8 years ago

Beyond thunder-project, this ticket is not useful.