techascent / tech.datatype

Efficient numerics for the jvm
Eclipse Public License 1.0
83 stars 8 forks source link

Fastutil and buffer #3

Closed cnuernber closed 5 years ago

cnuernber commented 5 years ago

Built up from random access read/write interfaces all the way to tensors and blas-accelerated matrix multiply.

Eliminates need for tech.compute outside of TVM. So, for instance, opencv doesn't need a compute dependency any more. TVM will need some refactoring and all the compute infrastructure will be significantly reduced as it will now assume some level of host compilation process and thus doesn't need hardcoded interfaces for a set of things. It also means that it can be done in a more functional manner as TVM's algorithmic language is functional. Potentially just becomes wrapper around running tvm like things on jvm as well as tvm.

Because I now expect the really perf sensitive things to be custom-compiled the datatype library can be much, much more generic and thus greatly simplified with regards to actual algorithmic design. It was expanded in terms of generality of types and generality of computation supported, hence the ability to do a simple APL program in both sparse and dense systems.

This makes doing things like the dataset and by extension the ml wrapper library much much easier as the number of concepts that need to be attached to, for instance, the tablesaw column datatype far fewer and possibilities for an expanded and corrected (I can account for missing values in the reader/writer interfaces) set of functionality.