Thought out approach to ONF optimizations

[costrouc]

MOA has the ability with multiple backends. Generally these optimizations fall under loop optimizations. Wikipedia has a good overview approaches are polyhedral and unimodular transformation. Immediate concerns since I am collecting multiple issues. Additional concerns are avx instructions (most likely a compiler should handle).

• [ ] loop fusion
• [ ] loop ordering
• [ ] loop reduction (psi reduction)
• [ ] reorder expressions (distributive property +-*/)
• [ ] dimension lifting for parallelizing algorithms
• [ ] cache intermediate results (immediately obvious in multiple matrix multiply)

Some frameworks claim to handle pieces of this problem so they should be looked into.

• https://github.com/inducer/loopy
• https://github.com/tensorflow/mlir

This consumes issues #13, #5

[rgommers]

@costrouc it would be good to summarize the options and pros/cons here. E.g. using the low-level thing that's the actual optimizer under loopy.

[rgommers]

@costrouc to summarize everything, and then consider this issue done - this topic will then be tackled piece by piece.

[costrouc]

Addressed in 26e4812bb4a3e132c845f82ed508973c8f4ffcbc