Open axsaucedo opened 3 years ago
alexander-g
commented
3 years ago 3D is still not enough for many use cases like CNNs. Consider using something like xtensor in combination with push constants for shape/stride parameters. It supports N dimensions and has also python bindings.
axsaucedo
commented
3 years ago I completely agree @alexander-g! I actually did quite a bit of research on this and I have some insights to share which I would be interested in your thoughts..
For the first point, I agree that image3D is not enough. Having said that, in other projects that I have seen handling multi-dimensional data, it seems that the projects just use buffers with respective strides/offsets to handle the dimensionality based on specific abstractions using the flat data structure underneath.
In regards to the underlying implementation, I actually did come across those libraries when doing further research on the C++ libraries available, such as NumCpp and xtensor. It seemed that xtensor was particularly interesting. When delving into xtensor and its internals, it's did seem like a fascinating project (the contributors have a very interesting collaboration with the Jupyter project to integrate xtensor with the interactive C++ Cling interprepter) https://blog.llvm.org/posts/2020-12-21-interactive-cpp-for-data-science/. I did quite a bit of research a few weeks ago around this, and found that there are three things that should be explored further:
1) work required to integrate to a library like xtensor (potentially as one of the backends to xtensor) 2) work required to extend Tensor class to support further functionality similar to xtensor (such as extending operations to be AST instead of linear) 3) work required to add further capabilities to Tensor aligned to numpy internals
It initially seemed that option number 3 may be a reasonable place to start, so I started by doing research on the internals of numpy, which interestingly enough reduced to the foundations of the interpretations of the internal buffers - e.g. offsets, strides, etc (which is quite well documented https://docs.scipy.org/doc/numpy-1.13.0/reference/internals.html#internal-organization-of-numpy-arrays).
However it seemed that largely adding functionality around this would end up quite shallow, as ultimately it's just about adding the concepts of strides and offsets as core part of the buffers. Having said that this is certainly something that I would be keen to explore, and more importantly I would be interested to see if there is a particular usecase that could be used as the basis to start, as this is always the best way to add further functionality.
If you have currently some insight on this I woudl certainly eb interested to hear your thoughts as well. Thanks @alexander-g
Currently there is no support for multidimensional vectors, it would be important to explore how this could be added through either abstraction of flattened vectors, or through image2D / image3D data structures.