Agree on the tensor template class parameters

[bassoy]

Discuss your intention with @stefanseefeld The initial idea is to provide template parameters that determine the

• data type and
• layout type (first-order and last-order storage formats)
• storage array type (now unbounded array).

and three private member variables:

• the extents class
• the strides class
• the data storage class

where the

• rank/order and
• dimension extents

of a tensor are runtime variable.

[stefanseefeld]

Making data type, layout, and storage type template parameters seems fine. But doesn't this conflict with rank / order being a runtime parameter ? I'm not quite sure what you mean in the member variables listing. Do you mean 'object' rather than 'class' there ?

[bassoy]

Making data type, layout, and storage type template parameters seems fine. But doesn't this conflict with rank / order being a runtime parameter ?

No. Fully orthogonal, see examples.

I'm not quite sure what you mean in the member variables listing. Do you mean 'object' rather than 'class' there ?

Sorry for the confusion. I mean data members that reside within the tensor template class.

[stefanseefeld]

I see the layout parameter only allows for column-major and row-major values. What if I want something else (in particular with higher-dimensional tensors, this seems to be a fair requirement, no ?) I'd really expect the layout parameter to be coupled to the rank of the tensor, which is why I don't see how the rank can be a runtime parameter while the layout parameter is not.

[stefanseefeld]

To illustrate my point a bit, please have a look at the OpenVSIP Layout specification in https://github.com/openvsip/openvsip/blob/master/src/vsip/layout.hpp (also formally standardized in [block.layout] in https://www.omg.org/spec/VSIPL++/1.3: There is an order_type defining the dimension-ordering, as well as other parameters related to layout (packing, complex format storage (split vs. interleaved), etc.). I'm not suggesting we should necessarily adapt the VSIPL++ model here, but I'd at least like to discuss it to establish a rationale for whatever choice(s) we make.

[bassoy]

Yes I understand. I will have a look. However, I have a mixed feeling about storage order when it comes to operations in the realm of linear and multilinear algebra. I have not found a paper showing that blocked storage formats such as z-morton layouts are beneficial in terms of runtime performance. The fastest implementations (OpenBLAS, MKL) consider only the standard row- or column-major formats. This might be different for image processing applications. Please note that in my case you could still be able to set storage format to a certain layout type, e.g. blocked<4,4> and specialize the template class.

[stefanseefeld]

I don't expect this to be of value for image processing (at least not if by "image" we are referring to a rank-2 tensor). Where it matters are really higher-order tensors where we want to perform operations on individual planes (a good example in VSIPL++ are Fourier transforms on arbitrary axes, where it's highly beneficial if the tensor is unit-stride in the given axis). Likewise, for any element-wise operation that is applied only to one axis it's useful for it to be unit-stride, notably if you want to vectorize the operation.

[bassoy]

Very interesting. I think we can integrate it in my current implementation without big changes. So I would like to close this issue for now. We can reopen it. Finally, I would like to keep the first- and last-order storage formats for GSOC18.