Question: Should tensor elements be required to be distinct?

[devinamatthews]

Using the general stride layout, it is often assumed that the strides are set such that no two elements share the same location. In some cases, strides are also assumed to be positive, so that stride[i+1] >= stride[i]*shape[i] e.g.

But, allowing more flexible representations may make sense in some circumstances, e.g. one can broadcast a lower-dimensional tensor into a higher-dimensional one by adding modes with 0 strides. Of course, it would be very difficult to support writing to such tensors.

[springer13]

I think a stride of 0 is fine, any other overlap of elements should probably be avoided. For instance, broadcasting is a common operation within numpy.

[scemama]

Hi,

I think that by default, the memory domain in which the result is written should be distinct from every domain where the memory is read.

When we do D := bC + a A.B, A B and C should be read-only and D should be distinct from all of them, because it could lead to an undefined behavior of the compiler to let writing to D modify on the fly A,B or C.

If we want to do C := bC + a A.B, then we should use a different function from the previous one, where C is distinct from A and B.

I think it is important to separate these two behaviors in two different functions, because joining the two behaviors in a common function would lead to a logical contradiction where the same memory domain would be const through C and mutable through D, and the compiler might wither do wrong code assuming no aliasing between C and D, or sub-optimal assuming aliasing between C and D.

Also, if one wants to implement the library in Rust, having a common interface for the 2 behaviors would be forbidden by the language, and this function would need to be declared as unsafe (for good reasons!).