Document why Forward is needed

[Happypig375]

When is it not sufficient to assign references directly?

[WhiteBlackGoose]

Yeah, there's a poor sample in readme, but the answer is that your other operations might work with an instance without a creating new. For example, for some reason you have

class MyInt
{
    int val;
    public void Add(MyInt b)
        => val += b.val;
}

And you override the Add operation:

ConstantsAndFunctions<MyInt>.Add = (a, b) {
    a.Add(b);
    return a;
}

So if you assign to it directly, then your values will be affected even if you use static operations (those that return value). While if you create a new MyInt with the same int (it could be a class), it will be fine.

At the same time, you can't afford Copy, because your inner data might be huge and this operation is pricy. So you only need .Copy() in the whole tensor copy operation (GenTensor<T>.Copy).

[Happypig375]

You don't expect arithmetic operators to mutate references. From https://stackoverflow.com/questions/6587107/c-sharp-operator-overload-for#comment64556737_6587134,

No, it's pretty strange no matter which language you're looking at. To make the statement v3 = v1 + v2; result in v1 being changed as well as v3 is unusual.

Just like operator +, mutation is not expected in ConstantsAndFunctions<MyInt>.Add.

On top of that, C# does not allow you to overload += to mutate inner data directly. From https://stackoverflow.com/a/5202868/5429648,

The reason that you can't overload += separately is most likely that it creates strange semantics for reference types. If += modified the existing instance of the left side then the following code would behave strangely:

MyClass x=new MyClass();
MyClass y=x;
x+=SomeThing;

y has changed now too. On the other hand with += meaning x = x+y this code creates a new instance which it assigns to x. And the original instance remains unmodified in y.

Therefore, it does not make sense to implement ConstantsAndFunctions<MyInt>.Add with mutation. From https://stackoverflow.com/questions/6587107/c-sharp-operator-overload-for#comment7812845_6587107,

Vectors should behave like immutable mathematical objects. When you add 2 to 3, you don't mutate the object 3 into the object 5. You create an entirely new object, 5. The point of overloading addition operators is to make your own mathematical objects; making them mutable works against that aim. I would make your vector type an immutable value type.

Elements in Tensors are mathematical objects and are supposed to be immutable. We should not create extra operators just to cater to bad design. ConstantsAndFunctions<MyInt>.Add should contain no mutations, leaving ConstantsAndFunctions<MyInt>.Forward (and ConstantsAndFunctions<MyInt>.Copy) with no use. They should be removed.

[WhiteBlackGoose]

I might consider that, though I want to leave up to user as much as I can.

[WhiteBlackGoose]

Forward removed