Clarification for non-portable behavior for in-place Python operations

[oleksandr-pavlyk]

The wording for in-place operators may be more explicit to warn users that in-place operations, e.g., x1 += x2, where Type Promotion Rules require x1 + x2 to have data type different from data type of x1 are implementation defined.

Present wording hints at it:

For example, after in-place addition x1 += x2, the modified array x1 must always equal the result of the equivalent binary arithmetic operation x1 = x1 + x2.

but states that result of the in-place operation must "equal" the result of the out-of-place operation and the equality may hold true for arrays of different data types.

[asmeurer]

Did we agree that it should always be implementation defined if the types are different, or should x1 += x2 be OK if the type promoted type is the same as x1.dtype (i.e., x2 upcasts to x1)?

[kgryte]

should x1 += x2 be OK if the type promoted type is the same as x1.dtype (i.e., x2 upcasts to x1)?

@asmeurer I believe that is what @oleksandr-pavlyk was getting at in the OP. IMO, it should be okay to allow type promotion to x1.dtype and we can be more explicit in stating that the relationship x1 = x1 + x2 must hold provided the constraints of "limited" type promotion are satisfied.

[asmeurer]

So there are two cases:

Case 1: x1 + x2 promotes to x1.dtype

x1 = asarray([0], dtype=int64)
x2 = asarray([0], dtype=int32)
x1 += x2

Case 2: x1 + x2 promotes to x2.dtype

x1 = asarray([0], dtype=int32)
x2 = asarray([0], dtype=int64)
x1 += x2

I think @oleksandr-pavlyk was asking about case 2:

... where Type Promotion Rules require x1 + x2 to have data type different from data type of x1 ...

In my opinion, this is actually spelled out already https://data-apis.org/array-api/latest/API_specification/array_object.html#in-place-operators:

An in-place operation must not change the data type or shape of the in-place array as a result of Type Promotion Rules or Broadcasting.

In other words, case 2 is currently required to error (which is stronger than implementation defined).

Case 1 is perhaps more ambiguous whether it is required or not. My reading of the current text is that it is. If we want to make it implementation defined, we should explicitly state that. I'm not aware of any reasons why it would be a problem, though.

[ndgrigorian]

In other words, case 2 is currently required to error (which is stronger than implementation defined).

Does this make current NumPy behavior non-compliant, then?

Because in NumPy

In [26]: x_np = np.arange(10, dtype="i4")

In [27]: x_np += np.ones(10, dtype="i8")

In [28]: x_np
Out[28]: array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10], dtype=int32)

it simply casts the second array into the type of the first array per same_kind casting.

And per the spec

An in-place operation must have the same behavior (including special cases) as its respective binary (i.e., two operand, non-assignment) operation. For example, after in-place addition x1 += x2, the modified array x1 must always equal the result of the equivalent binary arithmetic operation x1 = x1 + x2.

This seems like a very strong condition implying that x1 += x2 of this case should be disallowed, and may be an even stronger condition than what you've quoted, because the data type of x1 (the in-place array) does not change.

[asmeurer]

I think the x1 = x1 + x2 sentence is not really intended to be saying anything about the data type of x1. It's only saying that whatever behavior is specified in __add__ (and thus add()) also applies to the += operator, i.e., all the special cases, nan behaviors, and so on. The wording should probably be improved since this isn't clear.

The sentence before that, which I quoted, "An in-place operation must not change the data type or shape of the in-place array..." does indeed imply that NumPy is currently noncompliant, because it uses must and not should. We could potentially loosen this to be implementation defined. I guess one question is whether the NumPy team agrees that this is nonideal behavior and should be deprecated. I certainly find the behavior surprising (it really is an in-place change of dtype: even views of x1 are updated to the promoted dtype).

Also we should check if other libraries allow this. This is the behavior in PyTorch:

>>> x1 = torch.asarray([0], dtype=torch.int32)
>>> x2 = torch.asarray([1], dtype=torch.int64)
>>> x1 += x2
>>> x1
tensor([1], dtype=torch.int32)

That actually could arguably be within what the spec says, because it didn't change the dtype of x1. So we should be clear whether this is actually OK, or whether it should be an error (or implementation defined).