search

thouis / numpy-trac-migration

numpy Trac to github issues migration
2 stars 3 forks source link

abs does not work with -maxint (migrated from Trac #1172) #2725

Open thouis opened 12 years ago

thouis commented 12 years ago

Original ticket http://projects.scipy.org/numpy/ticket/1172 Reported 2009-07-15 by trac user etu, assigned to unknown.

a = array([-128], int8) a array([-128], dtype=int8) abs(a) array([-128], dtype=int8)

thouis commented 12 years ago

Comment in Trac by atmention:charris, 2009-07-15

Yeah, that is because there is no +128 available in int8. There was a discussion of this problem buried in a [http://tinyurl.com/mp23lw long thread] on the mailing list but no solution was arrived at; there really isn't one except to return uint8 or a higher precision. Matlab returns 127 in this case O_o.

thouis commented 12 years ago

Comment in Trac by atmention:pv, 2010-09-05

thouis commented 12 years ago

Comment in Trac by atmention:rgommers, 2011-03-01

I agree with Charles comment in that thread: This is hardware integer math. It has it's own definition, and apparently abs(-128) = -128. Returning 127 makes no sense, returning another dtype would be inconsistent. The only alternative to -128 imho is to just say the operation is undefined and throw an error.

thouis commented 12 years ago

Comment in Trac by atmention:mwiebe, 2011-03-24

Treating it as an overflow and following the ufunc seterr policy might make sense.