v923z / micropython-ulab

a numpy-like fast vector module for micropython, circuitpython, and their derivatives
https://micropython-ulab.readthedocs.io/en/latest
MIT License
403 stars 111 forks source link

[BUG] division of an ndarray with int16 dtype produces incorrect results #610

Closed ned-pcs closed 1 year ago

ned-pcs commented 1 year ago

Describe the bug When an ndarray with np.int16 dtype is divided by a scalar, the results are often incorrect.

ulab version 6.0.7-2D f2dd2230c4fdf1aa5c7a160782efdde18e8204bb extmod/ulab (5.1.1-20-gf2dd223)

To Reproduce In Micropython or Circuitpython, do this:

>>> from ulab import numpy as np
>>> a = np.array([-102, -1750, 202, -1638], dtype=np.int16)
>>> print(a)
array([-102, -1750, 202, -1638], dtype=int16)
>>> print(a/32768)
array([1.99689, 1.94659, 0.00616455, 1.95001], dtype=float32)
>>> a/32768.0
array([1.99689, 1.94659, 0.00616455, 1.95001], dtype=float32)
>>> print(a[0]/32768)
-0.00311279
>>> [v/32768 for v in a]
[-0.00311279, -0.0534058, 0.00616455, -0.0499878]
>>> a/100
array([-1.0, -17.0, 2.0, -16.0], dtype=float32)
>>> a/100.0
array([654.34, 637.86, 2.02, 638.98], dtype=float32)

Expected behavior Here is what I got with a desktop version of numpy:

>>> a = np.array([-102, -1750, 202, -1638], dtype=np.int16)
>>> print(a)
[ -102 -1750   202 -1638]
>>> print(a/32768)
[-0.00311279 -0.05340576  0.00616455 -0.04998779]
>>> a/32868.0
array([-0.00310332, -0.05324328,  0.0061458 , -0.04983571])
>>> a/32768.0
array([-0.00311279, -0.05340576,  0.00616455, -0.04998779])
>>> a/100
array([ -1.02, -17.5 ,   2.02, -16.38])
>>> a/100.0
array([ -1.02, -17.5 ,   2.02, -16.38])
v923z commented 1 year ago

Thanks for bringing up the issue! https://github.com/v923z/micropython-ulab/pull/611 implements the fix.