_n_word_max is set to 32 (and not to 64 as one would expect) on a windows 10 64 bit machine

francof2a / fxpmath

A python library for fractional fixed-point (base 2) arithmetic and binary manipulation with Numpy compatibility.

MIT License

183 stars 21 forks source link

_n_word_max is set to 32 (and not to 64 as one would expect) on a windows 10 64 bit machine #64

Closed alb-pol closed 2 years ago

alb-pol commented 2 years ago

The reason is this check: _test_precision = lambda prec_bits: abs(int(2**np.array(prec_bits-1))) == abs(2**(prec_bits-1)) specifically np.array(prec_bits-1) . Default integer type np.int_ is C long, but C long is int32 in win64: Stackoverflow

francof2a commented 2 years ago

This is not an issue of fxpmath but numpy in windows. This is one of the reasons fxpmath checks the n_word_max value to work with int type in numpy instead with object python type.