search

SciRuby / nmatrix

Dense and sparse linear algebra library for Ruby via SciRuby
Other
469 stars 133 forks source link

Proposed spec change: Integer should be treated as :object dtype rather than :int64 #605

Open translunar opened 6 years ago

translunar commented 6 years ago

This issue follows from https://github.com/SciRuby/nmatrix/pull/604#discussion_r158081343.

In the past, guess_dtype() would take a Fixnum and auto-assign a dtype of :int64. Now that Fixnum and Bignum are deprecated, an Integer may contain values that won't fit in :int64. Before we can release a new version, we need to adjust Integer to default to :object instead.

The other option (bad) would be to check each and every item in an array for :int64 overflow before allocating.

This raises the following question: do we even still want the :int64 dtype, or is it better to just use :object for anything that looks like an integer?

translunar commented 6 years ago

I'm just revisiting this issue to note that using :object for integers will (a) speed up compilation, and (b) possibly make it easier for us to support faster element-wise math for float32, float64, and the complex dtypes.