dge results in an exception when called with matrices larger than Integer.MAX_INT

[Alcamech]

dge results in an exception when called with matrices larger than Integer.MAX_INT.

Error:

java.lang.IllegalArgumentException: Value out of range for int: 2973885280

clojure.lang.Compiler$CompilerException: java.lang.IllegalArgumentException: 
Value out of range for int: 2973885280

StackTrace:

uncomplicate.neanderthal.native/dge                    native.clj:   99
uncomplicate.neanderthal.core/ge                      core.clj:  268
uncomplicate.neanderthal.core/ge                      core.clj:  262
uncomplicate.neanderthal.internal.host.mkl.MKLRealFactory/create-ge                       mkl.clj: 4468
uncomplicate.neanderthal.internal.host.buffer-block/real-ge-matrix              buffer_block.clj: 1000
uncomplicate.neanderthal.internal.host.buffer-block/real-ge-matrix              buffer_block.clj:  997
uncomplicate.neanderthal.internal.host.buffer-block.DoubleBufferAccessor/createDataSource              buffer_block.clj:  100
uncomplicate.commons.utils/direct-buffer                     utils.clj:  140                                                                                      ...                     

java.lang.IllegalArgumentException: Value out of range for int: 2973885280
clojure.lang.Compiler$CompilerException: java.lang.IllegalArgumentException: Value out of range for int: 2973885280

[blueberry]

This is an issue in Java rather than neanderthal. Java buffers are indexed by ints, so that's it. Larger matrices can be supported in theory by programming indexing methods in C, but I am not sure how often that is really required, since, if you're using matrices larger than 2GB, it's probably time to look for some shortcuts and more specialized algorithms, and sparse matrices anyway.

[larstvei]

Hi!

I'm having the same issue, and it looks like the limit is at 2^28-1.

> (dge (dec (Math/pow 2 28)) 1)
#RealGEMatrix[double, mxn:268435455x1, layout:column, offset:0]
   ▥       ↓       ┓    
   →       0.00         
   →       0.00         
   →       ⁙            
   →       0.00         
   →       0.00         
   ┗               ┛    

> (dge (Math/pow 2 28) 1)
Execution error (IllegalArgumentException) at uncomplicate.commons.utils/direct-buffer (utils.clj:130).
Value out of range for int: 2147483648

It would be really nice if the limitation would be the amount of memory on the computer, rather than fixed, but I understand that it might not be feasible.