64-bit sizes in blas and lapack

[JeffBezanson]

These library interfaces are currently limited to Int32. We should build them with 64-bit size support, or give errors for arrays that are too big.

[ViralBShah]

We should pass 64-bit build options to openblas if on a 64-bit machine. The interface in blas.jl and lapack.jl can probably be quickly updated to use Int instead of Int32.

[staticfloat]

Homebrew by default compiles 64-bit binaries, so as it stands right now, it looks like Julia already interoperates with OpenBLAS properly in this case.

This, assuming that the following constitutes a 64-bit build:

$ file libopenblas.dylib
libopenblas.dylib: Mach-O 64-bit dynamically linked shared library x86_64

[JeffBezanson]

This is a different issue. Blas is still using 32-bit ints for matrix dimension arguments, even in the 64-bit build, which works fine but limits size. On Nov 11, 2012 3:14 PM, "Elliot Saba" notifications@github.com wrote:

Homebrew by default compiles 64-bit binaries, so as it stands right now, it looks like Julia already interoperates with OpenBLAS properly in this case.

This, assuming that the following constitutes a 64-bit build:

$ file libopenblas.dylib libopenblas.dylib: Mach-O 64-bit dynamically linked shared library x86_64

— Reply to this email directly or view it on GitHubhttps://github.com/JuliaLang/julia/issues/1527#issuecomment-10271556.

[staticfloat]

Ah, understood. You're talking about setting INTERFACE64=1 in the makevars for OpenBLAS.

EDIT: INTERFACE64, not BINARY64

[ViralBShah]

@staticfloat That's right. We need INTERFACE64=1 on amd64 architectures, and then corresponding changes in blas.jl and lapack.jl.

[ViralBShah]

The fact that we have wrapped the BLAS and LAPACK routines in a julia interface, will make it much easier to make this change.

[ViralBShah]

@xianyi If we build openblas with INTERFACE64=1 and are building the patched lapack, will lapack also get built with a 64-bit interface?

[xianyi]

Hi @ViralBShah ,

Yes, OpenBLAS and LAPACK support ILP64 model with INTERFACE64=1.

However, LAPACKE, a C interface for LAPACK, has a building bug which we don't enable ILP64 when INTERFACE64=1. I will fix this bug immediately on develop branch.

Xianyi

[ViralBShah]

We use the fortran interface. I guess we should be ok.

[ViralBShah]

https://github.com/xianyi/OpenBLAS/issues/158

[JeffBezanson]

DSFMT has the same problem, using int for size arguments.

[ViralBShah]

DSFMT problem filed as separate issue in #1556

[ViralBShah]

I have been able to do dot(a,a) where a = ones(2^32+1) on the vs/blas64 branch.

[StefanKarpinski]

Did you get the correct answer too?

[ViralBShah]

Yup.

[StefanKarpinski]

Great work, Viral. This was a slog – thanks for doing it. Obviously, this is super important for doing work with really large amounts of data.

[ViralBShah]

I also hope this will set a precedent for libraries that are integrated with julia in the future.