netlib-java natives and offset arguments

[fommil]

Hi! I'm the author of netlib-java and maintainer of matrix-toolkits-java.

I'm in the process of a major rewrite (but API preserving) of netlib-java with the goal of easily deployed native libraries (this has required manual compilation to now. At the very least, I want a one line maven command to get machine optimised natives).

Some time ago, your predecessor asked about offset arguments to the netlib code

https://github.com/fommil/netlib-java/issues/6

I'd like to open up the discussion again and find out where else you could benefit from improvements in the netlib stack.

[dlwh]

I think that's all we need. Making configuration/loading as easy as possible is good. jblas does a great job of that.

[fommil]

@dlwh that's one of the goals. Are you using netlib-java anymore or are you moved onto jblas?

[fommil]

sorry, do you mean you need offsets?

[dlwh]

We need offsets: we use them extensively for slicing and what not.

I use jblas now, but I am definitely not wedded to it. I used it because it was super easy to configure (i.e. no configuration) and it supports offsets. We fall back to netlib-java if jblas can't load its native libraries.

On Fri, Jul 5, 2013 at 1:33 PM, Sam Halliday notifications@github.comwrote:

sorry, do you mean you need offsets?

— Reply to this email directly or view it on GitHubhttps://github.com/scalanlp/breeze/issues/79#issuecomment-20537754 .

[fommil]

ok. I'll look into supporting the offsets then.

The only config that netlib-java should require is a property set to point to the implementation that is wanted. Is that too much?

[fommil]

(if I remember correctly, the offsets were problematic because the C functions didn't support them directly. Maybe there is some funky pointer arithmetic I could do in the JNI that means this is OK).

BTW, which platforms are you targetting primarily? I have access to 64 bit OSX, Linux and Windows 8. For anything else, I'll need help to compile binaries so I can package them.

[dlwh]

On Fri, Jul 5, 2013 at 2:01 PM, Sam Halliday notifications@github.comwrote:

ok. I'll look into supporting the offsets then.

The only config that netlib-java should require is a property set to point to the implementation that is wanted. Is that too much?

You also have to download/compile/install those libraries, yes? I know that isn't much, but it has to be done on every machine you use. My workflow is usually to debug locally, then package all dependencies into one fatjar/assembly, and then scp it to a cluster that I don't have much control over. jblas's model is just way more convenient for that:

In jblas, jars are kept in the native libs are kept in the jar, then extracted and loaded on demand. So, it's not so much that it's "too much" versus "not much" configuration. It's "there is configuration" or "there is no configuration". (The jblas libs aren't necessarily the fastest native libraries sense they're presumably tuned for whatever architecture they were compiled on, but jblas also supports the same kind of configuration you do.)

offsets: presumably offsets are relatively easy? I doubt the pointer arithmetic is anything more complicated than ptrToArrayElements + offset, i guess with array bounds checking on the native side ? It looks like the hard part is retrofitting the generator in netlib-java to generate two arguments for pointer arguments.

architectures: that's all i have access to.

Thanks!

-- David

— Reply to this email directly or view it on GitHubhttps://github.com/scalanlp/breeze/issues/79#issuecomment-20538802 .

[fommil]

@dlwh the idea is that the user will not need to do any compilation anymore. I'm hoping to package pre-build binaries using the maven-native plugin, so that the only thing the end user needs to do (or a middle-tier developer could do it for them) is to identify the correct OS and set a property value. Sound good?

[fommil]

re: offsets, yes it's trivial for vectors. But I'm not sure it's so trivial for the matrix operations. There might be some funky differences in C/Fortran 2D array storage that I vaguely recall. I'll need to research it further.

[fommil]

Re: netlib-java generator. I've completely rewritten this now as a maven-plugin. It should now be super easy to make changes to the generated code. The hard bit is automating the JNI building. I have no experience of building C code from Maven.

I still want power users to be able to compile their own binaries (e.g. they want to link against a commercial LAPACK optimised for their machine, or use a specific compiler).

Incidentally, I really hope to do some OpenCL in matrix-toolkit-java, e.g. for matrix multiplication. Have you done anything? (I guess you're not using MTJ?)

[dlwh]

C 2-d arrays are row major, and Fortran's are column major. I think that's it. netlib-java has required they be column major for as long as I've known. Since the memory is laid out linearly, C will never be the wiser. (I'm pretty sure jblas does it this way.)

On Fri, Jul 5, 2013 at 2:37 PM, Sam Halliday notifications@github.comwrote:

re: offsets, yes it's trivial for vectors. But I'm not sure it's so trivial for the matrix operations. There might be some funky differences in C/Fortran 2D array storage that I vaguely recall. I'll need to research it further.

— Reply to this email directly or view it on GitHubhttps://github.com/scalanlp/breeze/issues/79#issuecomment-20540080 .

[dlwh]

re: configuration, that sounds great! If I get excited I might try to port over jblas' auto-extract logic if you're up for it. I might need to do it for hdf5 support, anyway.

On Fri, Jul 5, 2013 at 2:36 PM, Sam Halliday notifications@github.comwrote:

@dlwh https://github.com/dlwh the idea is that the user will not need to do any compilation anymore. I'm hoping to package pre-build binaries using the maven-native plugin, so that the only thing the end user needs to do (or a middle-tier developer could do it for them) is to identify the correct OS and set a property value. Sound good?

— Reply to this email directly or view it on GitHubhttps://github.com/scalanlp/breeze/issues/79#issuecomment-20540025 .

[fommil]

I have seen similar auto-extract code in projects such as https://github.com/cibuddy/cibuddy

I'm using the USB library part of cibuddy in https://github.com/fommil/emokit-java/ and https://github.com/fommil/neurofiction which is what gave me the idea to revamp https://github.com/fommil/netlib-java (that and the embarrassing realisation that loads of companies are using netlib-java but it is the Worst Code I Have Ever Written!)

[dlwh]

I definitely agree that you should be able to compile your own libraries.

I've been playing around with opencl/javacl a bit recently. Haven't done much (a matrix transpose operation), but it's fun to play around with.

On Fri, Jul 5, 2013 at 2:40 PM, Sam Halliday notifications@github.comwrote:

Re: netlib-java generator. I've completely rewritten this now as a maven-plugin. It should now be super easy to make changes to the generated code. The hard bit is automating the JNI building. I have no experience of building C code from Maven.

I still want power users to be able to compile their own binaries (e.g. they want to link against a commercial LAPACK optimised for their machine, or use a specific compiler).

Incidentally, I really hope to do some OpenCL in matrix-toolkit-java, e.g. for matrix multiplication. Have you done anything? (I guess you're not using MTJ?)

— Reply to this email directly or view it on GitHubhttps://github.com/scalanlp/breeze/issues/79#issuecomment-20540184 .

[dlwh]

Yeah, lots of libraries seem to do this auto-extract trick. Someone should probably generify it as much as is possible.

Your projects look really cool!

And... that's always how it goes... my least favorite publication has more citations than all of my other ones combined. Oh well.

On Fri, Jul 5, 2013 at 2:46 PM, Sam Halliday notifications@github.comwrote:

I have seen similar auto-extract code in projects such as https://github.com/cibuddy/cibuddy

I'm using the USB library part of cibuddy in https://github.com/fommil/emokit-java/ and https://github.com/fommil/neurofiction which is what gave me the idea to revamp https://github.com/fommil/netlib-java (that and the embarrassing realisation that loads of companies are using netlib-java but it is the Worst Code I Have Ever Written!)

— Reply to this email directly or view it on GitHubhttps://github.com/scalanlp/breeze/issues/79#issuecomment-20540375 .

[twitwi]

Hi,

I'm following different projects and if I understand well, bridj might be exactly the multi-architecture "generic auto-extract" library that you are looking for.

Also it avoid requiring to write any JNI code. The same author, @ochafik, also have written JavaCL which seems useful to this discussion.

All can be found there: https://github.com/ochafik/nativelibs4java/

Sorry for the noise if I'm misunderstood the point. Rémi

[fommil]

thanks @twitwi ... those kind of libraries use a dynamic binding (JNA is similar, which is more widely adopted). Unfortunately, such an approach is not appropriate here as we're dealing with high performance linear algebra routines that require direct access to native code and JVM data structures.

What myself and @dlwh are discussing is just a very lightweight "unbundler" that does the loading of the binary that will be distributed as part of a jar file. It's pretty trivial, but it would be nice if there were a common project that did it instead of copying and editing e.g. https://github.com/cibuddy/cibuddy/blob/master/drivers/hid/src/main/java/com/codeminders/hidapi/ClassPathLibraryLoader.java

For developers, the ideal solution is just the http://mojo.codehaus.org/maven-native/native-maven-plugin/ but that doesn't flow to end users and there isn't an SBT equivalent at the moment (I've tried to poke some people in the right direction as I have a similar need in another project)

OpenCL code is becoming very popular from Java: I had planned to use LWJGL's OpenCL to do dense matrix multiplication in MTJ. I'm not aware of a full netlib implementation in OpenCL (not all the netlib routines are appropriate for a graphics card anyway as they require iterated pivoting and so on).

[fommil]

@dlwh this is the sort of thing I was thinking of (it might make sense to pull this out as a separate library): JniLoader.class

[dlwh]

nice! Yeah, pulling this out as its own library makes a lot of sense. Are you volunteering? :-)

I can also get to it, though maybe not for a few weeks.

On Fri, Jul 26, 2013 at 2:44 PM, Sam Halliday notifications@github.comwrote:

@dlwh https://github.com/dlwh this is the sort of thing I was thinking of (it might make sense to pull this out as a separate library): JniLoader.classhttps://github.com/fommil/netlib-java/blob/master/native/src/main/java/com/github/fommil/jni/JniLoader.java

— Reply to this email directly or view it on GitHubhttps://github.com/scalanlp/breeze/issues/79#issuecomment-21650041 .

[fommil]

if I decide to do that, I'll do it myself. It would be good if it was a zero dependency lib and I should look into integrating it with maven native on the dev side as well as writing up some lessons-learnt docs on how to actually build binaries for all major platforms (when I figure it out!).

[dlwh]

Sounds great. Let me know if I can be of any help!

On Sat, Jul 27, 2013 at 4:25 AM, Sam Halliday notifications@github.comwrote:

if I decide to do that, I'll do it myself. It would be good if it was a zero dependency lib and I should look into integrating it with maven native on the dev side as well as writing up some lessons-learnt docs on how to actually build binaries for all major platforms (when I figure it out!).

— Reply to this email directly or view it on GitHubhttps://github.com/scalanlp/breeze/issues/79#issuecomment-21664034 .

[fommil]

@dlwh of course you can help :-D TESTS! I have severely lacking coverage. If you have any good input/output examples that I could convert into unit tests, that would be perfect. They could even double as perf-tests.

[fommil]

(also, if you have access to any non-OSS implementations of BLAS/LAPACK that would be good. I'm going to ship the first new release with reference implementation of native netlib, ATLAS and the OS X vecLib support).

[dlwh]

Sounds good! I have access to Intel's mkl (yay academia) so I can probably give that a try.

You want unit tests just testing the API? Probably the easiest thing is for me to hook the updated library into Breeze and run its test suite. The tests starting from math/src/test/breeze/linalg/LinearAlgebra.scala test most of the Lapack api that we use, besides the more normal daxpt/dgemm etc that's tested in the DenseVector/DenseMatrix tests. They have input/output examples.

-- David

On Sun, Jul 28, 2013 at 4:21 AM, Sam Halliday notifications@github.comwrote:

(also, if you have access to any non-OSS implementations of BLAS/LAPACK that would be good. I'm going to ship the first new release with reference implementation of native netlib, ATLAS and the OS X vecLib support).

— Reply to this email directly or view it on GitHubhttps://github.com/scalanlp/breeze/issues/79#issuecomment-21682081 .

[fommil]

direct unit tests would be good, although integration tests at the level of breeze are also good.

BLAS/LAPACK come with a lot of tests in the netlib reference imlementation. I'm keen to see if I can get F2J to spit out unit tests that use the abstract API. I'll maybe speak to Keith Seymour about that when I've finished this sprint.

[fommil]

moving to https://github.com/fommil/netlib-java/issues/15 and https://github.com/fommil/netlib-java/issues/6 to close

[fommil]

FYI, the test stuff is being tracked under https://github.com/fommil/netlib-java/issues/21