search

bmillare / dj.math

idiomatic computer algebra system in clojure
5 stars 1 forks source link

Use common matrix interface? #1

Open mikera opened 11 years ago

mikera commented 11 years ago

This is a very interesting project! I'm very interested in all the symbolic manipulation stuff.

I notice however that you are creating a new vector / matrix implementation.

Some of us have noticed that there are already many matrix implementation (JBLAS/Clatrix, Parallel Colt, javax.vecmath, EJML, Vectorz, and I'm sure many others....). We're trying to build a common matrix API that will support multiple back-end implementations. See:

Can I encourage you to consider building dj.math on top of the common API? This will give benefits all round:

bmillare commented 11 years ago

Hi Mike,

The main reason I use my own matrix implementation is because all the operations, from the ground up, need to be defined in terms of generic operators. In other words, I cannot use clojure.core/+ -, etc. The elements of my matrices may potentially be symbols or expressions. I noticed you choose to use protocols for the operations, which is good for dispatching performance on the first arg, but for my work, many operations may depend on several factors, arg types, arg metadata like size, etc, I feel multimethods or some other dispatching method would be more convenient and manageable. The main way I see our libraries interacting at the moment is I would define a clojure code emitter that uses your library to do the computations. I'll still have to look into this more though.

Best, Brent

On Sun, Jan 6, 2013 at 9:25 PM, Mike Anderson notifications@github.comwrote:

This is a very interesting project! I'm very interested in all the symbolic manipulation stuff.

I notice however that you are creating a new vector / matrix implementation.

Some of us have noticed that there are already many matrix implementation (JBLAS/Clatrix, Parallel Colt, javax.vecmath, EJML, Vectorz, and I'm sure many others....). We're trying to build a common matrix API that will support multiple back-end implementations. See:

Can I encourage you to consider building dj.math on top of the common API? This will give benefits all round:

  • You will get the ability to plug and play with powerful back-end matrix libraries
  • We'll collectively avoid reinventing some wheels (we don't really need yet more matrix libraries!)
  • You will hopefully get easier interop with other Clojure libraries that use the common matrix API, e.g. there is interest in the Incanter community to move towards this

  • We'll get valuable feedback from field-testing using the API so we can refine it into something great

    — Reply to this email directly or view it on GitHubhttps://github.com/bmillare/dj.math/issues/1.

Brent Millare PhD Student Johns Hopkins University Department of Biomedical Engineering 3400 N. Charles Street Hackerman Hall 218 Baltimore, MD 21218 bmillar1@jhmi.edu

mikera commented 11 years ago

Hi Brent,

That certainly makes sense - I can see some very customised logic needed for manipulating the symbolic components of matrices.

Would you be able to use a generic n-dimensional array structure for your purposes? It's not on my roadmap at the moment, but there's always an option to implement one - rather like Python's ndarray. It wouldn't limit what you can put in the cells or what operations you could perform. Any use to you?

Apart from that, having an emitter after the symbolic logic is performed definitely seems to be the right point to interface in general. If you get a chance to have a go at this, do let me know - I'm very keen to get feedback on the matrix API with a few different use cases to make sure it is as useful for as many projects as possible.

All the best,

Mike.

bmillare commented 11 years ago

Hi Mike,

I can't currently think of a case for the n-dimensional array (for the n-part, I have use for the 2-dimensional generic array) but I can imagine people would. Personally I just needed something that worked and went for the simplest implementation I could think of and went with nested vectors. I inherited some of the benefits of immutability so "associng" and obtaining minors was very clojure idiomatic. For the matrix sizes I was using that was fine. If you provide some of the basic manipulation operators on other implementations that make some of the operations more efficient, I can build my "higher level" operations out of these and I can see that being very good. (Although I'd probably do that later down the road as I probably don't have much time to do that type of optimizations). I agree with your goal for building a consistent API for the operations. There will always be tradeoffs for the performance of different operations depending on the implementation and you want to give people options while providing a consistent interface.

As a take home message, my only real feedback is that it might be worth providing some protocols on some low level operations that are useful for building full matrix manipulation algorithms. This will allow defining some of the matrix operations as just functions and will help minimize how many protocols future matrix implementers have to extend to, especially if there implementation is really for only niche performance cases. People should be able to choose to just implement a basic subset, or for absolute performance implement the high level operation directly. I'll be able to provide more specific feedback in the future.

Best, Brent

On Mon, Jan 7, 2013 at 7:29 PM, Mike Anderson notifications@github.comwrote:

Hi Brent,

That certainly makes sense - I can see some very customised logic needed for manipulating the symbolic components of matrices.

Would you be able to use a generic n-dimensional array structure for your purposes? It's not on my roadmap at the moment, but there's always an option to implement one - rather like Python's ndarray. It wouldn't limit what you can put in the cells or what operations you could perform. Any use to you?

Apart from that, having an emitter after the symbolic logic is performed definitely seems to be the right point to interface in general. If you get a chance to have a go at this, do let me know - I'm very keen to get feedback on the matrix API with a few different use cases to make sure it is as useful for as many projects as possible.

All the best,

Mike.

— Reply to this email directly or view it on GitHubhttps://github.com/bmillare/dj.math/issues/1#issuecomment-11979142.

Brent Millare PhD Student Johns Hopkins University Department of Biomedical Engineering 3400 N. Charles Street Hackerman Hall 218 Baltimore, MD 21218 bmillar1@jhmi.edu

mikera commented 11 years ago

Thanks Brent - very useful feedback!

Will see what we can do building some of this into the prototype....