themetaschemer / malt

Machine Learning Toolkit accompanying The Little Learner by Daniel P. Friedman and Anurag Mendhekar
MIT License
212 stars 25 forks source link

Rank 0 elements of tensors become scalar? but not real? #37

Closed davidgrenier closed 1 year ago

davidgrenier commented 1 year ago

Hello,

I'm not sure if this is by design but what I would expect to be a real? (although a scalar?) at rank 0 isn't. This makes it quite horrible to work with any kind of libraries (in my particular case racket's plot lib).

For instance:

(not (equal? (+ 1 2.7) 3.7))
(equal? (tref (+ 1 2.7) 1) 3.7)

Both of these evaluate to #t. I'm not too far along in the book or external documentation but it feels like I have to dig down into an undocumented data structure in various contrived manners if I want to use points3d or surface3d.

Much appreciated.

themetaschemer commented 1 year ago

Thanks David.

This is somewhat by design. The reason the (+ 1 2.7) and 3.7 are considered NOT equal is because the result of the + is a dual (not sure if you're read Appendix A as yet), so that it can be differentiated when necessary. For the constant 3.7, all derivatives will be 0, but that is generally not the case for (+ a b) where the derivatives of the result w.r.t. a and b will be 1, so forcing them to be equal? did not seem like a good choice.

In any case, I do have a solution for you to be able to work with external libraries. The operator ρ will strip all the additional data from a dual and return. you the real number corresponding to it. If you're using the flat-tensor or nested-tensor representation, then (ρ t) will give you a tensor you can directly use with external libraries.

If you're using learner, then you'll have to descend recursively into the tensor using (map* ρ t) . To use map* you'll need to (require malt/learner).

davidgrenier commented 1 year ago

Makes perfect sense as far as I'm concerned. I'm too early still in the book but assumed there was a sensible explanation.