search

feiwang3311 / Lantern

BSD 3-Clause "New" or "Revised" License
167 stars 15 forks source link

Support rank-0 tensors #16

Open dan-zheng opened 5 years ago

dan-zheng commented 5 years ago

Mathematically, scalars are 0-dimensional tensors.

However, 0-dimensional tensors are currently disallowed:

class Tensor(val data: Rep[Array[Float]], val dimensions: NSeq[Int]) extends Serializable {
  val strides = (dimensions :\ NSeq[Int]()) {
    case (dimX, seq@(h +: t)) => (dimX * h) +: seq
    case (dimX, _) => NSeq(dimX)
  }
  // Rank-0 tensors aren't allowed. :(
  assert(strides.length >= 1)
  assert(strides(0) != 0, "Empty Tensor!!!")
}

I believe rank-0 tensors should be allowed everywhere. The assertions above should be removed. Functions that semantically return scalars (e.g. vector-vector dot product) should be modified to return rank-0 tensors rather than vectors of size 1.

Rank-0 tensor support might be blocked by broadcasting support (without broadcasting, binary ops like scalar + matrix won't work).

feiwang3311 commented 5 years ago

Make sense. I have added broadcasting to Lantern. Maybe I will look into rank-0 tensor soon as well. :)