Closed shane-huang closed 7 years ago
so....how about the progress of "Sparse Input/Tensor"? Is there any schedule for this issue?
@sperlingxx We have implemented the Wide&Deep model with SparseTensor. Now we are doing more tests on the SparseTensor. Do you have any requirement for the SparseTensor?
@qiuxin2012 是的,我们在尝试用bigDL做推荐/搜索相关的工作,基于Spark On ODPS
@sperlingxx 我们计划在下个月的v0.3.0的发布中加入SparseTensor的支持。 如果方便的话,最好可以告诉我们你需要SparseTensor的哪些功能,或者希望能支持一个什么样的模型。
@qiuxin2012 哇~那太棒了~ 我们会试一些最新的paper里提出的模型,差不多就是deep&wide的变种吧。 所以希望有类似 torch.nn.SparseLinear/IndexLinear 的Layer~
Have you tried with an optimizer? Looks like it just don't work with simple linear models..
java.lang.UnsupportedOperationException: SparseTensor: Unimplemented method
at com.intel.analytics.bigdl.tensor.SparseTensor.stride(SparseTensor.scala:124)
at com.intel.analytics.bigdl.tensor.DenseTensorMath$.addmm(DenseTensorMath.scala:565)
at com.intel.analytics.bigdl.tensor.DenseTensor.addmm(DenseTensor.scala:1230)
at com.intel.analytics.bigdl.nn.Linear.updateOutput(Linear.scala:108)
at com.intel.analytics.bigdl.nn.Linear.updateOutput(Linear.scala:45)
at com.intel.analytics.bigdl.nn.abstractnn.AbstractModule.forward(AbstractModule.scala:268)
at com.intel.analytics.bigdl.nn.Sequential.updateOutput(Sequential.scala:39)
at com.intel.analytics.bigdl.nn.abstractnn.AbstractModule.forward(AbstractModule.scala:268)
at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$9$$anonfun$10$$anonfun$apply$2.apply$mcI$sp(DistriOptimizer.scala:234)
at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$9$$anonfun$10$$anonfun$apply$2.apply(DistriOptimizer.scala:227)
at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$9$$anonfun$10$$anonfun$apply$2.apply(DistriOptimizer.scala:227)
at com.intel.analytics.bigdl.utils.ThreadPool$$anonfun$1$$anon$4.call(ThreadPool.scala:112)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
As well as the Sample object will throw on the isContinuous requirements with MethodNotImplemented error
def apply[T: ClassTag](
featureTensor: Tensor[T],
labelTensor: Tensor[T])(implicit ev: TensorNumeric[T]) : Sample[T] = {
require(featureTensor.isContiguous(), "featureTensor is not contiguous")
require(labelTensor.isContiguous(), "labelTensor is not contiguous")
if (featureTensor.getTensorType == DenseType) {
ArraySample(featureTensor, labelTensor)
} else {
TensorSample(featureTensor, labelTensor)
}
}
@michellemay I'm not sure how you create the SparseTensor. We have a Wide&Deep example written in Python. And SparseTensor should works with Sparse Layers, see SparseTensor and Sparse Layers for help.
It's better to have a introduction document under the Programming Guide section in our document website
My remark is about scala/spark. I haven't tried in python as it's not my use case. What I say is that it can't work (release 0.4.0) in scala because there are requirement checks and unsupported method calls when I try to bypass theses checks.
@michellemay Em, I see. Could you put your sparse tensor into an Array to work around this? I will fix this bug in 0.5.0-snapshot.
def apply[T: ClassTag](
featureTensors: Array[Tensor[T]],
labelTensor: Tensor[T])(implicit ev: TensorNumeric[T]) : Sample[T] = {
if (featureTensors.exists(_.getTensorType == SparseType)) {
TensorSample(featureTensors, labelTensor)
} else {
ArraySample(featureTensors, labelTensor)
}
}
I think this one will works fine.
Neural nets are not only used for dense inputs such as image and audio. Sparse inputs should also be supported.
Now there's a trend to use NN for recommendation (e.g. wide&deep recommendation, ncf, rnn-based recommender). Many recommender works on dataframes (columns of features) and in such cases, sparse inputs are quite common because of heavily used one-hot encoding.
We need to support Sparse Tensor as input for mostly used layers. As for which layers we need to support for Sparse Input, I think for now we can just support the layers that are used in Wide&Deep recommender. Then we can add support for new layers per request from customers.