search

IntelLabs / SkimCaffe

Caffe for Sparse Convolutional Neural Network
Other
238 stars 64 forks source link

libxsmm and other frameworks #10

Open cvasfi opened 7 years ago

cvasfi commented 7 years ago

our sparse convolution routine will be migrated to libxsmm library so that it can be also used by other frameworks like TensorFlow

I suppose libxsmm is already used in the project now. Does it mean that it can be used with tensorflow? If so, could you provide a short example/guideline on how to use/test it?

jspark1105 commented 7 years ago

You're right that SkimCaffe is already using libxsmm but only for sparse-matrix-times-dense-matrix (SpMDM) multiplication for fully-connected layers. Unfortunately, sparse convolution routine (in include/caffe/utils/sconv.hpp) has not been integrated into libxsmm. It has been on my TODO list so I'll try to do that as soon as I have time. It shouldn't be hard and will be mostly a matter of matching with the sparse matrix format used by SpMDM in libxsmm. Once integrated, we should be able to add a TF op that uses the libxsmm sparse convolution and SpMDM routines to speedup conv and fc layers.

hfp commented 7 years ago

Since you asked if any of LIBXSMM's sparse operations can be used in TF, I want to point you to:

TFROOT/tensorflow/python/kernel_tests/sparse_matmul_op_test.py

This is the test case covering SpMDM, but it is a regular Python script orchestrating TF's frontend. This should provide a reasonable piece of sample code as well. If you want to see the implementation, you may look at https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/kernels/sparse_matmul_op.cc. As a side note, TF also integrates LIBXSMM's convolution kernels (non-sparse).

cvasfi commented 7 years ago

@jspark1105 Thank you for the reply that's great, i'll be following the news.

@hfp Thanks, that looks quite interesting as well.