Does the mxnet support the binary operations?

apache / mxnet

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more

https://mxnet.apache.org

Apache License 2.0

20.78k stars 6.79k forks source link

Does the mxnet support the binary operations? #2870

Open Weyne168 opened 8 years ago

Weyne168 commented 8 years ago

Recently, accelerating the CNN by the hardware operation , such as binary shift, XOR, is very popular.

I have read the source code of mxnet, and I found that there are some files named by elementwise_binary_xxx.

I wonder that whether they have realized bit wise operation in some special cases, for example, to binary weights and binary inputs, bit wise operation can be conduct?

thanks!

sxjscience commented 8 years ago

@Weyne168 The file includes operations that calculate the value of two operands, e.g, a+b, a - b. Bitwise operations like XOR are not included currently. You need to implement these OPs.

Godricly commented 8 years ago

Are you trying to implement binary net?

Weyne168 commented 8 years ago

yes, I am trying to implement it with mxnet. It is a good choice?

starimpact commented 8 years ago

i don't know how to implement binarynet with mxnet. https://github.com/itayhubara/BinaryNet.

Godricly commented 8 years ago

Can you propose a list of operator like the #2302, so we can have some idea on what to be done?

I had some experience on FP16 implementation. But those changed are not merged yet(the lstm needs extra input which involves too many changes in example). I'll list the problems I met in fp16 below, so you have a idea of what you might encounter.

extending operators support to binary. (shadow enhancement is need.)
kvstore only support preliminary type now. you binary type might have issue in merging different device gradients and different machine gradients. (Binary enhancement is need in ps-lite.)
initialization of data in binary type(random for binary type?)

asmushetzel commented 7 years ago

Is this still an active issue? Am interested as I just implemented a bunch of new operators to MxNet and also am interested in running binarized networks. Has anyone made any progress here? In terms of implementation, it seems for me that we would have to make some similar technique as C++ does for vector. Basically have a tensor that stores all entries packed in individual bits but allows access to individual bits by standard indexing. Implementing Boolean operators on such tensors would be trivial, the effort is in the additional tensor specialization.

insikk commented 5 years ago

I wish there is xor operator. +1 for this feature request