huzi96 / Coarse2Fine-PyTorch

70 stars 6 forks source link

The speed of arithmetic coder #13

Open zhangzhichen111 opened 2 years ago

zhangzhichen111 commented 2 years ago

I'm sorry to bother you. I don't know why this encoder can be faster?Is it because the subprocess module of python speeds up the encoding process ? It's important to me to know this, thanks for your answer.

huzi96 commented 2 years ago

I assume that you mean the comparison to the autoregressive context model. Yes. In theory the encoding complexity is at the same level to the autoregressive context model. It is fast just because the implementation of the arithmetic coder is on C++. However, the decoding complexity of our method is in theory faster than the autoregressive version, the reason of which should be obvious.

zhangzhichen111 commented 2 years ago

Sorry, I didn't make my question clear. I mean why can your arithmetic coding implementation be faster(about 10 times)? Is it because of the multi-process acceleration that you adopted the subprocess module in the entropy encoding part?

zhangzhichen111 commented 2 years ago

I noticed that you have implemented your entropy coding part in python before. Is this part of the speedup because you implement it in c++?

huzi96 commented 2 years ago

Yes. You see executing the python AE multiple times introduces great overhead. C++ implementation makes the evaluation of the complexity focus on the network execution instead of AE.

zhangzhichen111 commented 2 years ago

OK, thank you very much.