search

rkcosmos / deepcut

A Thai word tokenization library using Deep Neural Network
MIT License
420 stars 96 forks source link

Pruning model #56

Closed p16i closed 4 years ago

p16i commented 5 years ago

Hi,

I've made some analysis, investigating whether some layers in DeepCut are redundant. There are some big layers that we can remove while DeepCut's tokenization performance drops slightly.

Having this result, you might consider pruning or doing quantisation to the model. As a result, DeepCut will become smaller and (hopefully) faster.

Please find more information in this notebook.

titipata commented 5 years ago

Nice one! I think it looks really good. Do you want to make the PR and potentially the trained model to the repository? However, we might have to change the structure of the library a bit from import deepcut; deepcut.tokenize(...) so that we allow a user to import different (shrink) version of the model.

P.S. We also have the version training using MobileNet available here: https://rkcosmos.github.io/deepcut/. I haven't tested the performance of it tho.

p16i commented 5 years ago

Hi,

Thanks for the comment. I personally think th shrieked model would run faster than the original one but not much. Actually, I think the majority component that significantly drags DeepCut's performance is the way input features are prepared. According to https://github.com/rkcosmos/deepcut/blob/master/deepcut/train.py#L107, the implementation transposes the sequence of characters in each line to become rows. IMHO, we would gain more speed if we reimplement the feature processing by convolving directly on each line directly.

titipata commented 5 years ago

@heytitle totally agree on this. The training part can be much much faster without transforming to a dataframe. However, in the prediction part, we have this function to transform input text to features. For the text with small/medium length, it seems to work perfectly fine.

The bottleneck is probably really coming from the prediction part here which can speed up a lot faster?

p16i commented 5 years ago

Hi,

I won't be able to make a PR for fixing this issue. However, Hacktoberfest 2020 is coming in October. Maybe we can create an issue and possibly tag it with a theme tag, i.e. hacktoberfest?

titipata commented 5 years ago

@heytitle, should I refer this issue to an analysis provided in attacut? Maybe you can update the documentation on deepcut performance and size of the model etc. etc.

titipata commented 5 years ago

Can we run the notebook provided above to see the performance of deepcut 0.7.0 (on TensorFlow 2.0)? @kittinan

rkcosmos commented 4 years ago

I'll just close this one but I will keep performance/speed issue in the back of my head.