search

buriburisuri / ByteNet

A tensorflow implementation of French-to-English machine translation using DeepMind's ByteNet .
MIT License
612 stars 95 forks source link

Implementation of dynamic unfolding & bucketed batching #4

Open randomrandom opened 7 years ago

randomrandom commented 7 years ago

Thanks for the implementation, looks really clean and simple!

I noticed thought that the dynamic unfolding is not quite implemented. There's a max_length parameter, but it's of a fixed size, in the paper they seem to determine it dynamically and use bucketing for more efficient batching. Do you plan to implement this at some point? I might work on that, can reach out if I finish the implementation

zaccharieramzi commented 5 years ago

I am under the same impression as you. I was actually looking at this pretty closely because I don't understand how dynamic unfolding works. This might not be the best place for it but if you could explain how it works in your opinion it would be cool.