Add support to make bucket to variable length data

mila-iqia / fuel

A data pipeline framework for machine learning

MIT License

868 stars 268 forks source link

Add support to make bucket to variable length data #380

Open rudaoshi opened 7 years ago

rudaoshi commented 7 years ago

Add bucket support alike Batch support.

dmitriy-serdyuk commented 7 years ago

Sorry? Can you elaborate? An example would be great.

rudaoshi commented 7 years ago

sorry, I means splitting sequences with variable length into buckets. In each bucket, the length of input sequences are similar so the data can be padded with minimum pad size and reduce the computation cost.