updating preprocesor-shards.py to support additional input features

harvardnlp / seq2seq-attn

Sequence-to-sequence model with LSTM encoder/decoders and attention

http://nlp.seas.harvard.edu/code

MIT License

1.26k stars 278 forks source link

Open mdasadul opened 8 years ago

mdasadul commented 8 years ago

To update preprocessor-shards.py along with preprocessor.py

guillaumekln commented 8 years ago

Thank you for updating this script as well.

@yoonkim Do you plan to factorize the preprocessing code?

yoonkim commented 8 years ago

Hmm yeah preprocess-shards should really be part of preprocess with a --shardsize option... I'll see if I have a chance to factorize it