search

allenai / deep_qa

A deep NLP library, based on Keras / tf, focused on question answering (but useful for other NLP too)
Apache License 2.0
404 stars 133 forks source link

Allow tokenizers to insert begin and end tokens #276

Open matt-gardner opened 7 years ago

matt-gardner commented 7 years ago

So that if you want, you can represents words as character sequences like [@BEGIN@, w, o, r, d, @END@]. This is potentially helpful for various kinds of encoders (probably not BOW, but many others).