Design tokenization

[perkss]

Has any work been done on this?

[turbopape]

Hey ,

No not really... Would you be interested ?

Le mar. 7 nov. 2017 à 2:44 PM, Stuart notifications@github.com a écrit :

Has any work been done on this?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/fekr/postagga/issues/4#issuecomment-342486167, or mute the thread https://github.com/notifications/unsubscribe-auth/ACpoH8c9o2CDQTclnkTix4Hp6XaCmavzks5s0F7LgaJpZM4Mb30g .

--

[image: --]

Rafik Naccache [image: https://]about.me/rafik_naccache https://about.me/rafik_naccache?promo=email_sig&utm_source=email_sig&utm_medium=email_sig&utm_campaign=external_links

[perkss]

Yes I would be, lets move this forward let me know some initial thoughts for this? Could just tokenize simply on grammer to begin with

[turbopape]

Yeah sure, please go ahead with this, no need for fancy hyphenation etc for starters :) Thank you !

2017-11-07 21:20 GMT+01:00 Stuart notifications@github.com:

Yes I would be, lets move this forward let me know some initial thoughts for this? Could just tokenize simply on grammer to begin with

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/fekr/postagga/issues/4#issuecomment-342609772, or mute the thread https://github.com/notifications/unsubscribe-auth/ACpoH8qLFClsiNYVYPrZGt5B4HU17Fc5ks5s0LuigaJpZM4Mb30g .

--

[image: --]

Rafik Naccache [image: https://]about.me/rafik_naccache https://about.me/rafik_naccache?promo=email_sig&utm_source=email_sig&utm_medium=email_sig&utm_campaign=external_links