search

MikeHopcroft / ShortOrder

A natural language conversational agent for ordering and organizing items from a catalog.
MIT License
13 stars 7 forks source link

Export Token Types #35

Closed ryanvolum closed 5 years ago

ryanvolum commented 5 years ago

In order to write an informed parser, I need to be able to distinguish between different kinds of tokens. short-order exposes a printTokens function, which utilizes AnyToken, UnknownToken, AttributeToken, EntityToken, IntentToken, and QuantityToken. Please make these types available so that a custom parser can take advantage of them

MikeHopcroft commented 5 years ago

My plan is to factor out the tokenization and place it in a package called token-flow. In token-flow, the token types will be part of a sample application. The code in short-order will then depend on token-flow and will define its own, domain-specific tokens which will be exported.

MikeHopcroft commented 5 years ago

This should be fixed in the tokenflow branch. Will resolve once I resolve #38.