search

zeno-ml / zeno-build

Build, evaluate, understand, and fix LLM-based apps
MIT License
482 stars 33 forks source link

Add number count features #148

Closed neubig closed 1 year ago

neubig commented 1 year ago

Description

This PR:

  1. Adds a tokenizer to tokenize text into words
  2. Adds a feature the counts number words in English and digits
  3. Adds the number count feature to the chatbot example

The motivation for this is that there are a lot of number-based utterances in the chatbot example, and the feature could uncover interesting patterns.

Blocked by