tiktoken dependency does not seem getting used, makes install process heavy

microsoft / sammo

A library for prompt engineering and optimization (SAMMO = Structure-aware Multi-Objective Metaprompt Optimization)

MIT License

513 stars 27 forks source link

tiktoken dependency does not seem getting used, makes install process heavy #6

Closed onkarbhardwaj closed 8 months ago

onkarbhardwaj commented 9 months ago

find . -iname "*" | xargs grep -in "tiktoken" does not yield any project files except toml file - is this dependency necessary? This is a heavyweight dependency which perhaps uses other heavy dependencies as well.

t-schn commented 8 months ago

You are right - it was only used in earlier versions. Removed it and published a new version on pip. Thanks!