Support cost estimation before execution and improve configuration

stanford-oval / storm

An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.

http://storm.genie.stanford.edu

MIT License

11.95k stars 1.08k forks source link

Support cost estimation before execution and improve configuration #22

Closed shaoyijia closed 2 months ago

shaoyijia commented 5 months ago

The default configurations in the Readme.md are corresponding to our hyper-parameters setup in the NAACL'24 paper.

As people are using/building on top of this project, having cost estimation (maximum number of call to language model, maximum number of call to search engine/retriever) is necessary.

shaoyijia commented 5 months ago

Currently, we provide a profiling function after processing a topic. You can call summary() to see the time, model usage for each step. Run reset() before starting a new topic.