search

namin / llm-verified-with-monte-carlo-tree-search

LLM verified with Monte Carlo Tree Search
https://arxiv.org/abs/2402.08147
MIT License
210 stars 25 forks source link

add workaround for StarCoder #54

Closed shenniger closed 2 months ago

shenniger commented 2 months ago

This adds a workaround to avoid the stop word issues with StarCoder 2 (and possibly other models).

Very ugly workaround unfortunately, that touches a few different files. Not super confident about this, it will need some more testing.

Usage example: python run_intermediate_expansion.py --base_model bigcode/starcoder2-15b --stop_token_workaround True

shenniger commented 2 months ago

@davidbrandfonbrener, can you review this?