hao-ai-lab / LookaheadDecoding

[ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
https://arxiv.org/abs/2402.02057
Apache License 2.0
1.11k stars 65 forks source link

Cannot use save_log and log_history #61

Open david-wei-01001 opened 4 months ago

david-wei-01001 commented 4 months ago

Hi, I want to access

CONFIG_MAP["log“]

to record the number of steps took, but after I try both

import lade
...
lade.save_log()

or

from lade.utils import save_log
...
save_log()

It seems all says that "save_log" does not exist in lade. same happens for "log_history"

Also, I have tryed to print want comes with lade, and it seems they are both not there:

import lade
print(dir(lade))
import lade.utils
print(dir(lade.utils))

And the output is:

['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'augment_all', 'augment_generate', 'augment_llama', 'config_lade', 'decoding', 'models', 'utils']
['AutoConfig', 'AutoModelForCausalLM', 'AutoTokenizer', 'CONFIG_MAP', 'FUNC_MAP', 'GenerationMixin', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'augment_all', 'augment_generate', 'augment_llama', 'config_lade', 'get_hf_model', 'get_model', 'greedy_search_proxy', 'llama', 'modeling_llama', 'os', 'torch']
david-wei-01001 commented 4 months ago

It turns out, if using

pip install lade

Then these functions cannot be accessed, but if just cloning the repo

git clone https://github.com/hao-ai-lab/LookaheadDecoding.git
cd LookaheadDecoding
pip install -r requirements.txt
pip install -e .

Then in minimal.py it can be used freely

I will leave this open as adding this feature to lade package would be great