Open mhamra opened 1 year ago
I've made a mistake running the download.sh script. I've passed my email instead of the URL received from FB.
Did your issue resolved? I am unable to run on windows 10 as well. I am getting "Distributed package doesnt have NCCL built-in error"
@manoj21192 This will work on windows
temperature = 0
top_p = 0
max_seq_len = 4096
max_batch_size = 1
max_gen_len = None
num_of_worlds = 1
torch.distributed.init_process_group(backend='gloo', init_method='tcp://localhost:23455', world_size=num_of_worlds, rank=0)
generator = Llama.build(
ckpt_dir="C:/AI/LLaMA2_Docker_FileSystem/codellama/CodeLlama-7b-Instruct",
tokenizer_path="C:/AI/LLaMA2_Docker_FileSystem/codellama/CodeLlama-7b-Instruct/tokenizer.model",
max_seq_len=max_seq_len,
max_batch_size=max_batch_size,
model_parallel_size = num_of_worlds
)
UPDATE
I've made a mistake running the download.sh script. I've passed my email instead of the URL received from FB.
Thank you! I can reproduce this. I at first entered my email and then noticed my error and entered the correct URL when running download.sh
, but loading was still not possible.
I cloned the repository again, entered the correct URL on first try and then it worked.
What mistake am I making here? from typing import Optional
import fire
from llama import Llama
def main( ckpt_dir: "D:\pathto\codellama\CodeLlama-7b", tokenizer_path: "D:\pathto\codellama\CodeLlama-7b\tokenizer.model", temperature: float = 0.2, top_p: float = 0.9, max_seq_len: int = 256, max_batch_size: int = 4, max_gen_len: Optional[int] = None, ): generator = Llama.build( ckpt_dir=ckpt_dir, tokenizer_path=tokenizer_path, max_seq_len=max_seq_len, max_batch_size=max_batch_size, ) "
I Am getting this error: "
D:\path2\codellama>python example_completion.py
ERROR: The function received no value for the required argument: ckpt_dir
Usage: example_completion.py CKPT_DIR TOKENIZER_PATH
For detailed information on this command, run: example_completion.py --help "
What mistake am I making here? from typing import Optional
import fire
from llama import Llama
def main( ckpt_dir: "D:\pathto\codellama\CodeLlama-7b", tokenizer_path: "D:\pathto\codellama\CodeLlama-7b\tokenizer.model", temperature: float = 0.2, top_p: float = 0.9, max_seq_len: int = 256, max_batch_size: int = 4, max_gen_len: Optional[int] = None, ): generator = Llama.build( ckpt_dir=ckpt_dir, tokenizer_path=tokenizer_path, max_seq_len=max_seq_len, max_batch_size=max_batch_size, ) "
I Am getting this error: "
D:\path2\codellama>python example_completion.py ERROR: The function received no value for the required argument: ckpt_dir Usage: example_completion.py CKPT_DIR TOKENIZER_PATH optional flags: --temperature | --top_p | --max_seq_len | --max_batch_size | --max_gen_len
For detailed information on this command, run: example_completion.py --help "
@bronzwikgk
Based on the code and error message you've provided, here are some issues I've identified:
Here's a revised version of the code:
from typing import Optional
import fire
from llama import Llama
def main(
ckpt_dir: str = r"D:\pathto\codellama\CodeLlama-7b",
tokenizer_path: str = r"D:\pathto\codellama\CodeLlama-7b\tokenizer.model",
temperature: float = 0.2,
top_p: float = 0.9,
max_seq_len: int = 256,
max_batch_size: int = 4,
max_gen_len: Optional[int] = None,
):
generator = Llama.build(
ckpt_dir=ckpt_dir,
tokenizer_path=tokenizer_path,
max_seq_len=max_seq_len,
max_batch_size=max_batch_size,
)
if __name__ == "__main__":
fire.Fire(main)
ckpt_dir
and tokenizer_path
to be str
.r
), which allow for backslashes to be interpreted correctly.if __name__ == "__main__": fire.Fire(main)
to run the function when the script is executed.Try running the updated code and see if the error persists.
Thanks, Moved One step ahead.
Getting this error now: {{
Traceback (most recent call last):
File "D:\shunyadotek\codellama\example_completion.py", line 55, in
torch.distributed.init_process_group(backend='gloo', init_method='tcp://localhost:23455', world_size=num_of_worlds, rank=0)
@bronzwikgk I don't see this line in your code : torch.distributed.init_process_group(backend='gloo', init_method='tcp://localhost:23455', world_size=num_of_worlds, rank=0)
Are you sure you have it in your code? See my answer with the full code with this line, few answers above.
@bronzwikgk Right, I see that you are using torch.distributed.init_process_group("nccl") nccl is for linux only, use my example above.
Hi, I've tried to run the examples, but I received this error.