run toy demo and error:"Header too Long"

[Ciao-CA]

I want to run the toy demo python simple_pipeline.py --retriever_path <e5-base-v2 route> --model_path <llama-2-7b-chat-hf route> and finally got error result Traceback (most recent call last): File "/mnt/publiccache/changao/lab/FlashRAG/examples/quick_start/simple_pipeline.py", line 35, in <module> pipeline = SequentialPipeline(config, prompt_template=prompt_templete) File "/mnt/publiccache/changao/lab/FlashRAG/flashrag/pipeline/pipeline.py", line 54, in __init__ self.retriever = get_retriever(config) File "/mnt/publiccache/changao/lab/FlashRAG/flashrag/utils/utils.py", line 77, in get_retriever return getattr( File "/mnt/publiccache/changao/lab/FlashRAG/flashrag/retriever/retriever.py", line 239, in __init__ self.encoder = Encoder( File "/mnt/publiccache/changao/lab/FlashRAG/flashrag/retriever/encoder.py", line 16, in __init__ self.model, self.tokenizer = load_model(model_path=model_path, File "/mnt/publiccache/changao/lab/FlashRAG/flashrag/retriever/utils.py", line 12, in load_model model = AutoModel.from_pretrained(model_path, trust_remote_code=True) File "/usr/local/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 563, in from_pretrained return model_class.from_pretrained( File "/usr/local/lib/python3.9/site-packages/transformers/modeling_utils.py", line 3531, in from_pretrained with safe_open(resolved_archive_file, framework="pt") as f: safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLarge

please tell me how to solve it , thank you!

[DaoD]

please check if your model files are successfully downloaded.

see https://github.com/huggingface/transformers/issues/27923

[Ciao-CA]

I have downloaded a new project from meta called 'llama-2-7b-chat', and run the same command again. finally I got such results: OSError: /mnt/publiccache/changao/lab/FlashRAG/models/llama/llama-2-7b-chat does not appear to have a file named config.json. The project structure is below what can I do? should I transform it into hf style? Sorry to bother you because I am just a newbie in RAG and an intern in UCAS.

[Ciao-CA]

my command is python simple_pipeline.py --retriever_path <route>/e5-base-v2 --model_path <route>/llama/llama-2-7b-chat

[ignorejjj]

Hello, you can try downloading model in huggingface version: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf

The model files should look like the following:

[Ciao-CA]

excuse me but could I use other models instead of llama-2-7b-chat-hf?

[ignorejjj]

Of course. You just need to modify the config dict in simple_pipeline.py. Change the model2path and generator_model in config to the model you want use.

For example:

new_config_dict = { 
              'data_dir': 'dataset/',
              'index_path': 'indexes/e5_flat_sample.index',
              'corpus_path': 'indexes/sample_data.jsonl',
              'model2path': {'e5': args.retriever_path, 'llama3-8B-instruct': args.model_path},
              'generator_model': 'llama3-8B-instruct',
              'retrieval_method': 'e5',
              'metrics': ['em','f1','sub_em'],
              'retrieval_topk': 1,
              'save_intermediate_data': True
 }

[Ciao-CA]

Thank you very much and my question has been solved! I re-downloaded my e5-base-v2 and it works! I have focused all my attention on llama2-7b-chat Thanks a lot!