想请教下如何运行该项目

[liuxinbin12138]

想请教一下模型必须用llama3吗？我用qwen 1.5的0.5B运行总出现下面的错误： Traceback (most recent call last): File "E:\flash rag\FlashRAG-main\examples\quick_start\simple_pipeline.py", line 36, in output_dataset = pipeline.run(test_data,do_eval=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\flash rag\FlashRAG-main\flashrag\pipeline\pipeline.py", line 118, in run pred_answer_list = self.generator.generate(input_prompts) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\hanqi.conda\envs\FlashRAG-main\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "E:\flash rag\FlashRAG-main\flashrag\generator\generator.py", line 294, in generate inputs = self.tokenizer(batched_prompts, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\hanqi.conda\envs\FlashRAG-main\Lib\site-packages\transformers\tokenization_utils_base.py", line 2803, in call encodings = self._call_one(text=text, text_pair=text_pair, all_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\hanqi.conda\envs\FlashRAG-main\Lib\site-packages\transformers\tokenization_utils_base.py", line 2889, in _call_one return self.batch_encode_plus( ^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\hanqi.conda\envs\FlashRAG-main\Lib\site-packages\transformers\tokenization_utils_base.py", line 3071, in batch_encode_plus padding_strategy, truncation_strategy, max_length, kwargs = self._get_padding_truncation_strategies( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\hanqi.conda\envs\FlashRAG-main\Lib\site-packages\transformers\tokenization_utils_base.py", line 2708, in _get_padding_truncation_strategies raise ValueError( ValueError: Asking to pad but the tokenizer does not have a padding token. Please select a token to use as pad_token (tokenizer.pad_token = tokenizer.eos_token e.g.) or add a new pad token via tokenizer.add_special_tokens({'pad_token': '[PAD]'}).

[ignorejjj]

抱歉 这应该是qwen模型tokenizer的设置上有bug，我们在最新的commit里面进行了修复。请拉取最新的代码再进行尝试。

[liuxinbin12138]

麻烦您看一下又出了新错误呢 Traceback (most recent call last): File "D:\flash rag\examples\quick_start\simple_pipeline.py", line 35, in output_dataset = pipeline.run(test_data,do_eval=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\flash rag\flashrag\pipeline\pipeline.py", line 118, in run pred_answer_list = self.generator.generate(input_prompts) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\hanqi.conda\envs\FlashRAG-main\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "D:\flash rag\flashrag\generator\generator.py", line 307, in generate outputs = self.model.generate( ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\hanqi.conda\envs\FlashRAG-main\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\hanqi.conda\envs\FlashRAG-main\Lib\site-packages\transformers\generation\utils.py", line 1479, in generate return self.greedy_search( ^^^^^^^^^^^^^^^^^^^ File "C:\Users\hanqi.conda\envs\FlashRAG-main\Lib\site-packages\transformers\generation\utils.py", line 2294, in greedy_search eos_token_id_tensor = torch.tensor(eos_token_id).to(input_ids.device) if eos_token_id is not None else None ^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: Could not infer dtype of NoneType

[ignorejjj]

你好，我们重新进行了测试，发现错误来自generator设置中eos_token的设置存在问题。我们在新的提交中进行了修复(a7d889e2066f493e53f4923ebe5b2598cb6835a4)。

目前测试的qwen1.5-0.5B-chat和qwen1.5-14B-chat都正常运行。

[liuxinbin12138]

感谢！现在能运行了！

[liuxinbin12138]

sorry，当我用到SelfRAG那个pipeline又报错了： Traceback (most recent call last): File "C:\Users\hanqi.conda\envs\FlashRAG-main\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "D:\flash rag\flashrag\generator\generator.py", line 300, in generate outputs = self.model.generate( ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\hanqi.conda\envs\FlashRAG-main\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\hanqi.conda\envs\FlashRAG-main\Lib\site-packages\transformers\generation\utils.py", line 1307, in generate self._validate_model_kwargs(model_kwargs.copy()) File "C:\Users\hanqi.conda\envs\FlashRAG-main\Lib\site-packages\transformers\generation\utils.py", line 1122, in _validate_model_kwargs raise ValueError( ValueError: The following model_kwargs are not used by the model: ['return_raw_output', 'logprobs'] (note: typos in the generate arguments will also show up in this list)

[ignorejjj]

由于实现原因，目前Self-RAG pipeline仅支持vllm框架 （后续可能会进行改进），请使用vllm框架加载对应的模型。

另外，Self-RAG流程依赖于经过特殊训练过程的llm (比如原始作者提供的训练好的llama2-7B,13B)。如果使用其他一般的模型可能无法完成整个流程。