search

run-llama / llama_index

LlamaIndex is a data framework for your LLM applications
https://docs.llamaindex.ai
MIT License
36.75k stars 5.27k forks source link

Use ChatGPTLLM to build Index to disk but load from disk the llm is default to text-davinci-003 #637

Closed 8Avalon8 closed 1 year ago

8Avalon8 commented 1 year ago

I use chatgptllm to build index

llm_predictor = ChatGPTLLMPredictor(prepend_messages=CUSTOM_MESSAGE, model_name="gpt-3.5-turbo-0301", temperature=0,max_tokens=4096) index = GPTSimpleVectorIndex( documents, llm_predictor=llm_predictor, prompt_helper=prompt_helper ) index.save_to_disk('index.json')

then load

index = GPTSimpleVectorIndex.load_from_disk('index.json')

I found that this index’s llm is default to text-davinci-0033 and max_output token is 256

so I changed this line index = GPTSimpleVectorIndex.load_from_disk('serverindex.json', llm_predictor=llm_predictor)

index.query(input_sentence, response_mode="default",refine_template=CHAT_REFINE_PROMPT, similarity_top_k=2)

bug will throw an exception: {'message': 'Unrecognized request argument supplied: model_name', 'type': 'invalid_request_error', 'param': None, 'code': None}

INFO:openai:error_code=None error_message='Unrecognized request argument supplied: model_name' error_param=None error_type=invalid_request_error message='OpenAI API error received' stream_error=False Traceback (most recent call last): File "D:\ChatGPTMemory\venv\Lib\site-packages\gradio\routes.py", line 384, in run_predict output = await app.get_blocks().process_api( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\gradio\blocks.py", line 1032, in process_api result = await self.call_function( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\gradio\blocks.py", line 844, in call_function prediction = await anyio.to_thread.run_sync( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\anyio\to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread return await future ^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run result = context.run(func, *args) ^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\main.py", line 188, in predict response = index.query(input_sentence, response_mode="default",refine_template=CHAT_REFINE_PROMPT, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\llama_index\indices\base.py", line 395, in query return query_runner.query(query_str, self._index_struct) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\llama_index\indices\query\query_runner.py", line 120, in query return query_obj.query(query_bundle) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\llama_index\token_counter\token_counter.py", line 55, in wrapped_llm_predict f_return_val = f(_self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\llama_index\indices\query\base.py", line 302, in query response = self._query(query_bundle) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\llama_index\indices\query\base.py", line 276, in _query response_str = self._give_response_for_nodes(query_bundle.query_str) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\llama_index\indices\query\base.py", line 218, in _give_response_for_nodes response = self.response_builder.get_response( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\llama_index\indices\response\builder.py", line 276, in get_response return self._get_response_default(query_str, prev_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\llama_index\indices\response\builder.py", line 201, in _get_response_default return self.get_response_over_chunks( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\llama_index\indices\response\builder.py", line 183, in get_response_over_chunks response = self.give_response_single( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\llama_index\indices\response\builder.py", line 144, in give_response_single response, _ = self.llm_predictor.predict( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\llama_index\langchain_helpers\chain_wrapper.py", line 131, in predict llm_prediction = self._predict(prompt, **prompt_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\llama_index\langchain_helpers\chatgpt.py", line 54, in _predict response = openai.ChatCompletion.create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\openai\api_resources\chat_completion.py", line 25, in create return super().create(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\openai\api_resources\abstract\engine_api_resource.py", line 153, in create response, _, api_key = requestor.request( ^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\openai\api_requestor.py", line 226, in request resp, got_stream = self._interpret_response(result, stream) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ChatGPTMemory\venv\Lib\site-packages\openai\api_requestor.py", line 619, in _interpret_response self._interpret_response_line( File "D:\ChatGPTMemory\venv\Lib\site-packages\openai\api_requestor.py", line 679, in _interpret_response_line raise self.handle_error_response( openai.error.InvalidRequestError: Unrecognized request argument supplied: model_name error_code=None error_message='Unrecognized request argument supplied: model_name' error_param=None error_type=invalid_request_error message='OpenAI API error received' stream_error=False

logan-markewich commented 1 year ago

@8Avalon8 try setting the model name to "gpt-3.5-turbo" instead, I think your existing model name is invalid.

For a VectorIndex, the LLM is not used at all during index construction (only the embed_model,) which is why you don't see the error until you call query()

8Avalon8 commented 1 year ago

@8Avalon8 try setting the model name to "gpt-3.5-turbo" instead, I think your existing model name is invalid.

For a VectorIndex, the LLM is not used at all during index construction (only the embed_model,) which is why you don't see the error until you call query()

Thanks for your help！I found it's my mistake， llm_predictor = ChatGPTLLMPredictor(prepend_messages=CUSTOM_MESSAGE, model_name="gpt-3.5-turbo-0301", “model_name” was a wrong argument and it should be model