Marker-Inc-Korea / AutoRAG

AutoRAG: An Open-Source Framework for Retrieval-Augmented Generation (RAG) Evaluation & Optimization with AutoML-Style Automation
https://auto-rag.com/
Apache License 2.0
2.79k stars 218 forks source link

[BUG] [milvus] RPC error: [create_index], <MilvusException: (code=0, message=nlist out of range: [1, 65536])> #998

Open e7217 opened 5 hours ago

e7217 commented 5 hours ago

Describe the bug When trying to use Milvus as a vectordb, the following error occurs. Should I add any additional configuration values in the YAML settings?

logs

[11/25/24 10:59:39] INFO     [evaluator.py:127] >>                                                                                                    evaluator.py:127
                                             _        _____            _____                                                                                          
                                  /\        | |      |  __ \     /\   / ____|                                                                                         
                                 /  \  _   _| |_ ___ | |__) |   /  \ | |  __                                                                                          
                                / /\ \| | | | __/ _ \|  _  /   / /\ \| | |_ |                                                                                         
                               / ____ \ |_| | || (_) | | \ \  / ____ \ |__| |                                                                                         
                              /_/    \_\__,_|\__\___/|_|  \_\/_/    \_\_____|                                                                                         

                    INFO     [evaluator.py:128] >> Start Validation input data and config YAML file first. If you want to skip this, put the          evaluator.py:128
                             --skip_validation flag or `skip_validation` at the start_trial function.                                                                 
                    WARNING  [validator.py:50] >> Minimal Requested sample size (5) is larger than available records (1). Sampling will be limited to  validator.py:50
                             1 records.                                                                                                                               
                    INFO     [evaluator.py:228] >> Embedding BM25 corpus...                                                                           evaluator.py:228
[11/25/24 10:59:53] INFO     [evaluator.py:248] >> BM25 corpus embedding complete.                                                                    evaluator.py:248
                    ERROR    [decorators.py:147] >> RPC error: [load_collection], <MilvusException: (code=0, message=index doesn't exist,            decorators.py:147
                             collectionID 453301748613080480)>, <Time:{'RPC start': '2024-11-25 10:59:53.542740', 'RPC error': '2024-11-25                            
                             10:59:53.545308'}>                                                                                                                       
[11/25/24 10:59:54] INFO     [_client.py:1787] >> HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"                            _client.py:1787
[11/25/24 10:59:58] ERROR    [decorators.py:147] >> RPC error: [create_index], <MilvusException: (code=0, message=nlist out of range: [1, 65536])>,  decorators.py:147
                             <Time:{'RPC start': '2024-11-25 10:59:58.492197', 'RPC error': '2024-11-25 10:59:58.498422'}>                                            
Ingesting VectorDB... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% 0/1 0:00:05

Desktop (please complete the following information):

configuration

vectordb:
- name: autorag_2024_11_21
  db_type: milvus
  embedding_model: openai
  collection_name: autorag_2024_11_21
  uri: http://192.xxx.xxx.xxx:19530
  # embedding_batch: 50
  similarity_metric: cosine
node_lines:
- node_line_name: retrieve_node_line
  nodes:
    - node_type: retrieval
      strategy:
        metrics: [retrieval_f1, retrieval_ndcg, retrieval_map]
      top_k: 3
      modules:
        - module_type: bm25
          bm25_tokenizer: [ ko_kiwi, ko_okt, ko_kkma ]
        - module_type: vectordb
          vectordb: autorag_2024_11_21
        - module_type: hybrid_rrf
        - module_type: hybrid_cc
          normalize_method: [ mm, tmm ]
- node_line_name: post_retrieve_node_line
  nodes:
    - node_type: prompt_maker
      strategy:
        metrics:
          - metric_name: rouge
          - metric_name: sem_score
            embedding_model: openai
#          - metric_name: bert_score
#            lang: ko
        generator_modules:
          - module_type: openai_llm
            llm: gpt-4o-mini
      modules:
        - module_type: fstring
          prompt:
          - | 
            단락을 읽고 질문에 답하세요. \n 질문 : {query} \n 단락: {retrieved_contents} \n 답변 :
          - |
            단락을 읽고 질문에 답하세요. 답할때 단계별로 천천히 고심하여 답변하세요. 반드시 단락 내용을 기반으로 말하고 거짓을 말하지 마세요. \n 질문: {query} \n 단락: {retrieved_contents} \n 답변 :
    - node_type: generator
      strategy:
        metrics: # bert_score 및 g_eval 사용 역시 추천합니다. 빠른 실행을 위해 여기서는 제외하고 하겠습니다.
          - metric_name: rouge
          - metric_name: sem_score
            embedding_model: openai
#          - metric_name: bert_score
#            lang: ko
      modules:
        - module_type: openai_llm
          llm: gpt-4o-mini
          temperature: [ 0.1, 1.0 ]
          batch: 16

Desktop (please complete the following information):

vkehfdl1 commented 4 hours ago

@bwook00 I think we have problem to ingest large data into the Milvus. https://github.com/milvus-io/milvus/issues/16343 Can you investigate this bug?

@e7217 Looks like we have to change Milvus ingest code in milvus.py Thank you for reporting us bug.