search

TheAiSingularity / graphrag-local-ollama

Local models support for Microsoft's graphrag using ollama (llama3, mistral, gemma2 phi3)- LLM & Embedding extraction
MIT License
345 stars 39 forks source link

Why does the project ask for a key when querying? #7

Open XinyuShe opened 3 weeks ago

XinyuShe commented 3 weeks ago 
(graphrag-ollama-local) $ python3 -m graphrag.query --data ./ragtest/output/20240709-054243/artifacts/ --method local "What is machine learning?"

INFO: Reading settings from environment variables
Traceback (most recent call last):
  File "xxx/envs/graphrag-ollama-local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "xxx/envs/graphrag-ollama-local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "xx/graphrag-local-ollama/graphrag/query/__main__.py", line 76, in <module>
    run_local_search(
  File "xxx/graphrag-local-ollama/graphrag/query/cli.py", line 102, in run_local_search
    data_dir, root_dir, config = _configure_paths_and_settings(data_dir, root_dir)
  File "xxx/graphrag-local-ollama/graphrag/query/cli.py", line 167, in _configure_paths_and_settings
    config = _create_graphrag_config(root_dir, data_dir)
  File "xxx/graphrag-local-ollama/graphrag/query/cli.py", line 185, in _create_graphrag_config
    return _read_config_parameters(cast(str, root or data_dir))
  File "xxx/graphrag-local-ollama/graphrag/query/cli.py", line 212, in _read_config_parameters
    return create_graphrag_config(root_dir=root)
  File "xxx/graphrag-local-ollama/graphrag/config/create_graphrag_config.py", line 231, in create_graphrag_config
    raise ApiKeyMissingError
graphrag.config.errors.ApiKeyMissingError: API Key is required for Completion API. Please set either the OPENAI_API_KEY, GRAPHRAG_API_KEY or GRAPHRAG_LLM_API_KEY environment variable.

It works before the query

TheAiSingularity commented 3 weeks ago

export GRAPHRAG_API_KEY=1234 This should fix it, the setup expects a dummy key. Let me know if this fixes your issue.

(graphrag-ollama-local) $ python3 -m graphrag.query --data ./ragtest/output/20240709-054243/artifacts/ --method local "What is machine learning?"

INFO: Reading settings from environment variables
Traceback (most recent call last):
  File "xxx/envs/graphrag-ollama-local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "xxx/envs/graphrag-ollama-local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "xx/graphrag-local-ollama/graphrag/query/__main__.py", line 76, in <module>
    run_local_search(
  File "xxx/graphrag-local-ollama/graphrag/query/cli.py", line 102, in run_local_search
    data_dir, root_dir, config = _configure_paths_and_settings(data_dir, root_dir)
  File "xxx/graphrag-local-ollama/graphrag/query/cli.py", line 167, in _configure_paths_and_settings
    config = _create_graphrag_config(root_dir, data_dir)
  File "xxx/graphrag-local-ollama/graphrag/query/cli.py", line 185, in _create_graphrag_config
    return _read_config_parameters(cast(str, root or data_dir))
  File "xxx/graphrag-local-ollama/graphrag/query/cli.py", line 212, in _read_config_parameters
    return create_graphrag_config(root_dir=root)
  File "xxx/graphrag-local-ollama/graphrag/config/create_graphrag_config.py", line 231, in create_graphrag_config
    raise ApiKeyMissingError
graphrag.config.errors.ApiKeyMissingError: API Key is required for Completion API. Please set either the OPENAI_API_KEY, GRAPHRAG_API_KEY or GRAPHRAG_LLM_API_KEY environment variable.

It works before the query

export GRAPHRAG_API_KEY=1234 This should fix it, the setup expects a dummy key. Let me know if this fixes your issue.

XinyuShe commented 3 weeks ago

Thank you for your quick response!

Unfortunately, it does not work

(graphrag-ollama-local) $ export GRAPHRAG_API_KEY=1234
(graphrag-ollama-local) $ python3 -m graphrag.query --data ./ragtest/output/20240709-054243/artifacts/ --method local "What is machine learning?"

INFO: Reading settings from environment variables
[2024-07-09T06:06:37Z WARN  lance::dataset] No existing dataset at xxx/graphrag-local-ollama/lancedb/description_embedding.lance, it will be created
creating llm client with {'api_key': 'REDACTED,len=4', 'type': "openai_chat", 'model': 'gpt-4-turbo-preview', 'max_tokens': 4000, 'temperature': 0.0, 'top_p': 1.0, 'request_timeout': 180.0, 'api_base': None, 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': None, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}
creating embedding llm client with {'api_key': 'REDACTED,len=4', 'type': "openai_embedding", 'model': 'text-embedding-3-small', 'max_tokens': 4000, 'temperature': 0, 'top_p': 1, 'request_timeout': 180.0, 'api_base': None, 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': None, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}
Error embedding chunk {'OpenAIEmbedding': "Error code: 401 - {'error': {'message': 'Incorrect API key provided: 1234. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}"}
Traceback (most recent call last):
  File "xxx/graphrag-ollama-local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "xxx/graphrag-ollama-local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "xxx/graphrag-local-ollama/graphrag/query/__main__.py", line 76, in <module>
    run_local_search(
  File "xxx/graphrag-local-ollama/graphrag/query/cli.py", line 154, in run_local_search
    result = search_engine.search(query=query)
  File "xxx/graphrag-local-ollama/graphrag/query/structured_search/local_search/search.py", line 118, in search
    context_text, context_records = self.context_builder.build_context(
  File "xxx/graphrag-local-ollama/graphrag/query/structured_search/local_search/mixed_context.py", line 139, in build_context
    selected_entities = map_query_to_entities(
  File "xxx/graphrag-local-ollama/graphrag/query/context_builder/entity_extraction.py", line 55, in map_query_to_entities
    search_results = text_embedding_vectorstore.similarity_search_by_text(
  File "xxx/graphrag-local-ollama/graphrag/vector_stores/lancedb.py", line 118, in similarity_search_by_text
    query_embedding = text_embedder(text)
  File "xxx/graphrag-local-ollama/graphrag/query/context_builder/entity_extraction.py", line 57, in <lambda>
    text_embedder=lambda t: text_embedder.embed(t),
  File "xxx/graphrag-local-ollama/graphrag/query/llm/oai/embedding.py", line 96, in embed
    chunk_embeddings = np.average(chunk_embeddings, axis=0, weights=chunk_lens)
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/numpy/lib/function_base.py", line 550, in average
    raise ZeroDivisionError(
ZeroDivisionError: Weights sum to zero, can't be normalized
TheAiSingularity commented 3 weeks ago

Thank you for your quick response!

Unfortunately, it does not work

(graphrag-ollama-local) $ export GRAPHRAG_API_KEY=1234
(graphrag-ollama-local) $ python3 -m graphrag.query --data ./ragtest/output/20240709-054243/artifacts/ --method local "What is machine learning?"

INFO: Reading settings from environment variables
[2024-07-09T06:06:37Z WARN  lance::dataset] No existing dataset at xxx/graphrag-local-ollama/lancedb/description_embedding.lance, it will be created
creating llm client with {'api_key': 'REDACTED,len=4', 'type': "openai_chat", 'model': 'gpt-4-turbo-preview', 'max_tokens': 4000, 'temperature': 0.0, 'top_p': 1.0, 'request_timeout': 180.0, 'api_base': None, 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': None, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}
creating embedding llm client with {'api_key': 'REDACTED,len=4', 'type': "openai_embedding", 'model': 'text-embedding-3-small', 'max_tokens': 4000, 'temperature': 0, 'top_p': 1, 'request_timeout': 180.0, 'api_base': None, 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': None, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}
Error embedding chunk {'OpenAIEmbedding': "Error code: 401 - {'error': {'message': 'Incorrect API key provided: 1234. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}"}
Traceback (most recent call last):
  File "xxx/graphrag-ollama-local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "xxx/graphrag-ollama-local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "xxx/graphrag-local-ollama/graphrag/query/__main__.py", line 76, in <module>
    run_local_search(
  File "xxx/graphrag-local-ollama/graphrag/query/cli.py", line 154, in run_local_search
    result = search_engine.search(query=query)
  File "xxx/graphrag-local-ollama/graphrag/query/structured_search/local_search/search.py", line 118, in search
    context_text, context_records = self.context_builder.build_context(
  File "xxx/graphrag-local-ollama/graphrag/query/structured_search/local_search/mixed_context.py", line 139, in build_context
    selected_entities = map_query_to_entities(
  File "xxx/graphrag-local-ollama/graphrag/query/context_builder/entity_extraction.py", line 55, in map_query_to_entities
    search_results = text_embedding_vectorstore.similarity_search_by_text(
  File "xxx/graphrag-local-ollama/graphrag/vector_stores/lancedb.py", line 118, in similarity_search_by_text
    query_embedding = text_embedder(text)
  File "xxx/graphrag-local-ollama/graphrag/query/context_builder/entity_extraction.py", line 57, in <lambda>
    text_embedder=lambda t: text_embedder.embed(t),
  File "xxx/graphrag-local-ollama/graphrag/query/llm/oai/embedding.py", line 96, in embed
    chunk_embeddings = np.average(chunk_embeddings, axis=0, weights=chunk_lens)
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/numpy/lib/function_base.py", line 550, in average
    raise ZeroDivisionError(
ZeroDivisionError: Weights sum to zero, can't be normalized

For now, only "--method global" search would work. working on fixing it for the local search too. Did exporting the dummy key worked?

XinyuShe commented 3 weeks ago

I don't known if it works or not now......

(graphrag-ollama-local) $ python3 -m graphrag.query --data ./ragtest/output/20240709-054243/artifacts/ --method global "What is machine learning?"

INFO: Reading settings from environment variables
creating llm client with {'api_key': 'REDACTED,len=4', 'type': "openai_chat", 'model': 'gpt-4-turbo-preview', 'max_tokens': 4000, 'temperature': 0.0, 'top_p': 1.0, 'request_timeout': 180.0, 'api_base': None, 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': None, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}
Exception in _map_response_single_batch
Traceback (most recent call last):
  File "xxx/graphrag-local-ollama/graphrag/query/structured_search/global_search/search.py", line 182, in _map_response_single_batch
    search_response = await self.llm.agenerate(
  File "xxx/graphrag-local-ollama/graphrag/query/llm/oai/chat_openai.py", line 110, in agenerate
    async for attempt in retryer:
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/tenacity/asyncio/__init__.py", line 166, in __anext__
    do = await self.iter(retry_state=self._retry_state)
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/tenacity/asyncio/__init__.py", line 153, in iter
    result = await action(retry_state)
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/tenacity/_utils.py", line 99, in inner
    return call(*args, **kwargs)
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/tenacity/__init__.py", line 398, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
  File "xxx/graphrag-ollama-local/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "xxx/graphrag-ollama-local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "xxx/graphrag-local-ollama/graphrag/query/llm/oai/chat_openai.py", line 112, in agenerate
    return await self._agenerate(
  File "xxx/graphrag-local-ollama/graphrag/query/llm/oai/chat_openai.py", line 171, in _agenerate
    response = await self.async_client.chat.completions.create(  # type: ignore
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/openai/resources/chat/completions.py", line 1289, in create
    return await self._post(
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/openai/_base_client.py", line 1816, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/openai/_base_client.py", line 1514, in request
    return await self._request(
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/openai/_base_client.py", line 1610, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.AuthenticationError: Error code: 401 - {'error': {'message': 'Incorrect API key provided: 1234. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}

SUCCESS: Global Search Response: I am sorry but I am unable to answer this question given the provided data.
TheAiSingularity commented 3 weeks ago

I don't known if it works or not now......

(graphrag-ollama-local) $ python3 -m graphrag.query --data ./ragtest/output/20240709-054243/artifacts/ --method global "What is machine learning?"

INFO: Reading settings from environment variables
creating llm client with {'api_key': 'REDACTED,len=4', 'type': "openai_chat", 'model': 'gpt-4-turbo-preview', 'max_tokens': 4000, 'temperature': 0.0, 'top_p': 1.0, 'request_timeout': 180.0, 'api_base': None, 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': None, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}
Exception in _map_response_single_batch
Traceback (most recent call last):
  File "xxx/graphrag-local-ollama/graphrag/query/structured_search/global_search/search.py", line 182, in _map_response_single_batch
    search_response = await self.llm.agenerate(
  File "xxx/graphrag-local-ollama/graphrag/query/llm/oai/chat_openai.py", line 110, in agenerate
    async for attempt in retryer:
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/tenacity/asyncio/__init__.py", line 166, in __anext__
    do = await self.iter(retry_state=self._retry_state)
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/tenacity/asyncio/__init__.py", line 153, in iter
    result = await action(retry_state)
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/tenacity/_utils.py", line 99, in inner
    return call(*args, **kwargs)
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/tenacity/__init__.py", line 398, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
  File "xxx/graphrag-ollama-local/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "xxx/graphrag-ollama-local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "xxx/graphrag-local-ollama/graphrag/query/llm/oai/chat_openai.py", line 112, in agenerate
    return await self._agenerate(
  File "xxx/graphrag-local-ollama/graphrag/query/llm/oai/chat_openai.py", line 171, in _agenerate
    response = await self.async_client.chat.completions.create(  # type: ignore
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/openai/resources/chat/completions.py", line 1289, in create
    return await self._post(
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/openai/_base_client.py", line 1816, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/openai/_base_client.py", line 1514, in request
    return await self._request(
  File "xxx/graphrag-ollama-local/lib/python3.10/site-packages/openai/_base_client.py", line 1610, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.AuthenticationError: Error code: 401 - {'error': {'message': 'Incorrect API key provided: 1234. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}

SUCCESS: Global Search Response: I am sorry but I am unable to answer this question given the provided data.

Can you provide your ./ragtest/settings.yaml file here

XinyuShe commented 3 weeks ago

I just change the model to llama3

encoding_model: cl100k_base
skip_workflows: []
llm:
  api_key: ${GRAPHRAG_API_KEY}
  type: openai_chat # or azure_openai_chat
  model: llama3
  model_supports_json: true # recommended if this is available for your model.
  # max_tokens: 4000
  # request_timeout: 180.0
  api_base: http://localhost:11434/v1
  # api_version: 2024-02-15-preview
  # organization: <organization_id>
  # deployment_name: <azure_model_deployment_name>
  # tokens_per_minute: 150_000 # set a leaky bucket throttle
  # requests_per_minute: 10_000 # set a leaky bucket throttle
  # max_retries: 10
  # max_retry_wait: 10.0
  # sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
  # concurrent_requests: 25 # the number of parallel inflight requests that may be made

parallelization:
  stagger: 0.3
  # num_threads: 50 # the number of threads to use for parallel processing

async_mode: threaded # or asyncio

embeddings:
  ## parallelization: override the global parallelization settings for embeddings
  async_mode: threaded # or asyncio
  llm:
    api_key: ${GRAPHRAG_API_KEY}
    type: openai_embedding # or azure_openai_embedding
    model: nomic_embed_text
    api_base: http://localhost:11434/api
    # api_version: 2024-02-15-preview
    # organization: <organization_id>
    # deployment_name: <azure_model_deployment_name>
    # tokens_per_minute: 150_000 # set a leaky bucket throttle
    # requests_per_minute: 10_000 # set a leaky bucket throttle
    # max_retries: 10
    # max_retry_wait: 10.0
    # sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
    # concurrent_requests: 25 # the number of parallel inflight requests that may be made
    # batch_size: 16 # the number of documents to send in a single request
    # batch_max_tokens: 8191 # the maximum number of tokens to send in a single request
    # target: required # or optional

chunks:
  size: 300
  overlap: 100
  group_by_columns: [id] # by default, we don't allow chunks to cross documents

input:
  type: file # or blob
  file_type: text # or csv
  base_dir: "input"
  file_encoding: utf-8
  file_pattern: ".*\\.txt$"

cache:
  type: file # or blob
  base_dir: "cache"
  # connection_string: <azure_blob_storage_connection_string>
  # container_name: <azure_blob_storage_container_name>

storage:
  type: file # or blob
  base_dir: "output/${timestamp}/artifacts"
  # connection_string: <azure_blob_storage_connection_string>
  # container_name: <azure_blob_storage_container_name>

reporting:
  type: file # or console, blob
  base_dir: "output/${timestamp}/reports"
  # connection_string: <azure_blob_storage_connection_string>
  # container_name: <azure_blob_storage_container_name>

entity_extraction:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  prompt: "prompts/entity_extraction.txt"
  entity_types: [organization,person,geo,event]
  max_gleanings: 0

summarize_descriptions:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  prompt: "prompts/summarize_descriptions.txt"
  max_length: 500

claim_extraction:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  # enabled: true
  prompt: "prompts/claim_extraction.txt"
  description: "Any claims or facts that could be relevant to information discovery."
  max_gleanings: 0

community_report:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  prompt: "prompts/community_report.txt"
  max_length: 2000
  max_input_length: 8000

cluster_graph:
  max_cluster_size: 10

embed_graph:
  enabled: false # if true, will generate node2vec embeddings for nodes
  # num_walks: 10
  # walk_length: 40
  # window_size: 2
  # iterations: 3
  # random_seed: 597832

umap:
  enabled: false # if true, will generate UMAP embeddings for nodes

snapshots:
  graphml: yes
  raw_entities: yes
  top_level_nodes: yes

local_search:
  # text_unit_prop: 0.5
  # community_prop: 0.1
  # conversation_history_max_turns: 5
  # top_k_mapped_entities: 10
  # top_k_relationships: 10
  # max_tokens: 12000

global_search:
  # max_tokens: 12000
  # data_max_tokens: 12000
  # map_max_tokens: 1000
  # reduce_max_tokens: 2000
  # concurrency: 32
TheAiSingularity commented 3 weeks ago

I just change the model to llama3

encoding_model: cl100k_base
skip_workflows: []
llm:
  api_key: ${GRAPHRAG_API_KEY}
  type: openai_chat # or azure_openai_chat
  model: llama3
  model_supports_json: true # recommended if this is available for your model.
  # max_tokens: 4000
  # request_timeout: 180.0
  api_base: http://localhost:11434/v1
  # api_version: 2024-02-15-preview
  # organization: <organization_id>
  # deployment_name: <azure_model_deployment_name>
  # tokens_per_minute: 150_000 # set a leaky bucket throttle
  # requests_per_minute: 10_000 # set a leaky bucket throttle
  # max_retries: 10
  # max_retry_wait: 10.0
  # sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
  # concurrent_requests: 25 # the number of parallel inflight requests that may be made

parallelization:
  stagger: 0.3
  # num_threads: 50 # the number of threads to use for parallel processing

async_mode: threaded # or asyncio

embeddings:
  ## parallelization: override the global parallelization settings for embeddings
  async_mode: threaded # or asyncio
  llm:
    api_key: ${GRAPHRAG_API_KEY}
    type: openai_embedding # or azure_openai_embedding
    model: nomic_embed_text
    api_base: http://localhost:11434/api
    # api_version: 2024-02-15-preview
    # organization: <organization_id>
    # deployment_name: <azure_model_deployment_name>
    # tokens_per_minute: 150_000 # set a leaky bucket throttle
    # requests_per_minute: 10_000 # set a leaky bucket throttle
    # max_retries: 10
    # max_retry_wait: 10.0
    # sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
    # concurrent_requests: 25 # the number of parallel inflight requests that may be made
    # batch_size: 16 # the number of documents to send in a single request
    # batch_max_tokens: 8191 # the maximum number of tokens to send in a single request
    # target: required # or optional

chunks:
  size: 300
  overlap: 100
  group_by_columns: [id] # by default, we don't allow chunks to cross documents

input:
  type: file # or blob
  file_type: text # or csv
  base_dir: "input"
  file_encoding: utf-8
  file_pattern: ".*\\.txt$"

cache:
  type: file # or blob
  base_dir: "cache"
  # connection_string: <azure_blob_storage_connection_string>
  # container_name: <azure_blob_storage_container_name>

storage:
  type: file # or blob
  base_dir: "output/${timestamp}/artifacts"
  # connection_string: <azure_blob_storage_connection_string>
  # container_name: <azure_blob_storage_container_name>

reporting:
  type: file # or console, blob
  base_dir: "output/${timestamp}/reports"
  # connection_string: <azure_blob_storage_connection_string>
  # container_name: <azure_blob_storage_container_name>

entity_extraction:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  prompt: "prompts/entity_extraction.txt"
  entity_types: [organization,person,geo,event]
  max_gleanings: 0

summarize_descriptions:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  prompt: "prompts/summarize_descriptions.txt"
  max_length: 500

claim_extraction:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  # enabled: true
  prompt: "prompts/claim_extraction.txt"
  description: "Any claims or facts that could be relevant to information discovery."
  max_gleanings: 0

community_report:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  prompt: "prompts/community_report.txt"
  max_length: 2000
  max_input_length: 8000

cluster_graph:
  max_cluster_size: 10

embed_graph:
  enabled: false # if true, will generate node2vec embeddings for nodes
  # num_walks: 10
  # walk_length: 40
  # window_size: 2
  # iterations: 3
  # random_seed: 597832

umap:
  enabled: false # if true, will generate UMAP embeddings for nodes

snapshots:
  graphml: yes
  raw_entities: yes
  top_level_nodes: yes

local_search:
  # text_unit_prop: 0.5
  # community_prop: 0.1
  # conversation_history_max_turns: 5
  # top_k_mapped_entities: 10
  # top_k_relationships: 10
  # max_tokens: 12000

global_search:
  # max_tokens: 12000
  # data_max_tokens: 12000
  # map_max_tokens: 1000
  # reduce_max_tokens: 2000
  # concurrency: 32

Updated the readme.md with new changes, it should work now.

python -m graphrag.query --root ./ragtest --method global "What is machinelearning?"

XinyuShe commented 3 weeks ago

I can't find the difference before and after. If you mean inputs to input,I corrected that the first time I run the command

TheAiSingularity commented 3 weeks ago

I can't find the difference before and after. If you mean inputs to input,I corrected that the first time I run the command

Please use this : python -m graphrag.query --root ./ragtest --method global "What is machinelearning?"

XinyuShe commented 3 weeks ago

emmmm，maybe something wrong with me this time

(graphrag-ollama-local) $ python -m graphrag.query --root ./ragtest --method global "What is machinelearning?"

INFO: Reading settings from ragtest/settings.yaml
creating llm client with {'api_key': 'REDACTED,len=4', 'type': "openai_chat", 'model': 'llama3', 'max_tokens': 4000, 'temperature': 0.0, 'top_p': 1.0, 'request_timeout': 180.0, 'api_base': 'http://localhost:11434/v1', 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': True, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}
Error parsing search response json
Traceback (most recent call last):
  File "xxx/graphrag-local-ollama/graphrag/query/structured_search/global_search/search.py", line 194, in _map_response_single_batch
    processed_response = self.parse_search_response(search_response)
  File "xxx/graphrag-local-ollama/graphrag/query/structured_search/global_search/search.py", line 232, in parse_search_response
    parsed_elements = json.loads(search_response)["points"]
  File "xxx/graphrag-ollama-local/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "xxx/graphrag-ollama-local/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "xxx/graphrag-ollama-local/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

SUCCESS: Global Search Response: I am sorry but I am unable to answer this question given the provided data.
TheAiSingularity commented 3 weeks ago

emmmm，maybe something wrong with me this time

(graphrag-ollama-local) $ python -m graphrag.query --root ./ragtest --method global "What is machinelearning?"

INFO: Reading settings from ragtest/settings.yaml
creating llm client with {'api_key': 'REDACTED,len=4', 'type': "openai_chat", 'model': 'llama3', 'max_tokens': 4000, 'temperature': 0.0, 'top_p': 1.0, 'request_timeout': 180.0, 'api_base': 'http://localhost:11434/v1', 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': True, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}
Error parsing search response json
Traceback (most recent call last):
  File "xxx/graphrag-local-ollama/graphrag/query/structured_search/global_search/search.py", line 194, in _map_response_single_batch
    processed_response = self.parse_search_response(search_response)
  File "xxx/graphrag-local-ollama/graphrag/query/structured_search/global_search/search.py", line 232, in parse_search_response
    parsed_elements = json.loads(search_response)["points"]
  File "xxx/graphrag-ollama-local/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "xxx/graphrag-ollama-local/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "xxx/graphrag-ollama-local/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

SUCCESS: Global Search Response: I am sorry but I am unable to answer this question given the provided data.

I have ran the setup once again with the exact steps that i mentioned in readme.md and it works pretty fine for me and my team. 

(graphrag-ollama-local) graphrag-local-ollama$python -m graphrag.query --root ./ragtest --method global "What is machinelearning?"

INFO: Reading settings from ragtest/settings.yaml
creating llm client with {'api_key': 'REDACTED,len=9', 'type': "openai_chat", 'model': 'llama3', 'max_tokens': 4000, 'temperature': 0.0, 'top_p': 1.0, 'request_timeout': 180.0, 'api_base': 'http://localhost:11434/v1', 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': True, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}

SUCCESS: Global Search Response: **Machine Learning: A Comprehensive Overview**

According to Analyst 1, machine learning is a subfield of artificial intelligence that involves training algorithms to learn from data and make predictions or decisions without being explicitly programmed. It's a type of self-improving system that can analyze patterns, relationships, and trends in data to improve its performance over time [Data: Reports (5)].

**Key Characteristics**

Machine learning is characterized by its ability to:

* Learn from data without being explicitly programmed
* Improve its performance over time through self-analysis and adaptation
* Analyze complex patterns, relationships, and trends in data

**Applications**

Machine learning has numerous applications across various industries, including:

* **Predictive Analytics**: Machine learning can be used for predictive analytics, such as forecasting sales or predicting customer churn.
* **Natural Language Processing**: Machine learning can be applied to natural language processing tasks, such as sentiment analysis or text classification.
* **Computer Vision**: Machine learning can be used for computer vision tasks, such as object detection or facial recognition.

**Data References**

Some relevant data references include:

* Reports (5) - Provides an overview of machine learning and its applications
* +more - Additional data references may provide further insights into specific machine learning techniques or applications
sunil-thapa99 commented 3 weeks ago

I changed openai_embedding_llm.py file in graphrag lib to: `from typing_extensions import Unpack from graphrag.llm.base import BaseLLM from graphrag.llm.types import ( EmbeddingInput, EmbeddingOutput, LLMInput, ) from .openai_configuration import OpenAIConfiguration from .types import OpenAIClientTypes import ollama

class OpenAIEmbeddingsLLM(BaseLLM[EmbeddingInput, EmbeddingOutput]): _client: OpenAIClientTypes _configuration: OpenAIConfiguration

def __init__(self, client: OpenAIClientTypes, configuration: OpenAIConfiguration):
    self._client = client
    self._configuration = configuration

async def _execute_llm(
    self, input: EmbeddingInput, **kwargs: Unpack[LLMInput]
) -> EmbeddingOutput | None:
    args = {
        "model": self._configuration.model,
        **(kwargs.get("model_parameters") or {}),
    }
    embedding_list = []
    for inp in input:
        embedding = ollama.embeddings(model="nomic-embed-text", prompt=inp)
        embedding_list.append(embedding["embedding"])
    return embedding_list`

This helped me use Ollama models instead of calling OpenAI API.

My output: `python -m graphrag.query --root ./ragtest --method global "What is machinelearning?"

INFO: Reading settings from ragtest/settings.yaml creating llm client with {'api_key': 'REDACTED,len=9', 'type': "openai_chat", 'model': 'llama3', 'max_tokens': 4000, 'request_timeout': 180.0, 'api_base': 'http://localhost:11434/v1', 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': True, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}

SUCCESS: Global Search Response: Machine Learning: A Comprehensive Overview

Machine learning refers to the application of artificial intelligence (AI) that enables computers to learn from data without being explicitly programmed. It involves training algorithms on large datasets, allowing them to identify patterns and make predictions or decisions based on that information [Data: Reports (2)].

According to analysts, machine learning has become a crucial aspect of modern computing, with its potential applications spanning various fields, including natural language processing, computer vision, and predictive analytics. Researchers are working on improving Convolutional Neural Networks (CNNs) to address existing limitations and drive future advancements [Data: Entities (9); Relationships (11)].

Key Takeaways

Data References

kochunchung commented 3 weeks ago

it work for me thx!

I changed openai_embedding_llm.py file in graphrag lib to: `from typing_extensions import Unpack from graphrag.llm.base import BaseLLM from graphrag.llm.types import ( EmbeddingInput, EmbeddingOutput, LLMInput, ) from .openai_configuration import OpenAIConfiguration from .types import OpenAIClientTypes import ollama

class OpenAIEmbeddingsLLM(BaseLLM[EmbeddingInput, EmbeddingOutput]): _client: OpenAIClientTypes _configuration: OpenAIConfiguration

def __init__(self, client: OpenAIClientTypes, configuration: OpenAIConfiguration):
    self._client = client
    self._configuration = configuration

async def _execute_llm(
    self, input: EmbeddingInput, **kwargs: Unpack[LLMInput]
) -> EmbeddingOutput | None:
    args = {
        "model": self._configuration.model,
        **(kwargs.get("model_parameters") or {}),
    }
    embedding_list = []
    for inp in input:
        embedding = ollama.embeddings(model="nomic-embed-text", prompt=inp)
        embedding_list.append(embedding["embedding"])
    return embedding_list`

This helped me use Ollama models instead of calling OpenAI API.

My output: `python -m graphrag.query --root ./ragtest --method global "What is machinelearning?"

INFO: Reading settings from ragtest/settings.yaml creating llm client with {'api_key': 'REDACTED,len=9', 'type': "openai_chat", 'model': 'llama3', 'max_tokens': 4000, 'request_timeout': 180.0, 'api_base': 'http://localhost:11434/v1', 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': True, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}

SUCCESS: Global Search Response: Machine Learning: A Comprehensive Overview

Machine learning refers to the application of artificial intelligence (AI) that enables computers to learn from data without being explicitly programmed. It involves training algorithms on large datasets, allowing them to identify patterns and make predictions or decisions based on that information [Data: Reports (2)].

According to analysts, machine learning has become a crucial aspect of modern computing, with its potential applications spanning various fields, including natural language processing, computer vision, and predictive analytics. Researchers are working on improving Convolutional Neural Networks (CNNs) to address existing limitations and drive future advancements [Data: Entities (9); Relationships (11)].

Key Takeaways

  • Machine learning is a subfield of AI that enables computers to learn from data without explicit programming.
  • It involves training algorithms on large datasets, allowing them to identify patterns and make predictions or decisions.
  • Applications of machine learning include natural language processing, computer vision, and predictive analytics.
  • Researchers are working on improving CNNs to drive future advancements in the field.

Data References

  • Reports (2)
  • Entities (9)
  • Relationships (11)`
sunil-thapa99 commented 3 weeks ago

it work for me thx!

I changed openai_embedding_llm.py file in graphrag lib to: `from typing_extensions import Unpack from graphrag.llm.base import BaseLLM from graphrag.llm.types import ( EmbeddingInput, EmbeddingOutput, LLMInput, ) from .openai_configuration import OpenAIConfiguration from .types import OpenAIClientTypes import ollama class OpenAIEmbeddingsLLM(BaseLLM[EmbeddingInput, EmbeddingOutput]): _client: OpenAIClientTypes _configuration: OpenAIConfiguration

def __init__(self, client: OpenAIClientTypes, configuration: OpenAIConfiguration):
    self._client = client
    self._configuration = configuration

async def _execute_llm(
    self, input: EmbeddingInput, **kwargs: Unpack[LLMInput]
) -> EmbeddingOutput | None:
    args = {
        "model": self._configuration.model,
        **(kwargs.get("model_parameters") or {}),
    }
    embedding_list = []
    for inp in input:
        embedding = ollama.embeddings(model="nomic-embed-text", prompt=inp)
        embedding_list.append(embedding["embedding"])
    return embedding_list`

This helped me use Ollama models instead of calling OpenAI API. My output: `python -m graphrag.query --root ./ragtest --method global "What is machinelearning?" INFO: Reading settings from ragtest/settings.yaml creating llm client with {'api_key': 'REDACTED,len=9', 'type': "openai_chat", 'model': 'llama3', 'max_tokens': 4000, 'request_timeout': 180.0, 'api_base': 'http://localhost:11434/v1', 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': True, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25} SUCCESS: Global Search Response: Machine Learning: A Comprehensive Overview Machine learning refers to the application of artificial intelligence (AI) that enables computers to learn from data without being explicitly programmed. It involves training algorithms on large datasets, allowing them to identify patterns and make predictions or decisions based on that information [Data: Reports (2)]. According to analysts, machine learning has become a crucial aspect of modern computing, with its potential applications spanning various fields, including natural language processing, computer vision, and predictive analytics. Researchers are working on improving Convolutional Neural Networks (CNNs) to address existing limitations and drive future advancements [Data: Entities (9); Relationships (11)]. Key Takeaways

  • Machine learning is a subfield of AI that enables computers to learn from data without explicit programming.
  • It involves training algorithms on large datasets, allowing them to identify patterns and make predictions or decisions.
  • Applications of machine learning include natural language processing, computer vision, and predictive analytics.
  • Researchers are working on improving CNNs to drive future advancements in the field.

Data References

  • Reports (2)
  • Entities (9)
  • Relationships (11)`

For me it doesn't work on --method local. This only worked on global. Did you tried running on local method?

kochunchung commented 3 weeks ago

it work for me thx!

I changed openai_embedding_llm.py file in graphrag lib to: `from typing_extensions import Unpack from graphrag.llm.base import BaseLLM from graphrag.llm.types import ( EmbeddingInput, EmbeddingOutput, LLMInput, ) from .openai_configuration import OpenAIConfiguration from .types import OpenAIClientTypes import ollama class OpenAIEmbeddingsLLM(BaseLLM[EmbeddingInput, EmbeddingOutput]): _client: OpenAIClientTypes _configuration: OpenAIConfiguration

def __init__(self, client: OpenAIClientTypes, configuration: OpenAIConfiguration):
    self._client = client
    self._configuration = configuration

async def _execute_llm(
    self, input: EmbeddingInput, **kwargs: Unpack[LLMInput]
) -> EmbeddingOutput | None:
    args = {
        "model": self._configuration.model,
        **(kwargs.get("model_parameters") or {}),
    }
    embedding_list = []
    for inp in input:
        embedding = ollama.embeddings(model="nomic-embed-text", prompt=inp)
        embedding_list.append(embedding["embedding"])
    return embedding_list`

This helped me use Ollama models instead of calling OpenAI API. My output: `python -m graphrag.query --root ./ragtest --method global "What is machinelearning?" INFO: Reading settings from ragtest/settings.yaml creating llm client with {'api_key': 'REDACTED,len=9', 'type': "openai_chat", 'model': 'llama3', 'max_tokens': 4000, 'request_timeout': 180.0, 'api_base': 'http://localhost:11434/v1', 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': True, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25} SUCCESS: Global Search Response: Machine Learning: A Comprehensive Overview Machine learning refers to the application of artificial intelligence (AI) that enables computers to learn from data without being explicitly programmed. It involves training algorithms on large datasets, allowing them to identify patterns and make predictions or decisions based on that information [Data: Reports (2)]. According to analysts, machine learning has become a crucial aspect of modern computing, with its potential applications spanning various fields, including natural language processing, computer vision, and predictive analytics. Researchers are working on improving Convolutional Neural Networks (CNNs) to address existing limitations and drive future advancements [Data: Entities (9); Relationships (11)]. Key Takeaways

  • Machine learning is a subfield of AI that enables computers to learn from data without explicit programming.
  • It involves training algorithms on large datasets, allowing them to identify patterns and make predictions or decisions.
  • Applications of machine learning include natural language processing, computer vision, and predictive analytics.
  • Researchers are working on improving CNNs to drive future advancements in the field.

Data References

  • Reports (2)
  • Entities (9)
  • Relationships (11)`

For me it doesn't work on --method local. This only worked on global. Did you tried running on local method?

I think the '--method local' command has some bugs. You can learn from this article how to modify it, then execute '--method global'. It will use the local model. I have tried it in an environment without setting the OpenAI key.

sunil-thapa99 commented 3 weeks ago

it work for me thx!

I changed openai_embedding_llm.py file in graphrag lib to: `from typing_extensions import Unpack from graphrag.llm.base import BaseLLM from graphrag.llm.types import ( EmbeddingInput, EmbeddingOutput, LLMInput, ) from .openai_configuration import OpenAIConfiguration from .types import OpenAIClientTypes import ollama class OpenAIEmbeddingsLLM(BaseLLM[EmbeddingInput, EmbeddingOutput]): _client: OpenAIClientTypes _configuration: OpenAIConfiguration

def __init__(self, client: OpenAIClientTypes, configuration: OpenAIConfiguration):
    self._client = client
    self._configuration = configuration

async def _execute_llm(
    self, input: EmbeddingInput, **kwargs: Unpack[LLMInput]
) -> EmbeddingOutput | None:
    args = {
        "model": self._configuration.model,
        **(kwargs.get("model_parameters") or {}),
    }
    embedding_list = []
    for inp in input:
        embedding = ollama.embeddings(model="nomic-embed-text", prompt=inp)
        embedding_list.append(embedding["embedding"])
    return embedding_list`

This helped me use Ollama models instead of calling OpenAI API. My output: `python -m graphrag.query --root ./ragtest --method global "What is machinelearning?" INFO: Reading settings from ragtest/settings.yaml creating llm client with {'api_key': 'REDACTED,len=9', 'type': "openai_chat", 'model': 'llama3', 'max_tokens': 4000, 'request_timeout': 180.0, 'api_base': 'http://localhost:11434/v1', 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': True, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25} SUCCESS: Global Search Response: Machine Learning: A Comprehensive Overview Machine learning refers to the application of artificial intelligence (AI) that enables computers to learn from data without being explicitly programmed. It involves training algorithms on large datasets, allowing them to identify patterns and make predictions or decisions based on that information [Data: Reports (2)]. According to analysts, machine learning has become a crucial aspect of modern computing, with its potential applications spanning various fields, including natural language processing, computer vision, and predictive analytics. Researchers are working on improving Convolutional Neural Networks (CNNs) to address existing limitations and drive future advancements [Data: Entities (9); Relationships (11)]. Key Takeaways

  • Machine learning is a subfield of AI that enables computers to learn from data without explicit programming.
  • It involves training algorithms on large datasets, allowing them to identify patterns and make predictions or decisions.
  • Applications of machine learning include natural language processing, computer vision, and predictive analytics.
  • Researchers are working on improving CNNs to drive future advancements in the field.

Data References

  • Reports (2)
  • Entities (9)
  • Relationships (11)`

For me it doesn't work on --method local. This only worked on global. Did you tried running on local method?

I think the '--method local' command has some bugs. You can learn from this article how to modify it, then execute '--method global'. It will use the local model. I have tried it in an environment without setting the OpenAI key.

Can you provide the article link?