search

SciPhi-AI / R2R

The all-in-one solution for RAG. Build, scale, and deploy state of the art Retrieval-Augmented Generation applications
https://r2r-docs.sciphi.ai/
MIT License
3.51k stars 263 forks source link

How to point llama host besides the local one ? #1047

Closed MyraBaba closed 1 month ago

MyraBaba commented 1 month ago

Hi ,

How to use ollama which running in different host ?

Best

NolanTrem commented 1 month ago

You can set your OLLAMA_API_BASE variable. By default, this is set to http://host.docker.internal:11434 for docker and http://127.0.0.1:11434 for local deployments of Ollama.

Was the other issue that you had open resolved?

MyraBaba commented 1 month ago

@NolanTrem

1080 is partly resolved the docker side. the 1003 is resolved.

But the login problem still exist even I removed the postresql volume from docker and test again.

I am running the r2r at 192.168.5.110 and opening the wen client from. x.x.x.100 same subnet . changing the server ip to : 192.168.5.110

Screen Shot 2024-09-04 at 22 39 19
NolanTrem commented 1 month ago

Can you check if there are any requests hitting your R2R server? The server logs should tell us if the login request is reaching it, or what the exact issue is.

MyraBaba commented 1 month ago

@NolanTrem additional info:

if rune with :

r2r serve --docker

I can login system and able to use it.

if start with : r2r serve --config-path=core/config/local_llm_neo4j_kg --docker

I cant login as above error.

NolanTrem commented 1 month ago

There is a bug in the local_llm_neo4j_kg that we have in our dev branch. The issue is likely that the server is not successfully starting, so the login request doesn't get handled correctly. Can you try to use this one? Note that batch_size was missing previously.

[completion]
provider = "litellm"
concurrent_request_limit = 1

  [completion.generation_config]
  model = "ollama/llama3.1"
  temperature = 0.1
  top_p = 1
  max_tokens_to_sample = 1_024
  stream = false
  add_generation_kwargs = { }

[embedding]
provider = "ollama"
base_model = "mxbai-embed-large"
base_dimension = 1_024
batch_size = 32
add_title_as_prefix = true

[parsing]
excluded_parsers = [ "gif", "jpeg", "jpg", "png", "svg", "mp3", "mp4" ]

[kg]
provider = "neo4j"
batch_size = 64
kg_extraction_prompt = "graphrag_triplet_extraction_zero_shot"

  [kg.kg_extraction_config]
  model = "ollama/llama3.1"
  temperature = 1
  top_p = 1
  max_tokens_to_sample = 1_024
  stream = false
  add_generation_kwargs = { }

  [kg.kg_creation_settings]
    generation_config = { model = "ollama/llama3.1" } # and other params, model used for triplet extraction

  [kg.kg_enrichment_settings]
    max_knowledge_triples = 100
    generation_config = { model = "ollama/llama3.1" } # and other params, model used for node description and graph clustering
    leiden_params = { max_cluster_size = 1000 } # more params

  [kg.kg_search_config]
    model = "ollama/llama3.1"

[database]
provider = "postgres"

[agent]
system_instruction_name = "rag_agent"
tool_names = ["search"]

  [agent.generation_config]
  model = "ollama/llama3.1"
MyraBaba commented 1 month ago

@NolanTrem

tested above one with remote ollama:

logs:

2024-09-04 19:59:31,810 - INFO - core.main.app_entry - Environment CONFIG_NAME: 2024-09-04 19:59:31,810 - INFO - core.main.app_entry - Environment CONFIG_PATH: /home/bc/Projects/ODS/R2R/py/core/configs/xlocal_llm_neo4j_kg.toml 2024-09-04 19:59:31,810 - INFO - core.main.app_entry - Environment PIPELINE_TYPE: qna 2024-09-04 19:59:31,815 - INFO - core.main.r2r - Starting R2R with version 3.0.8 2024-09-04 19:59:31,815 - INFO - core.providers.prompts.r2r_prompts - Loading prompts from /app/core/providers/prompts/defaults 2024-09-04 19:59:31,851 - INFO - core.base.providers.prompt - Initializing PromptProvider with config extra_fields={} provider='r2r' default_system_name='default_system' default_task_name='default_rag' file_path=None. 2024-09-04 19:59:31,851 - INFO - core.base.providers.embedding - Initializing EmbeddingProvider with config extra_fields={} provider='ollama' base_model='mxbai-embed-large' base_dimension=1024 rerank_model=None rerank_dimension=None rerank_transformer_type=None batch_size=32 prefixes=None add_title_as_prefix=True concurrent_request_limit=256 max_retries=2 initial_backoff=1.0 max_backoff=60.0.

g_enrichment_settings=KGEnrichmentSettings(max_knowledge_triples=100, generation_config_triplet=GenerationConfig(model='ollama/llama3.1', temperature=0.1, top_p=1.0, max_tokens_to_sample=1024, stream=False, functions=None, tools=None, add_generation_kwargs={}, api_base=None), generation_config_enrichment=GenerationConfig(model='ollama/llama3.1', temperature=0.1, top_p=1.0, max_tokens_to_sample=1024, stream=False, functions=None, tools=None, add_generation_kwargs={}, api_base=None), leiden_params={'max_cluster_size': 1000}) 2024-09-04 19:59:32,908 - INFO - core.base.providers.database - Initializing DatabaseProvider with config extra_fields={} provider='postgres'. 2024-09-04 19:59:32,908 - INFO - core.providers.database.vector - Using TCP connection Traceback (most recent call last): File "/usr/local/bin/uvicorn", line 8, in sys.exit(main()) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1157, in call return self.main(args, kwargs) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 783, in invoke return __callback(args, **kwargs) File "/usr/local/lib/python3.10/site-packages/uvicorn/main.py", line 418, in main run( File "/usr/local/lib/python3.10/site-packages/uvicorn/main.py", line 587, in run server.run() File "/usr/local/lib/python3.10/site-packages/uvicorn/server.py", line 62, in run return asyncio.run(self.serve(sockets=sockets)) File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete return future.result() File "/usr/local/lib/python3.10/site-packages/uvicorn/server.py", line 69, in serve config.load() File "/usr/local/lib/python3.10/site-packages/uvicorn/config.py", line 458, in load self.loaded_app = import_from_string(self.app) File "/usr/local/lib/python3.10/site-packages/uvicorn/importer.py", line 21, in import_from_string module = importlib.import_module(module_str) File "/usr/local/lib/python3.10/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_modul File "/app/core/main/assembly/factory.py", line 277, in create_providers or self.create_database_provider( File "/app/core/main/assembly/factory.py", line 143, in create_database_provider database_provider = PostgresDBProvider( File "/app/core/providers/database/postgres.py", line 93, in init super().init(config) File "/app/core/base/providers/database.py", line 46, in init self.vector: VectorDBProvider = self._initialize_vector_db() File "/app/core/providers/database/postgres.py", line 96, in _initialize_vector_db return PostgresVectorDBProvider( File "/app/core/providers/database/vector.py", line 119, in init self._initialize_vector_db(dimension) File "/app/core/providers/database/vector.py", line 131, in _initialize_vector_db self.collection = self.vx.get_or_create_collection( File "/app/core/providers/database/vecs/client.py", line 182, in get_or_create_collection return collection._create_if_not_exists() File "/app/core/providers/database/vecs/collection.py", line 331, in _create_if_not_exists raise MismatchedDimension( core.providers.database.vecs.exc.MismatchedDimension: Mismatch in the reported dimensions of the selected vector collection and embedding model. Correct the selected embedding model or specify a new vector collection by modifying the POSTGRES_VECS_COLLECTION environment variable. 2024-09-04 19:59:38,043 - INFO - core.main.app_entry - Environment CONFIG_NAME: 2024-09-04 19:59:38,044 - INFO - core.main.app_entry - Environment CONFIG_PATH: /home/bc/Projects/ODS/R2R/py/core/configs/xlocal_llm_neo4j_kg.toml 2024-09-04 19:59:38,044 - INFO - core.main.app_entry - Environment PIPELINE_TYPE: qna 2024-09-04 19:59:38,048 - INFO - core.main.r2r - Starting R2R with version 3.0.8 2024-09-04 19:59:38,048 - INFO - core.providers.prompts.r2r_prompts - Loading prompts from /app/core/providers/prompts/defaults 2024-09-04 19:59:38,082 - INFO - core.base.providers.prompt - Initializing PromptProvider with config extra_fields={} provider='r2r' default_system_name='default_system' default_task_name='default_rag' file_path=None. 2024-09-04 19:59:38,082 - INFO - core.base.providers.embedding - Initializing EmbeddingProvider with config extra_fields={} provider='ollama' base_model='mxbai-embed-large' base_dimension=1024 rerank_model=None rerank_dimension=None rerank_transformer_type=None batch_size=32 prefixes=None add_title_as_prefix=True concurrent_request_limit=256 max_retries=2 initial_backoff=1.0 max_backoff=60.0. 2024-09-04 19:59:38,082 - INFO - core.providers.embeddings.ollama - Using Ollama API base URL: http://10.144.86.53:11434 2024-09-04 19:59:38,134 - INFO - core.base.providers.llm - Initializing CompletionProvider with config: extra_fields={'concurrent_request_limit': 1} provider='litellm' generation_config=GenerationConfig(model='ollama/llama3.1', temperature=0.1, top_p=1.0, max_tokens_to_sample=1024, stream=False, functions=None, tools=None, add_generation_kwargs={}, api_base=None) concurrency_limit=16 max_retries=2 initial_backoff=1.0 max_backoff=60.0 2024-09-04 19:59:38,351 - INFO - core.base.providers.kg - Initializing KG provider with config: extra_fields={'kg_extraction_config': {'model': 'ollama/llama3.1', 'temperature': 1, 'top_p': 1, 'max_tokens_to_sample': 1024, 'stream': False, 'add_generation_kwargs': {}}, 'kg_creation_settings': {'generation_config': {'model': 'ollama/llama3.1'}}} provider='neo4j' batch_size=64 kg_extraction_prompt='graphrag_triplet_extraction_zero_shot' kg_search_prompt='kg_search' kg_search_config=GenerationConfig run( File "/usr/local/lib/python3.10/site-packages/uvicorn/main.py", line 587, in run server.run() File "/usr/local/lib/python3.10/site-packages/uvicorn/server.py", line 62, in run return asyncio.run(self.serve(sockets=sockets)) File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete return future.result() File "/usr/local/lib/python3.10/site-packages/uvicorn/server.py", line 69, in serve config.load() File "/usr/local/lib/python3.10/site-packages/uvicorn/config.py", line 458, in load self.loaded_app = import_from_string(self.app) File "/usr/local/lib/python3.10/site-packages/uvicorn/importer.py", line 21, in import_from_string module = importlib.import_module(module_str) File "/usr/local/lib/python3.10/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "/app/core/main/app_entry.py", line 68, in app = r2r_app( File "/app/core/main/app_entry.py", line 48, in r2r_app r2r_deployment = R2R(config=config) File "/app/core/main/r2r.py", line 52, in init built = builder.build() File "/app/core/main/assembly/builder.py", line 239, in build providers = provider_factory(self.config).create_providers( File "/app/core/main/assembly/factory.py", line 277, in create_providers or self.create_database_provider( File "/app/core/main/assembly/factory.py", line 143, in create_database_provider database_provider = PostgresDBProvider( File "/app/core/providers/database/postgres.py", line 93, in init super().init(config) File "/app/core/base/providers/database.py", line 46, in init self.vector: VectorDBProvider = self._initialize_vector_db() File "/app/core/providers/database/postgres.py", line 96, in _initialize_vector_db return Postgre File "/app/core/providers/database/vector.py", line 131, in _initialize_vector_db self.collection = self.vx.get_or_create_collection( File "/app/core/providers/database/vecs/client.py", line 182, in get_or_create_collection return collection._create_if_not_exists() File "/app/core/providers/database/vecs/collection.py", line 331, in _create_if_not_exists raise MismatchedDimension( core.providers.database.vecs.exc.MismatchedDimension: Mismatch in the reported dimensions of the selected vector collection and embedding model. Correct the selected embedding model or specify a new vector collection by modifying the POSTGRES_VECS_COLLECTION environment variable. 2024-09-04 19:59:43,660 - INFO - core.main.app_entry - Environment CONFIG_NAME: 2024-09-04 19:59:43,660 - INFO - core.main.app_entry - Environment CONFIG_PATH: /home/bc/Projects/ODS/R2R/py/core/configs/xlocal_llm_neo4j_kg.toml 2024-09-04 19:59:43,660 - INFO - core.main.app_entry - Environment PIPELINE_TYPE: qna 2024-09-04 19:59:43,665 - INFO - core.main.r2r - Starting R2R with version 3.0.8 2024-09-04 19:59:43,665 - INFO - core.providers.prompts.r2r_prompts - Loading prompts from /app/core/providers/prompts/defaults 2024-09-04 19:59:43,700 - INFO - core.base.providers.prompt - Initializing PromptProvider with config extra_fields={} provider='r2r' default_system_name='default_system' default_task_name='default_rag' file_path=None. 2024-09-04 19:59:43,700 - INFO - core.base.providers.embedding - Initializing EmbeddingProvider with config extra_fields={} provider='ollama' base_model='mxbai-embed-large' base_dimension=1024 rerank_model=None rerank_dimension=None rerank_transformer_type=None batch_size=32 prefixes=None add_title_as_prefix=True concurrent_request_limit=256 max_retries=2 initial_backoff=1.0 max_backoff=60.0. 2024-09-04 19:59:43,700 - INFO - core.providers.embeddings.ollama - Using Ollama API base URL: http://10.144.86.53:11434 2024-09-04 19:59:43,753 - INFO - core.base.providers.llm - Initializing CompletionProvider with config: extra_fields={'concurrent_request_limit': 1} provider='litellm' generation_config=GenerationConfig(model='ollama/llama3.1', temperature=0.1, top_p=1.0, max_tokens_to_sample=1024, stream=False, functions=None, tools=None, add_generation_kwargs={}, api_base=None) concurrency_limit=16 max_retries=2 initial_backoff=1.0 max_backoff=60.0 2024-09-04 19:59:43,973 - INFO - core.base.providers.kg - Initializing KG provider with config: extra_fields={'kg_extraction_config': {'model': 'ollama/llama3.1', 'temperature': 1, 'top_p': 1, 'max_tokens_to_sample': 1024, 'stream': False, 'add_generation_kwargs': {}}, 'kg_creation_settings': {'generation_config': {'model': 'ollama/llama3.1'}}} provider='neo4j' batch_size=64 kg_extraction_prompt='graphrag_triplet_extraction_zero_shot' kg_search_prompt='kg_search' kg_search_config=GenerationConfig(model='ollama/llama3.1', temperature=0.1, top_p=1.0, max_tokens_to_sample=1024, stream=False, functions=None, tools=None, add_generation_kwargs={}, api_base=None) kg_store_path=None kg_enrichment_settings=KGEnrichmentSettings(max_knowledge_triples=100, generation_config_triplet=GenerationConfig(model='ollama/llama3.1', temperature=0.1, top_p=1.0, max_tokens_to_sample=1024, stream=False, functions=None, tools=None, add_generation_kwargs={}, api_base=None), generation_config_enrichment=GenerationConfig(model='ollama/llama3.1', temperature=0.1, top_p=1.0, max_tokens_to_sample=1024, stream=False, functions=None, tools=None, add_generation_kwargs={}, api_base=None), leiden_params={'max_cluster_size': 1000}) 2024-09-04 19:59:43,987 - INFO - core.base.providers.database - Initializing DatabaseProvider with config extra_fields={} provider='postgres'. 2024-09-04 19:59:43,987 - INFO - core.providers.database.vector - Using TCP connection Traceback (most recent call last): File "/usr/local/bin/uvicorn", line 8, in sys.exit(main()) Fi self.loaded_app = import_from_string(self.app) File "/usr/local/lib/python3.10/site-packages/uvicorn/importer.py", line 21, in import_from_string module = importlib.import_module(module_str) File "/usr/local/lib/python3.10/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "/app/core/main/app_entry.py", line 68, in app = r2r_app( File "/app/core/main/app_entry.py", line 48, in r2r_app r2r_deployment = R2R(config=config) File "/app/core/main/r2r.py", line 52, in init built = builder.build() File "/app/core/main/assembly/builder.py", line 239, in build providers = provider_factory(self.config).create_providers( File "/app/core/main/assembly/factory.py", line 277, in create_providers or self.create_database_provider( File "/app/core/main/assembly/factory.py", line 143, in create_database_provider database_provider = PostgresDBProvider( File "/app/core/providers/database/postgres.py", line 93, in init super().init(config) File "/app/core/base/providers/database.py", line 46, in init self.vector: VectorDBProvider = self._initialize_vector_db() File "/app/core/providers/database/postgres.py", line 96, in _initialize_vector_db return PostgresVectorDBProvider( File "/app/core/providers/database/vector.py", line 119, in init self._initialize_vector_db(dimension) File "/app/core/providers/database/vector.py", line 131, in _initialize_vector_db self.collection = self.vx.get_or_create_collection( File "/app/core/providers/database/vecs/client.py", line 182, in get_or_create_collection return collection._create_if_not_exists() File "/app/core/providers/database/vecs/collection.py", line 331, in _create_if_not_exists raise MismatchedDimension( core.providers.database.vecs.exc.MismatchedDimension: Mismatch in the reported dimensions of the selected vector collection and embedding model. Correct the selected embedding model or specify a new vector collection by modifying the POSTGRES_VECS_COLLECTION environment variable.

NolanTrem commented 1 month ago

It looks like your vector database was initialized with another embedding model at some point. Either delete the docker Postgres volume to ensure that you have no existing db, or set a new POSTGRES_VECS_COLLECTION environment variable run again. It will flag that you don't need to set this by default and ask if you want to unset it if you do the later. Select no.

MyraBaba commented 1 month ago

@NolanTrem

I deleted volume and now I can login . Added document thant I have error:

INFO: 127.0.0.1:39422 - "GET /v2/health HTTP/1.1" 200 OK INFO: 127.0.0.1:58586 - "GET /v2/health HTTP/1.1" 200 OK INFO: 127.0.0.1:51762 - "GET /v2/health HTTP/1.1" 200 OK INFO: 127.0.0.1:45748 - "GET /v2/health HTTP/1.1" 200 OK INFO: 127.0.0.1:33382 - "GET /v2/health HTTP/1.1" 200 OK INFO: 127.0.0.1:57168 - "GET /v2/health HTTP/1.1" 200 OK INFO: 192.168.5.108:55870 - "POST /v2/login HTTP/1.1" 200 OK INFO: 192.168.5.108:55870 - "OPTIONS /v2/app_settings HTTP/1.1" 200 OK INFO: 192.168.5.108:55870 - "GET /v2/app_settings HTTP/1.1" 200 OK INFO: 192.168.5.108:55870 - "GET /v2/health HTTP/1.1" 200 OK INFO: 192.168.5.108:55870 - "GET /v2/health HTTP/1.1" 200 OK INFO: 192.168.5.108:55875 - "OPTIONS /v2/logs HTTP/1.1" 200 OK INFO: 192.168.5.108:55876 - "OPTIONS /v2/server_stats HTTP/1.1" 200 OK INFO: 192.168.5.108:55875 - "GET /v2/server_stats HTTP/1.1" 200 OK INFO: 192.168.5.108:55876 - "GET /v2/logs HTTP/1.1" 200 OK INFO: 192.168.5.108:55870 - "GET /v2/server_stats HTTP/1.1" 200 OK INFO: 127.0.0.1:37314 - "GET /v2/health HTTP/1.1" 200 OK INFO: 192.168.5.108:55917 - "OPTIONS /v2/documents_overview HTTP/1.1" 200 OK INFO: 192.168.5.108:55917 - "GET /v2/documents_overview HTTP/1.1" 200 OK INFO: 127.0.0.1:58184 - "GET /v2/health HTTP/1.1" 200 OK INFO: 192.168.5.108:55938 - "OPTIONS /v2/ingest_files HTTP/1.1" 200 OK INFO: 127.0.0.1:56538 - "GET /v2/health HTTP/1.1" 200 OK 2024-09-04 20:13:50,081 - INFO - core.main.api.routes.ingestion.base - No chunking config override provided. Using default. 2024-09-04 20:13:50,087 - INFO - core.main.services.ingestion_service - No chunking provider specified. Using default. 2024-09-04 20:13:54,386 - ERROR - core.providers.embeddings.ollama - Error getting embe 2024-09-04 20:13:54,387 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,387 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,387 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,387 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,389 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,389 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,389 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,389 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,390 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,390 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,390 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,390 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,391 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,391 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,391 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,391 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,392 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,392 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,392 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,392 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,393 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,393 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,394 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,394 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,395 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,395 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,395 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,395 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: 2024-09-04 20:13:54,397 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,397 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,399 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,399 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,400 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,400 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,400 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,400 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,401 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,401 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,402 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,402 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,402 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,402 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,403 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,403 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,403 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:54,404 - WARNING - core.base.providers.embedding - Request failed (attempt 1): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,416 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,416 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,417 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,417 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,418 - ERROR - core.base.pipeline.base_pipeline - Pipeline failed with error: Error getting embeddings: All connection attempts failed

Stack trace: Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 72, in map_httpcore_exceptions yield File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.p Traceback (most recent call last): File "/app/core/providers/embeddings/ollama.py", line 66, in _execute_task response = await self.aclient.embed( File "/usr/local/lib/python3.10/site-packages/ollama/_client.py", line 680, in embed response = await self._request( File "/usr/local/lib/python3.10/site-packages/ollama/_client.py", line 483, in _request response = await self._client.request(method, url, **kwargs) File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1585, in request return await self.send(request, auth=auth, follow_redirects=follow_redirects) File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1674, in send response = await self._send_handling_auth( File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1702, in _send_handling_auth response = await self._send_handling_redirects( File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1739, in _send_handling_redirects response = await self._send_single_request(request) File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1776, in _send_single_request response = await transport.handle_async_request(request) File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 376, in handle_async_request with map_httpcore_exceptions(): File "/usr/local/lib/python3.10/contextlib.py", line 153, in exit self.gen.throw(typ, value, traceback) File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 89, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.ConnectError: All connection attempts failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/app/core/base/pipeline/base_pipeline.py", line 74, in run else await self._consume_all(current_input) File "/app/core/base/pipeline/base_pipeline.py", line 85, in _consume_all async for item in gen: File "/app/core/base/pipeline/base_pipeline.py", line 153, in _run_pipe async for ele in await pipe.run( File "/app/core/base/pipes/base_pipe.py", line 142, in wrapped_run async for result in self._run_logic( File "/app/core/pipes/ingestion/vector_storage_pipe.py", line 78, in _run_logic async for msg in input.message: File "/app/core/base/pipeline/base_pipeline.py", line 153, in _run_pipe async for ele in await pipe.run( File "/app/core/base/pipes/base_pipe.py", line 142, in wrapped_run return await self.embedding_provider.async_get_embeddings( File "/app/core/providers/embeddings/ollama.py", line 156, in async_get_embeddings return await self._execute_with_backoff_async(task) File "/app/core/base/providers/embedding.py", line 62, in _execute_with_backoff_async return await self._execute_task(task) File "/app/core/providers/embeddings/ollama.py", line 74, in _execute_task raise R2RException(error_msg, 400) core.base.abstractions.exception.R2RException: Error getting embeddings: All connection attempts failed

2024-09-04 20:13:55,421 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,421 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed INFO: 192.168.5.108:55938 - "POST /v2/ingest_files HTTP/1.1" 400 Bad Request 2024-09-04 20:13:55,421 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,421 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,421 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,421 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,422 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,422 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,422 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,422 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,422 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,422 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,422 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,422 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,422 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,422 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,423 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,423 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,423 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,423 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55 2024-09-04 20:13:55,423 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,423 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,423 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,423 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,423 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,423 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,424 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,424 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,424 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,424 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,425 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,425 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,425 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,425 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,425 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,425 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,426 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,426 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,427 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,427 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,427 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,427 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,427 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,427 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,427 - ERROR - core.providers.embeddings.ollama - Error getting embeddings: All connection attempts failed 2024-09-04 20:13:55,427 - WARNING - core.base.providers.embedding - Request failed (attempt 2): Error getting embeddings: All connection attempts failed INFO: 127.0.0.1:50298 - "GET /v2/health HTTP/1.1" 200 OK

OLLAMA_API_BASE=http://10.144.86.53:11434 set in environment

remote ollama list is:

ollama list NAME ID SIZE MODIFIED llama3.1:latest 91ab477bec9d 4.7 GB 8 days ago mxbai-embed-large:latest 468836162de7 669 MB 8 days ago sciphi/triplex:latest 6071d91b5626 2.4 GB 8 days ago nomic-embed-text:latest 0a109f422b47 274 MB 2 weeks ago phi3:instruct 4f2222927938 2.2 GB 2 weeks ago llama3.1:8b 91ab477bec9d 4.7 GB 3 weeks ago

NolanTrem commented 1 month ago

From whatever environment you are trying to launch your r2r server, can you run: curl http://10.144.86.53:11434?

It seems like the ollama server is either not up or your connection is getting blocked. If it is running, it will respond with Ollama is running.

MyraBaba commented 1 month ago

Finally I connect upload and make search.

But ollama3.1:8 results :( terrible. Couldn’t find even obvious information.

BC

On 4 Sep 2024, at 23:22, Nolan Tremelling @.***> wrote:

From whatever environment you are trying to launch your r2r server, can you run: curl http://10.144.86.53:11434?

It seems like the ollama server is either not up or your connection is getting blocked. If it is running, it will respond with Ollama is running.

— Reply to this email directly, view it on GitHub https://github.com/SciPhi-AI/R2R/issues/1047#issuecomment-2329900871, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEFRZH5TCERURFHFLBDJKXLZU5TWVAVCNFSM6AAAAABNU7FQ5WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRZHEYDAOBXGE. You are receiving this because you authored the thread.

NolanTrem commented 1 month ago

Looks like you're using Llama3.1:8b. That's a small model. Working with a knowledge graph is tricky, and some of these small models cannot even follow prompts to return parsable output. You'll likely want to play around with things to see what works best for you.

If you aren't already in our Discord, I would encourage you to join. We've had a lot of people go into production using local LLMs.

Before I close these out, I just wanted to make sure everything was resolved?

MyraBaba commented 1 month ago

Yes at the moment resolved.

We want to put few tests servers also.

PS: when the document collection and usr based collection acmes available ?

On 4 Sep 2024, at 23:44, Nolan Tremelling @.***> wrote:

Looks like you're using Llama3.1:8b. That's a small model. Working with a knowledge graph is tricky, and some of these small models cannot even follow prompts to return parsable output. You'll likely want to play around with things to see what works best for you.

If you aren't already in our Discord, I would encourage you to join. We've had a lot of people go into production using local LLMs.

Before I close these out, I just wanted to make sure everything was resolved?

— Reply to this email directly, view it on GitHub https://github.com/SciPhi-AI/R2R/issues/1047#issuecomment-2329951488, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEFRZH4XXIPR3YG6O7ZJ2QDZU5WJFAVCNFSM6AAAAABNU7FQ5WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRZHE2TCNBYHA. You are receiving this because you authored the thread.

NolanTrem commented 1 month ago

We are making it easier for hot swapping of vector collections in the next release. Excited to see what you build!