Error activating local language model with Ollama [Bug]

[start-life]

Model: codellama, Enum: OLLAMA 24.05.06 12:40:06: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '0.00'} 24.05.06 12:40:06: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '0.51'} 24.05.06 12:40:07: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '1.01'} 24.05.06 12:40:07: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '1.52'} 24.05.06 12:40:08: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '2.03'} 24.05.06 12:40:08: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '2.53'} 24.05.06 12:40:09: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '3.04'} 24.05.06 12:40:09: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '3.55'} 24.05.06 12:40:10: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '4.05'} 24.05.06 12:40:10: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '4.56'} 24.05.06 12:40:11: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '5.06'} 24.05.06 12:40:11: root: INFO : SOCKET inference MESSAGE: {'type': 'warning', 'message': 'Inference is taking longer than expected'} 24.05.06 12:40:11: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '5.56'} 24.05.06 12:40:11: root: INFO : SOCKET inference MESSAGE: {'type': 'warning', 'message': 'Inference is taking longer than expected'} 24.05.06 12:40:12: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '6.09'} 24.05.06 12:40:12: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '6.60'} 24.05.06 12:40:13: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '7.11'} 24.05.06 12:40:13: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '7.62'} 24.05.06 12:40:14: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '8.13'} 24.05.06 12:40:14: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '8.64'} 24.05.06 12:40:15: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '9.15'} 24.05.06 12:40:15: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '9.66'} 24.05.06 12:40:16: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '10.17'} 24.05.06 12:40:17: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '10.68'} 24.05.06 12:40:17: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '11.19'} 24.05.06 12:40:18: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '11.69'} 24.05.06 12:40:18: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '12.20'} 24.05.06 12:40:19: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '12.70'} 24.05.06 12:40:19: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '13.20'} 24.05.06 12:40:20: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '13.71'} 24.05.06 12:40:20: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '14.21'} 24.05.06 12:40:21: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '14.72'} 24.05.06 12:40:21: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '15.23'} 24.05.06 12:40:22: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '15.73'} 24.05.06 12:40:22: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '16.24'} 24.05.06 12:40:23: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '16.75'} 24.05.06 12:40:23: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '17.25'} 24.05.06 12:40:24: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '17.76'} 24.05.06 12:40:24: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '18.26'} 24.05.06 12:40:25: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '18.76'} 24.05.06 12:40:25: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '19.27'} 24.05.06 12:40:26: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '19.77'} 24.05.06 12:40:26: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '20.27'} 24.05.06 12:40:27: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '20.78'} 24.05.06 12:40:27: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '21.29'} 24.05.06 12:40:28: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '21.79'} 24.05.06 12:40:28: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '22.30'} 24.05.06 12:40:29: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '22.83'} 24.05.06 12:40:29: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '23.34'} 24.05.06 12:40:30: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '23.85'} 24.05.06 12:40:30: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '24.36'} 24.05.06 12:40:31: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '24.86'} 24.05.06 12:40:31: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '25.38'} 24.05.06 12:40:32: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '25.88'} 24.05.06 12:40:32: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '26.39'} 24.05.06 12:40:33: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '26.90'} 24.05.06 12:40:33: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '27.40'} 24.05.06 12:40:34: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '27.91'} 24.05.06 12:40:34: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '28.42'} 24.05.06 12:40:35: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '28.93'} 24.05.06 12:40:35: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '29.44'} 24.05.06 12:40:36: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '29.95'} 24.05.06 12:40:36: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '30.46'} 24.05.06 12:40:37: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '30.97'} 24.05.06 12:40:37: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '31.47'} 24.05.06 12:40:38: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '31.97'} 24.05.06 12:40:38: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '32.48'} 24.05.06 12:40:39: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '32.99'} 24.05.06 12:40:39: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '33.49'} 24.05.06 12:40:40: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '34.00'} 24.05.06 12:40:40: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '34.51'} 24.05.06 12:40:41: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '35.01'} 24.05.06 12:40:41: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '35.52'} 24.05.06 12:40:42: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '36.03'} 24.05.06 12:40:42: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '36.53'} 24.05.06 12:40:43: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '37.04'} 24.05.06 12:40:43: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '37.54'} 24.05.06 12:40:44: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '38.05'} 24.05.06 12:40:44: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '38.56'} 24.05.06 12:40:45: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '39.07'} 24.05.06 12:40:45: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '39.58'} 24.05.06 12:40:46: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '40.09'} 24.05.06 12:40:46: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '40.60'} 24.05.06 12:40:47: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '41.11'} 24.05.06 12:40:47: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '41.62'} 24.05.06 12:40:48: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '42.12'} 24.05.06 12:40:48: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '42.62'} 24.05.06 12:40:49: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '43.12'} 24.05.06 12:40:49: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '43.63'} 24.05.06 12:40:50: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '44.14'} 24.05.06 12:40:50: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '44.64'} 24.05.06 12:40:51: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '45.15'} 24.05.06 12:40:51: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '45.65'} 24.05.06 12:40:52: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '46.16'} 24.05.06 12:40:53: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '46.67'} 24.05.06 12:40:53: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '47.17'} 24.05.06 12:40:54: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '47.68'} 24.05.06 12:40:54: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '48.19'} 24.05.06 12:40:55: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '48.69'} 24.05.06 12:40:55: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '49.19'} 24.05.06 12:40:56: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '49.70'} 24.05.06 12:40:56: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '50.21'} 24.05.06 12:40:57: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '50.72'} 24.05.06 12:40:57: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '51.22'} 24.05.06 12:40:58: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '51.73'} 24.05.06 12:40:58: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '52.23'} 24.05.06 12:40:59: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '52.74'} 24.05.06 12:40:59: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '53.24'} 24.05.06 12:41:00: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '53.74'} 24.05.06 12:41:00: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '54.24'} 24.05.06 12:41:01: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '54.75'} 24.05.06 12:41:01: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '55.25'} 24.05.06 12:41:02: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '55.77'} 24.05.06 12:41:02: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '56.27'} 24.05.06 12:41:03: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '56.78'} 24.05.06 12:41:03: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '57.28'} 24.05.06 12:41:04: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '57.78'} 24.05.06 12:41:04: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '58.29'} 24.05.06 12:41:05: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '58.80'} 24.05.06 12:41:05: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '59.31'} 24.05.06 12:41:06: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '59.81'} 24.05.06 12:41:06: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '60.32'} 24.05.06 12:41:06: root: ERROR : Inference failed. took too long. Model: OLLAMA, Model ID: codellama 24.05.06 12:41:06: root: INFO : SOCKET inference MESSAGE: {'type': 'error', 'message': 'Inference took too long. Please try again.'}

[Polygons1]

nothing is wrong, just the CPU can’t handle Devika and Ollama at the same time (it happened to me)

[Orizune]

i got the same problem, but it is not a hardware problem, since i can use the ollama without ptoblem and other demanding applications too.

[ARajgor]

you can increase the timeout from settings page

[start-life]

It's not a matter of a time limit, it just can't connect to the model, even if I give it 20 minutes, it just won't work. There's some kind of glitch in the code

[ARajgor]

if you see ollama model being listed on UI then it connected to devika.

[start-life]

אם אתה רואה את מודל ollama מופיע בממשק המשתמש, הוא התחבר ל-devika.

True it connects but gives an error as you can see in the log output I sent above in the first message

[hqnicolas]

@start-life @ARajgor I have a Devika for Ollama on that REPO if it works, dont forgot the start on it

[stollae]

I experience the same issue where Devika gets to work immediately when I have an external API added to the config.toml, and Devika detects which local llms I've installed and I can select them for a new project. Unfortunately, whenever I attempt to use a local, the backlog either displays into a loop where it continues retrying its response and ultimately times out, or it simply times out after a few minutes without reattempting and results in this error: Socket connected :: {'data': 'frontend connected!'} 24.05.13 12:54:25: root: INFO : SOCKET socket_response MESSAGE: {'data': 'Server Connected'} 24.05.13 12:54:37: root: INFO : User message: {'message': 'Hello!', 'base_model': 'dolphin-mixtral', 'project_name': 'Baseball card App', 'search_engine': 'Google'} 24.05.13 12:54:37: root: INFO : SOCKET info MESSAGE: {'type': 'warning', 'message': "previous agent doesn't completed it's task."} 24.05.13 12:54:37: root: INFO : SOCKET server-message MESSAGE: {'messages': {'from_devika': False, 'message': 'Hello!', 'timestamp': '2024-05-13 12:54:37'}} 24.05.13 12:54:37: root: INFO : SOCKET agent-state MESSAGE: [{'internal_monologue': "I'm starting the work...", 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': 1, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 0, 'timestamp': '2024-05-13 12:54:37'}] 24.05.13 12:54:37: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 1090} Model: dolphin-mixtral, Enum: OLLAMA 24.05.13 12:54:37: root: INFO : SOCKET inference MESSAGE: {'type': 'time', 'elapsed_time': '0.00'} 24.05.13 12:55:38: root: ERROR : Inference took too long. Model: OLLAMA, Model ID: dolphin-mixtral 24.05.13 12:55:38: root: INFO : SOCKET inference MESSAGE: {'type': 'error', 'message': 'Inference took too long. Please try again.'} 24.05.13 12:55:38: root: WARNING: Inference failed

I did watch the youtube video by Md Zahair, where he edited the response parameters that are saved in Devika's root folder, but the difference in my situation seems to me that whatever the local is trying to say isn't making it to/through devika or the front-end. This doesn't even sound like proper sense to me, I'm a bit n00b in this area.

I updated the system environment variables as required (I believe), and the local lm runs just fine in a command prompt, but not Devika.

Is there something else specifically that I have to set manually? Something else, similar to environment variables?

Is this actually simply a small issue with devika not quite understanding how to respond and then timing out? It seems like if I were to give it an example (like in the video) and specify further rules for output, that Devika might just end up sitting in silence for an extended period of time and simply not print the loop it's experiencing in the logs before it fails. I'll review the video again and attempt to edit accordingly in the meantime.

EDIT: Now my logs look identical to the post above. Can it be something to do with a port incongruity of some sort? It seems like the local llm isn't getting through, or is discarding every response it generates as invalid. Possibly why further refining how it answers helps? Summation: Devika operates as expected when I select an exterior API, just not with any local for some reason.

[haddadnidal]

I am having this issue too, any updates ? Thank you