search

mudler / LocalAI

:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed inference
https://localai.io
MIT License
23.22k stars 1.76k forks source link

Schema required possible not implemented #865

Open andr6 opened 1 year ago

andr6 commented 1 year ago

LocalAI version:

LocalAI build with j with CMAKE_ARGS="-DLLAMA_F16C=OFF -DLLAMA_AVX512=OFF -DLLAMA_AVX2=OFF -DLLAMA_FMA=OFF" make build INF LocalAI version: v1.23.1-7-g4aa5dac (4aa5dac768a5255667097db1c043581196fa46b2)

Environment, CPU architecture, OS, and Version:

VM machine - Linux Ubuntu20 5.15.0-76-generic #83-Ubuntu SMP Thu Jun 15 19:16:32 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux -

Describe the bug

Localai does not provide a completion to the prompt .

To Reproduce

Expected behavior Completion to finish successfully.

Logs

Currently running LocalAI with AutoGPT and having the following error on autogpt.:
AttributeError: 'NoneType' object has no attribute 'lower'

LocalAI logs : [127.0.0.1]:49724 200 - GET /v1/models 2:21PM DBG Request received: 2:21PM DBG Configuration read: &{PredictionOptions:{Model:gpt-3.5-turbo Language: N:0 TopP:0.7 TopK:80 Temperature:0.9 Maxtokens:3256 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0} Name: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:2048 F16:true NUMA:false Threads:4 Debug:true Roles:map[] Embeddings:false Backend: TemplateConfig:{Chat: ChatMessage: Completion: Edit: Functions:} MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 MMap:false MMlock:false LowVRAM:false TensorSplit: MainGPU: ImageGenerationAssets: PromptCachePath: PromptCacheAll:false PromptCacheRO:false Grammar: PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} SystemPrompt: RMSNormEps:0 NGQA:0} 2:21PM DBG Parameters: &{PredictionOptions:{Model:gpt-3.5-turbo Language: N:0 TopP:0.7 TopK:80 Temperature:0.9 Maxtokens:3256 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0} Name: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:2048 F16:true NUMA:false Threads:4 Debug:true Roles:map[] Embeddings:false Backend: TemplateConfig:{Chat: ChatMessage: Completion: Edit: Functions:} MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 MMap:false MMlock:false LowVRAM:false TensorSplit: MainGPU: ImageGenerationAssets: PromptCachePath: PromptCacheAll:false PromptCacheRO:false Grammar: PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} SystemPrompt: RMSNormEps:0 NGQA:0} .... PROMPT before templating ... Respond with only valid JSON conforming to the following schema: {"$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "properties": {"thoughts": {"type": "object", "properties": {"text": {"type": "string", "description": "thoughts"}, "reasoning": {"type": "string"}, "plan": {"type": "string", "description": "- short bulleted\n- list that conveys\n- long-term plan"}, "criticism": {"type": "string", "description": "constructive self-criticism"}, "speak": {"type": "string", "description": "thoughts summary to say to user"}}, "required": ["text", "reasoning", "plan", "criticism", "speak"], "additionalProperties": false}, "command": {"type": "object", "properties": {"name": {"type": "string"}, "args": {"type": "object"}}, "required": ["name", "args"], "additionalProperties": false}}, "required": ["thoughts", "command"], "additionalProperties": false}

The current time and date is Fri Aug 4 14:21:19 2023 Determine exactly one command to use, and respond using the JSON schema specified previously: 2:21PM DBG Template failed loading: failed loading a template for gpt-3.5-turbo ... PROMPT after templating ... Respond with only valid JSON conforming to the following schema: {"$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "properties": {"thoughts": {"type": "object", "properties": {"text": {"type": "string", "description": "thoughts"}, "reasoning": {"type": "string"}, "plan": {"type": "string", "description": "- short bulleted\n- list that conveys\n- long-term plan"}, "criticism": {"type": "string", "description": "constructive self-criticism"}, "speak": {"type": "string", "description": "thoughts summary to say to user"}}, "required": ["text", "reasoning", "plan", "criticism", "speak"], "additionalProperties": false}, "command": {"type": "object", "properties": {"name": {"type": "string"}, "args": {"type": "object"}}, "required": ["name", "args"], "additionalProperties": false}}, "required": ["thoughts", "command"], "additionalProperties": false}

The current time and date is Fri Aug 4 14:21:19 2023 Determine exactly one command to use, and respond using the JSON schema specified previously: 2:21PM DBG Model already loaded in memory: gpt-3.5-turbo 2:21PM DBG Model 'gpt-3.5-turbo' already loaded 3:02PM DBG GRPC(gpt-3.5-turbo-127.0.0.1:39897): stderr 3:02PM DBG GRPC(gpt-3.5-turbo-127.0.0.1:39897): stderr llama_print_timings: load time = 3290.69 ms 3:02PM DBG GRPC(gpt-3.5-turbo-127.0.0.1:39897): stderr llama_print_timings: sample time = 2523.88 ms / 3256 runs ( 0.78 ms per token, 1290.08 tokens per second) 3:02PM DBG GRPC(gpt-3.5-turbo-127.0.0.1:39897): stderr llama_print_timings: prompt eval time = 1141027.06 ms / 3941 tokens ( 289.53 ms per token, 3.45 tokens per second) 3:02PM DBG GRPC(gpt-3.5-turbo-127.0.0.1:39897): stderr llama_print_timings: eval time = 1319191.30 ms / 3252 runs ( 405.66 ms per token, 2.47 tokens per second) 3:02PM DBG GRPC(gpt-3.5-turbo-127.0.0.1:39897): stderr llama_print_timings: total time = 2463991.69 ms 3:02PM DBG Response: {"object":"chat.completion","model":"gpt-3.5-turbo","choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"\"name\": \"execute_python_code\", \"args\": {\"code\": \"print('Hello, world!')\", \"name\": \"my_script.py\"}, \"required\": [\"code\", \"name\"], \"additionalProperties\": false}\n{\"$schema\": \"http://json-schema.org/draft-07/schema#\", \"type\": \"object\", \"properties\": {\"thoughts\": {\"type\": \"object\", \"properties\": {\"text\": {\"type\": \"string\"}, \"reasoning\": {\"type\": \"string\"}, \"plan\": {\"type\": \"string\"}, \"criticism\": {\"type\": \"string\"}, \"speak\": {\"type\": \"string\", \"description\": \"thoughts summary to say to user\"}}, \"required\": [\"text\", \"reasoning\", \"plan\", \"criticism\", \"speak\"], \"additionalProperties\": false}, \"command\": {\"type\": \"object\", \"properties\": {\"name\": {\"type\": \"string\"}, \"args\": {\"type\": \"object\"}}, \"required\": [\"name\", \"args\"], \"additionalProperties\": false}, \"name\": \"execute_python_code\", \"args\": {\"code\": \"print('Hello, world!')\", \"name\": \"my_script.py\"}, \"required\": [\"code\", \"name\"], \"additionalProperties\": false}\n\"name\": \"execute_python_file\", \"args\": {\"filename\": \"my_script.py\"}, \"required\": [\"filename\"], \"additionalProperties\": false}\n{\"$schema\": \"http://json-schema.org/draft-07/schema#\", \"type\": \"object\", \"properties\": {\"thoughts\": {\"type\": \"object\", \"properties\": {\"text\": {\"type\": \"string\"}, \"reasoning\": {\"type\": \"string\"}, \"plan\": {\"type\": \"string\"}, \"criticism\": {\"type\": \"string\"}, \"speak\": {\"type\": \"string\", \"description\": \"thoughts summary to say to user\"}}, \"required\": [\"text\", \"reasoning\", \"plan\", \"criticism\", \"speak\"], \"additionalProperties\": false}, \"command\": {\"type\": \"object\", \"properties\": {\"name\": {\"type\": \"string\"}, \"args\": {\"type\": \"object\"}}, \"required\": [\"name\", \"args\"], \"additionalProperties\": false}, \"name\": \"execute_python_file\", \"args\": {\"filename\": \"my_script.py\"}, \"required\": [\"filename\"], \"additionalProperties\": false}\n{\"$schema\": \"http://json-schema.org/draft-07/schema#\", \"type\": \"object\", \"properties\": {\"thoughts\": {\"type\": \"object\", \"properties\": {\"text\": {\"type\": \"string\"}, \"reasoning\": {\"type\": \"string\"}, \"plan\": {\"type\": \"string\"}, \"criticism\": {\"type\": \"string\"}, \"speak\": {\"type\": \"string\", \"description\": \"thoughts summary to say to user\"}}, \"required\": [\"text\", \"reasoning\", \"plan\", \"criticism\", \"speak\"], \"additionalProperties\": false}, \"name\": \"write_to_file\", \"args\": {\"filename\": \"my_script.py\"}, \"required\": [\"filename\"], \"additionalProperties\": false}\n{\"$schema\": \"http://json-schema.org/draft-07/schema#\", \"type\": \"object\", \"properties\": {\"thoughts\": {\"type\": \"object\", \"properties\": {\"text\": {\"type\": \"string\"}, \"reasoning\": {\"type\": \"string\"}, \"plan\": {\"type\": \"string\"}, \"criticism\": {\"type\": \"string\"}, \"speak\": {\"type\": \"string\", \"description\": \"thoughts summary to say to user\"}}, \"required\": [\"text\", \"reasoning\", \"plan\", \"criticism\", \"speak\"], \"additionalProperties\": false}, \"command\": {\"type\": \"object\", \"properties\": {\"name\": {\"type\": \"string\"}, \"args\": {\"type\": \"object\"}}, \"required\": [\"name\", \"args\"], \"additionalProperties\": false}, \"name\": \"write_to_file\", \"args\": {\"filename\": \"my_script.py\"}, \"required\": [\"filename\"], \"additionalProperties\": false}\n{\"$schema\": \"http://json-schema.org/draft-07/schema#\", \"type\": \"object\", \"properties\": {\"thoughts\": {\"type\": \"object\", \"properties\": {\"text\": {\"type\": \"string\"}, \"reasoning\": {\"type\": \"string\"}, \"plan\": {\"type\": \"string\"}, \"criticism\": {\"type\": \"string\"}, \"speak\": {\"type\": \"string\", \"description\": \"thoughts summary to say to user\"}}, \"required\": [\"text\", \"reasoning\", \"plan\", \"criticism\", \"speak\"], \"additionalProperties\": false}, \"name\": \"read_file\", \"args\": {\"filename\": \"my_script.py\"}, \"required\": [\"filename\"], \"additionalProperties\": false}\n{\"$schema\": \"http://json-schema.org/draft-07/schema#\", \"type\": \"object\", \"properties\": {\"thoughts\": {\"type\": \"object\", \"properties\": {\"text\": {\"type\": \"string\"}, \"reasoning\": {\"type\": \"string\"}, \"plan\": {\"type\": \"string\"}, \"criticism\": {\"type\": \"string\"}, \"speak\": {\"type\": \"string\", \"description\": \"thoughts summary to say to user\"}}, \"required\": [\"text\", \"reasoning\", \"plan\", \"criticism\", \"speak\"], \"additionalProperties\": false}, \"command\": {\"type\": \"object\", \"properties\": {\"name\": {\"type\": \"string\"}, \"args\": {\"type\": \"object\"}}, \"required\": [\"name\", \"args\"], \"additionalProperties\": false}, \"name\": \"read_file\", \"args\": {\"filename\": \"my_script.py\"}, \"required\": [\"filename\"], \"additionalProperties\": false}\n{\"$schema\": \"http://json-schema.org/draft-07/schema#\", \"type\": \"object\", \"properties\": {\"thoughts\": {\"type\": \"object\", \"properties\": {\"text\": {\"type\": \"string\"}, \"reasoning\": {\"type\": \"string\"}, \"plan\": {\"type\": \"string\"}, \"criticism\": {\"type\": \"string\"}, \"speak\": {\"type\": \"string\", \"description\": \"thoughts summary to say to user\"}}, \"required\": [\"text\", \"reasoning\", \"plan\", \"criticism\", \"speak\"], \"additionalProperties\": false}, \"name\": \"execute_python_code\", \"args\": {\"code\": \"print('Hello, world!')\", \"name\": \"my_script.py\"}, \"required\": [\"code\", \"name\"], \"additionalProperties\": false}\n{\"$schema\": \"http://json-schema.org/draft-07/schema#\", \"type\": \"object\", \"properties\": {\"thoughts\": {\"type\": \"object\", \"properties\": {\"text\": {\"type\": \"string\"}, \"reasoning\": {\"type\": \"string\"}, \"plan\": {\"type\": \"string\"}, \"criticism\": {\"type\": \"string\"}, \"speak\": {\"type\": \"string\", \"description\": \"thoughts summary to say to user\"}}, \"required\": [\"text\", \"reasoning\", \"plan\", \"criticism\", \"speak\"], \"additionalProperties\": false}, \"name\": \"write_to_file\", \"args\": {\"filename\": \"my_script.py\"}, \"required\": [\"filename\"], \"additionalProperties\": false}\n{\"$schema\": \"http://json-schema.org/draft-07/schema#\", \"type\": \"object\", \"properties\": {\"thoughts\": {\"type\": \"object\", \"properties\": {\"text\": {\"type\": \"string\"}, \"reasoning\": {\"type\": \"string\"}, \"plan\": {\"type\": \"string\"}, \"criticism\": {\"type\": \"string\"}, \"speak\": {\"type\": \"string\", \"description\": \"thoughts summary to say to user\"}}, \"required\": [\"text\", \"reasoning\", \"plan\", \"criticism\", \"speak\"], \"additionalProperties\": false}, \"command\": {\"type\": \"object\", \"properties\": {\"name\": {\"type\": \"string\"}, \"args\": {\"type\": \"object\"}}, \"required\": [\"name\", \"args\"], \"additionalProperties\": false}, \"name\": \"write_to_file\", \"args\": {\"filename\": \"my_script.py\"}, \"required\": [\"filename\"], \"additionalProperties\": false}\n{\"$schema\": \"http://json-schema.org/draft-07/schema#\", \"type\": \"object\", \"properties\": {\"thoughts\": {\"type\": \"object\", \"properties\": {\"text\": {\"type\": \"string\"}, \"reasoning\": {\"type\": \"string\"}, \"plan\": {\"type\": \"string\"}, \"criticism\": {\"type\": \"string\"}, \"speak\": {\"type\": \"string\", \"description\": \"thoughts summary to say to user\"}}, \"required\": [\"text\", \"reasoning\", \"plan\", \"criticism\", \"speak\"], \"additionalProperties\": false}, \"name\": \"read_file\", \"args\": {\"filename\": \"my_script.py\"}, \"required\": [\"filename\"], \"additionalProperties\": false}\n{\"$schema\": \"http://json-schema.org/draft-07/schema#\", \"type\": \"object\", \"properties\": {\"thoughts\": {\"type\": \"object\", \"properties\": {\"text\": {\"type\": \"string\"}, \"reasoning\": {\"type\": \"string\"}, \"plan\": {\"type\": \"string\"}, \"criticism\": {\"type\": \"string\"}, \"speak\": {\"type\": \"string\", \"description\": \"thoughts summary to say to user\"}}, \"required\": [\"text\", \"reasoning\", \"plan\", \"criticism\", \"speak\"], \"additionalProperties\": false}, \"name\": \"execute_python_code\", \"args\": {\"code\": \"print('Hello, world!')\", \"name\": \"my_script.py\"}, \"required\": [\"code\", \"name\"], \"additionalProperties\": false}\n{\"$schema\": \"http://json-schema.org/draft-07/schema#\", \"type\": \"object\", \"properties\": {\"thoughts\": {\"type\": \"object\", \"properties\": {\"text\": {\"type\": \"string\"}, \"reasoning\": {\"type\": \"string\"}, \"plan\": {\"type\": \"string\"}, \"criticism\": {\"type\": \"string\"}, \"speak\": {\"type\": \"string\", \"description\": \"thoughts summary to say to user\"}}, \"required\": [\"text\", \"reasoning\", \"plan\", \"criticism\", \"speak\"], \"additionalProperties\": false}, \"name\": \"write_to_file\", \"args\": {\"filename\": \"my_script.py\"}, \"required\": [\"filename\"], \"additionalProperties\": false}\n{\"$schema\": \"http://json-schema.org/draft-07/schema#\", \"type\": \"object\", \"properties\": {\"thoughts\": {\"type\": \"object\", \"properties\": {\"text\": {\"type\": \"string\"}, \"reasoning\": {\"type\": \"string\"}, \"plan\": {\"type\": \"string\"}, \"criticism\": {\"type\": \"string\"}, \"speak\": {\"type\": \"string\", \"description\": \"thoughts summary to say to user\"}}, \"required\": [\"text\", \"reasoning\", \"plan\", \"criticism\", \"speak\"], \"additionalProperties\": false}, \"name\": \"read_file\", \"args\": {\"filename\": \"my_script.py\"}, \"required\": [\"filename\"], \"additionalProperties\": false}\n{\"$schema\": \"http://json-schema.org/draft-07/schema#\", \"type\": \"object\", \"properties\": {\"thoughts\": {\"type\": \"object\", \"properties\": {\"text\": {\"type\": \"string\"}, \"reasoning\": {\"type\": \"string\"}, \"plan\": {\"type\": \"string\"}, \"criticism\": {\"type\": \"string\"}, \"speak\": {\"type\": \"string\", \"description\": \"thoughts summary to say to user\"}}, \"required\": [\"text\", \"reasoning\", \"plan\", \"criticism\", \"speak\"], \"additionalProperties\": false}, \"name\": \"execute_python_code\", \"args\": {\"code\": \"print('Hello, world!')\", \"name\": \"my_script.py\"}, \"required\": [\"code\", \"name\"], \"additionalProperties\": false}\n{\"$schema\": \"http://json-schema.org/draft-07/schema#\", \"type\": \"object\", \"properties\": {\"thoughts\": {\"type\": \"object\", \"properties\": {\"text\": {\"type\": \"string\"}, \"reasoning\": {\"type\": \"string\"}, \"plan\": {\"type\": \"string\"}, \"criticism\": {\"type\": \"string\"}, \"speak\": {\"type\": \"string\", \"description\": \"thoughts summary to say to user\"}}, \"required\": [\"text\", \"reasoning\", \"plan\", \"criticism\", \"speak\"], \"additionalProperties\": false}, \"name\": \"write_to_file\", \"args\": {\"filename\": \"my_script.py\"}, \"required\": [\"filename\"], \"additionalProperties\": false}\n{\"$schema\": \"http://json-schema.org/draft-07/schema#\", \"type\": \"object\", \"properties\": {\"thoughts\": {\"type\": \"object\", \"properties\": {\"text\": {\"type\": \"string\"}, \"reasoning\": {\"type\": \"string\"}, \"plan\": {\"type\": \"string\"}, \"criticism\": {\"type\": \"string\"}, \"speak\": {\"type\": \"string\", \"description\": \"thoughts summary to say to user\"}}, \"required\": [\"text\", \"reasoning\", \"plan\", \"criticism\", \"speak\"], \"additionalProperties\": false}, \"name\": \"read_file\", \"args\": {\"filename\": \"my_script.py\"}, \"required\": [\"filename\"], \"additionalProperties\": false}\n{\"$schema\": \"http://json-schema.org/draft-07/schema#\", \"type\": \"object\", \"properties\": {\"thoughts\": {\"type\": \"object\", \"properties\": {\"text\": {\"type\": \"string\"}, \"reasoning\": {\"type\": \"string\"}, \"plan\": {\"type\": \"string\"}, \"criticism\": {\"type\": \"string\"}, \"speak\": {\"type\": \"string\", \"description\":"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}} [127.0.0.1]:49724 200 - POST /v1/chat/completions

Additional context

localai-bot commented 1 year ago

:warning::warning::warning::warning::warning:

Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!

_but.... I can also be funny or helpful :smilecat: and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.

Don't engage in conversation with me, I don't support (yet) replying!

:warning::warning::warning::warning::warning:

Sources: