search

microsoft / onnxruntime-genai

Generative AI extensions for onnxruntime
MIT License
487 stars 122 forks source link

LLama-3.2-1B-Instruct Function Calling error #994

Open madroidmaq opened 1 week ago

madroidmaq commented 1 week ago

Describe the bug

When I use the FunctionCalling feature of the LLama-3.2-1B-Instruct model on an Android device, an error occurs.

The prompt is as follows:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

Environment: ipython
Cutting Knowledge Date: December 2023
Today Date: 18 Oct 2024

Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.<|eot_id|><|start_header_id|>user<|end_header_id|>

Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt.

Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.Do not use variables.

{
    "type": "function",
    "function": {
        "name": "get_current_weather",
        "description": "Get the current weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The location to get the weather for, e.g. San Francisco, CA"
                },
                "format": {
                    "type": "string",
                    "description": "The format to return the weather in, e.g. 'celsius' or 'fahrenheit'",
                    "enum": [
                        "celsius",
                        "fahrenheit"
                    ]
                }
            },
            "required": [
                "location",
                "format"
            ]
        }
    }
}

What's the weather like today<|eot_id|><|start_header_id|>assistant<|end_header_id|>

The error message is as follows:

                                 What is the weather today<|eot_id|><|start_header_id|>assistant<|end_header_id|>
02:09:04.367 libc             A  Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xb400006cc0dd2040 in tid 29979 (DefaultDispatch), pid 29931 (ative.app:genai)
02:09:04.737 DEBUG            A  Cmdline: ai.allama.generative.app:genai
02:09:04.737 DEBUG            A  pid: 29931, tid: 29979, name: DefaultDispatch  >>> ai.allama.generative.app:genai <<<
02:09:04.737 DEBUG            A        #00 pc 0000000000b4531c  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #01 pc 0000000000e3952c  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #02 pc 0000000000e35b10  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #03 pc 0000000000e32cf0  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #04 pc 0000000000e328dc  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #05 pc 0000000000b43944  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #06 pc 0000000000b40c48  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #07 pc 0000000000cd46e8  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #08 pc 0000000000cd37a4  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #09 pc 0000000000cefb30  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #10 pc 0000000000cd609c  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #11 pc 0000000000d0b788  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #12 pc 0000000000d0b2b4  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #13 pc 00000000004db874  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #14 pc 00000000004ddb78  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #15 pc 00000000004a7dfc  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #16 pc 0000000000147ccc  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai.so (offset 0x178000) (Generators::State::Run(OrtSession&, OrtRunOptions&, int)+508) (BuildId: 3e67de3f11079c80638c65d55c883b0654441892)
02:09:04.737 DEBUG            A        #17 pc 0000000000138e28  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai.so (offset 0x178000) (Generators::DecoderOnly_State::Run(int, Generators::RoamingArray<int>, Generators::RoamingArray<int>)+140) (BuildId: 3e67de3f11079c80638c65d55c883b0654441892)
02:09:04.737 DEBUG            A        #18 pc 000000000012c190  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai.so (offset 0x178000) (Generators::Generator::ComputeLogits()+140) (BuildId: 3e67de3f11079c80638c65d55c883b0654441892)
02:09:04.737 DEBUG            A        #19 pc 000000000016ae7c  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai.so (offset 0x178000) (OgaGenerator_ComputeLogits+28) (BuildId: 3e67de3f11079c80638c65d55c883b0654441892)
02:09:04.737 DEBUG            A        #20 pc 00000000000047bc  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai-jni.so (offset 0x110000) (Java_ai_onnxruntime_genai_Generator_computeLogitsNative+20) (BuildId: a4099e8a78f869b68088fe2ce0e0af6aab84894a)
02:09:04.737 DEBUG            A        #31 pc 000000000000606c  /data/data/ai.allama.generative.app/code_cache/.overlay/base.apk/classes6.dex (ai.allama.generative.service.onnx.OnnxModel.generateText+0)
02:09:04.737 DEBUG            A        #37 pc 0000000000004bf0  /data/data/ai.allama.generative.app/code_cache/.overlay/base.apk/classes6.dex (ai.allama.generative.service.onnx.OnnxChatCompletion$chat$1.invokeSuspend+0)
---------------------------- PROCESS ENDED (29931) for package ai.allama.generative.app ----------------------------

Screenshots

Image

Desktop (please complete the following information):

Smartphone (please complete the following information):

Additional context

Everything works fine when I don't include FunctionCalling content in my normal use.

saibhaskerraju commented 1 week ago

I dont think ONNX runtime supports function calling. I searched their API docs but no reference