LLama-3.2-1B-Instruct Function Calling error

[madroidmaq]

Describe the bug

When I use the FunctionCalling feature of the LLama-3.2-1B-Instruct model on an Android device, an error occurs.

The prompt is as follows:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

Environment: ipython
Cutting Knowledge Date: December 2023
Today Date: 18 Oct 2024

Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.<|eot_id|><|start_header_id|>user<|end_header_id|>

Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt.

Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.Do not use variables.

{
    "type": "function",
    "function": {
        "name": "get_current_weather",
        "description": "Get the current weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The location to get the weather for, e.g. San Francisco, CA"
                },
                "format": {
                    "type": "string",
                    "description": "The format to return the weather in, e.g. 'celsius' or 'fahrenheit'",
                    "enum": [
                        "celsius",
                        "fahrenheit"
                    ]
                }
            },
            "required": [
                "location",
                "format"
            ]
        }
    }
}

What's the weather like today<|eot_id|><|start_header_id|>assistant<|end_header_id|>

The error message is as follows:

                                 What is the weather today<|eot_id|><|start_header_id|>assistant<|end_header_id|>
02:09:04.367 libc             A  Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xb400006cc0dd2040 in tid 29979 (DefaultDispatch), pid 29931 (ative.app:genai)
02:09:04.737 DEBUG            A  Cmdline: ai.allama.generative.app:genai
02:09:04.737 DEBUG            A  pid: 29931, tid: 29979, name: DefaultDispatch  >>> ai.allama.generative.app:genai <<<
02:09:04.737 DEBUG            A        #00 pc 0000000000b4531c  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #01 pc 0000000000e3952c  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #02 pc 0000000000e35b10  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #03 pc 0000000000e32cf0  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #04 pc 0000000000e328dc  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #05 pc 0000000000b43944  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #06 pc 0000000000b40c48  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #07 pc 0000000000cd46e8  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #08 pc 0000000000cd37a4  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #09 pc 0000000000cefb30  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #10 pc 0000000000cd609c  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #11 pc 0000000000d0b788  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #12 pc 0000000000d0b2b4  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #13 pc 00000000004db874  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #14 pc 00000000004ddb78  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #15 pc 00000000004a7dfc  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #16 pc 0000000000147ccc  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai.so (offset 0x178000) (Generators::State::Run(OrtSession&, OrtRunOptions&, int)+508) (BuildId: 3e67de3f11079c80638c65d55c883b0654441892)
02:09:04.737 DEBUG            A        #17 pc 0000000000138e28  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai.so (offset 0x178000) (Generators::DecoderOnly_State::Run(int, Generators::RoamingArray<int>, Generators::RoamingArray<int>)+140) (BuildId: 3e67de3f11079c80638c65d55c883b0654441892)
02:09:04.737 DEBUG            A        #18 pc 000000000012c190  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai.so (offset 0x178000) (Generators::Generator::ComputeLogits()+140) (BuildId: 3e67de3f11079c80638c65d55c883b0654441892)
02:09:04.737 DEBUG            A        #19 pc 000000000016ae7c  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai.so (offset 0x178000) (OgaGenerator_ComputeLogits+28) (BuildId: 3e67de3f11079c80638c65d55c883b0654441892)
02:09:04.737 DEBUG            A        #20 pc 00000000000047bc  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai-jni.so (offset 0x110000) (Java_ai_onnxruntime_genai_Generator_computeLogitsNative+20) (BuildId: a4099e8a78f869b68088fe2ce0e0af6aab84894a)
02:09:04.737 DEBUG            A        #31 pc 000000000000606c  /data/data/ai.allama.generative.app/code_cache/.overlay/base.apk/classes6.dex (ai.allama.generative.service.onnx.OnnxModel.generateText+0)
02:09:04.737 DEBUG            A        #37 pc 0000000000004bf0  /data/data/ai.allama.generative.app/code_cache/.overlay/base.apk/classes6.dex (ai.allama.generative.service.onnx.OnnxChatCompletion$chat$1.invokeSuspend+0)
---------------------------- PROCESS ENDED (29931) for package ai.allama.generative.app ----------------------------

Screenshots

Desktop (please complete the following information):

• OS: macOS M3
• Browser：chrome
• Version： 14.6.1

Smartphone (please complete the following information):

• Device: Android simulators
• OS: Android
• Browser： chrome
• Version ：Android 15

Additional context

Everything works fine when I don't include FunctionCalling content in my normal use.

[saibhaskerraju]

I dont think ONNX runtime supports function calling. I searched their API docs but no reference

[natke]

Can you share your environment please?

[madroidmaq]

@natke Can you specify what environmental information you need？

As mentioned in the description of the problem, I am using an Android emulator (ARM) with Android version number Android 15.

[scsonic]

I can execute your prompt with default android example the function call are text output, same as text chat you must process string match you self in android the python package like langchain already handle them all