Add iOS Support for sherpa-onnx-go

[JatosDetking]

Description: I am currently encountering an issue while attempting to build the sherpa-onnx-go library for iOS. The build process fails with the following error message:

bash Copy code (.venv) aleksandarsvinarov@Aleksandars-MacBook-Pro test % GOOS=ios GOARCH=arm64 CGO_ENABLED=1 CC=$(go env GOROOT)/misc/ios/clangwrap.sh go build -ldflags -w -v -tags ios -o "IOS.a" -buildmode=c-archive

package test imports github.com/k2-fsa/sherpa-onnx-go/sherpa_onnx: build constraints exclude all Go files in /Users/aleksandarsvinarov/go/pkg/mod/github.com/k2-fsa/sherpa-onnx-go@v1.8.7/sherpa_onnx Steps to Reproduce: Set up the environment: sh Copy code (.venv) aleksandarsvinarov@Aleksandars-MacBook-Pro test % Run the following build command: sh Copy code GOOS=ios GOARCH=arm64 CGO_ENABLED=1 CC=$(go env GOROOT)/misc/ios/clangwrap.sh go build -ldflags -w -v -tags ios -o "IOS.a" -buildmode=c-archive Expected Behavior: The library should build successfully for the iOS target architecture.

Actual Behavior: The build process fails with the message:

bash Copy code package test imports github.com/k2-fsa/sherpa-onnx-go/sherpa_onnx: build constraints exclude all Go files in /Users/aleksandarsvinarov/go/pkg/mod/github.com/k2-fsa/sherpa-onnx-go@v1.8.7/sherpa_onnx Environment: macOS (version details) Go version: (version details) sherpa-onnx-go version: v1.8.7 Additional Information: It appears that there may be build constraints within the sherpa-onnx-go package that exclude files necessary for the iOS build. Any guidance or modifications to support iOS would be greatly appreciated.

Thank you for your attention to this issue.

[csukuangfj]

Please follow the instructions below strictly

1. Build sherpa-onnx for iOS

# Assume you put sherpa-onnx inside /tmp
# You can change it to any directory you want
cd /tmp

git clone https://github.com/k2-fsa/sherpa-onnx
cd sherpa-onnx
./build-ios.sh

cd scripts/go/_internal
ln -s /tmp/sherpa-onnx/build-ios ./

2. Create build_ios.go

cd /tmp/sherpa-onnx/scripts/go/_internal
touch build_ios.go

and put the following content inside build_ios.go:

//go:build darwin && arm64 && ios

package sherpa_onnx

// #cgo LDFLAGS: -L ${SRCDIR}/build-ios/sherpa-onnx.xcframework/ios-arm64 -L ${SRCDIR}/build-ios/ios-onnxruntime/onnxruntime.xcframework/ios-arm64 -lsherpa-onnx -lonnxruntime -Wl,-rpath,${SRCDIR}/lib/aarch64-apple-darwin
import "C"

3. Ready to build your go program

Suppose that you want to build https://github.com/k2-fsa/sherpa-onnx/tree/master/scripts/go/_internal/streaming-decode-files

cd /tmp/sherpa-onnx/scripts/go/_internal/streaming-decode-files
go mod tidy

GOOS=ios GOARCH=arm64 CGO_ENABLED=1 CC=$(go env GOROOT)/misc/ios/clangwrap.sh go build -ldflags -w -v -tags ios -o "IOS.a" -buildmode=c-archive

In the end, you should get the following file

(py38) fangjuns-MacBook-Pro:streaming-decode-files fangjun$ ls -lh IOS.a
-rw-r--r--  1 fangjun  staff   2.3M Jul 16 21:42 IOS.a

(py38) fangjuns-MacBook-Pro:streaming-decode-files fangjun$ ar t ./IOS.a
__.SYMDEF SORTED
go.o
000000.o
000001.o
000002.o
000003.o
000004.o
000005.o
000006.o
000007.o
000008.o
000009.o
000010.o
000011.o
000012.o

(py38) fangjuns-MacBook-Pro:streaming-decode-files fangjun$ ar x ./IOS.a
(py38) fangjuns-MacBook-Pro:streaming-decode-files fangjun$ file *.o
000000.o: Mach-O 64-bit object arm64
000001.o: Mach-O 64-bit object arm64
000002.o: Mach-O 64-bit object arm64
000003.o: Mach-O 64-bit object arm64
000004.o: Mach-O 64-bit object arm64
000005.o: Mach-O 64-bit object arm64
000006.o: Mach-O 64-bit object arm64
000007.o: Mach-O 64-bit object arm64
000008.o: Mach-O 64-bit object arm64
000009.o: Mach-O 64-bit object arm64
000010.o: Mach-O 64-bit object arm64
000011.o: Mach-O 64-bit object arm64
000012.o: Mach-O 64-bit object arm64
go.o:     Mach-O 64-bit object arm64

By the way, could you tell us how to use IOS.a?

[JatosDetking]

I'm currently working on a project that involves integrating voice activity detection (VAD) using Go and Swift. Below is a sample of my Go code for the VAD functionality:

package main

import ( "C" "bytes" "encoding/binary" "log" "os" "strings"

"streaming-decode-files/speexdsp"

sherpa "github.com/k2-fsa/sherpa-onnx-go/sherpa_onnx"
flag "github.com/spf13/pflag"
"github.com/youpy/go-wav"

) import ( "fmt"

"github.com/gordonklaus/portaudio"

)

//export testVad func testVad() { log.SetFlags(log.LstdFlags | log.Lmicroseconds)

config := sherpa.VadModelConfig{}

// Please download silero_vad.onnx from
// https://github.com/snakers4/silero-vad/raw/master/src/silero_vad/data/silero_vad.onnx

config.SileroVad.Model = "./silero_vad.onnx"
config.SileroVad.Threshold = 0.5
config.SileroVad.MinSilenceDuration = 0.04
config.SileroVad.MinSpeechDuration = 0.04
config.SileroVad.WindowSize = 512
config.SampleRate = 16000
config.NumThreads = 1
config.Provider = "cpu"

var bufferSizeInSeconds float32 = 5

vad := sherpa.NewVoiceActivityDetector(&config, bufferSizeInSeconds)

err := portaudio.Initialize()
if err != nil {
    log.Fatalf("Failed to initialize PortAudio: %v", err)
}

// resampler, err := speexdsp.NewResampler(1, 48000, 16000, 5)
if err != nil {
    log.Fatalf("Failed to initialize resampler: %v", err)
}

stream, err := portaudio.OpenDefaultStream(1, 1, 48000, 1920, func(in, out []int16) {
    noice := make([]int16, 1920, 1920)
    copy(out, noice)

    // start := time.Now()

    outputFloat, err := speexdsp.ResampleSpeex(in, 48000, 16000)
    if err != nil {
        log.Fatal(err)
    }

    vad.AcceptWaveform(outputFloat)

    if vad.IsSpeech() {
        // duration := time.Since(start)
        // fmt.Printf("Execution time is: %v\n", duration)
    }

    for !vad.IsEmpty() {
        speechSegment := vad.Front()
        vad.Pop()

        duration := float32(len(speechSegment.Samples)) / float32(config.SampleRate)
        fmt.Println("Time:", duration, "s")
    }
})

if err != nil {
    log.Fatal(err)
}

err = stream.Start()
if err != nil {
    log.Fatal(err)
}
fmt.Println("playAndRecordsPortAudioVAD")

} When I import the IOS.a library into my Swift application, I encounter several errors.

[csukuangfj]

When I import the IOS.a library into my Swift application, I encounter several errors.

You need to link with

-lc++

[csukuangfj]

Please have a look at https://k2-fsa.github.io/sherpa/ncnn/ios/for-the-more-curious-swift.html

Even if it is for sherpa-ncnn,you can just change ncnn to onnx and everything applies also to sherpa-onnx.

[csukuangfj]

@JatosDetking Does it work for you now?

[JatosDetking]

Yes thanks for IOS works. Can you help me with MACOS form swift build from arm64?

[csukuangfj]

Can you help me with MACOS form swift build from arm64?

Could you describe in detail the issue you have?

Please post the complete logs and commands.