TTS with piper: Error 500, terminate called after throwing an instance of 'nlohmann::json_abi_v3_11_2::detail::parse_error'

[dionysius]

LocalAI version:

REPOSITORY                   TAG                                IMAGE ID       CREATED        SIZE
quay.io/go-skynet/local-ai   v2.4.0-cublas-cuda12-ffmpeg-core   25b0987c5e1c   20 hours ago   11.4GB

Environment, CPU architecture, OS, and Version:

Environment: Docker Desktop 4.26.1 (131620)
Operating System: Windows 10 Pro 64-bit (10.0, Build 19045) (19041.vb_release.191206-1406)
Processor: AMD Ryzen 5 7600X 6-Core Processor              (12 CPUs), ~4.7GHz
Memory: 32768MB RAM
Card name: NVIDIA GeForce RTX 3060 Ti
Dedicated Memory: 8038 MB
Driver Version: 31.0.15.4584

Describe the bug Can't receive audio when requesting /tts endpoint. Errors into a stack trace.

To Reproduce Download en_US-amy-medium.onnx in my case (not one already existing in the model-gallery - the low one is)

curl http://localhost:8080/tts -H "Content-Type: application/json" -d '{
  "model": "en_US-amy-medium.onnx",
  "input": "Hello, world!"
}'
{"error":{"code":500,"message":"rpc error: code = Unavailable desc = error reading from server: EOF","type":""}}

Expected behavior No error and returns some data

Logs

1:36PM INF Loading model 'en_US-amy-medium.onnx' with backend piper
1:36PM DBG Loading model in memory from file: /models/en_US-amy-medium.onnx
1:36PM DBG Loading Model en_US-amy-medium.onnx with gRPC (file: /models/en_US-amy-medium.onnx) (backend: piper): {backendString:piper model:en_US-amy-medium.onnx threads:0 assetDir:/tmp/localai/backend_data context:{emptyCtx:{}} gRPCOptions:0xc0004261e0 externalBackends:map[autogptq:/build/backend/python/autogptq/run.sh bark:/build/backend/python/bark/run.sh coqui:/build/backend/python/coqui/run.sh diffusers:/build/backend/python/diffusers/run.sh exllama:/build/backend/python/exllama/run.sh exllama2:/build/backend/python/exllama2/run.sh huggingface-embeddings:/build/backend/python/sentencetransformers/run.sh petals:/build/backend/python/petals/run.sh sentencetransformers:/build/backend/python/sentencetransformers/run.sh transformers:/build/backend/python/transformers/run.sh transformers-musicgen:/build/backend/python/transformers-musicgen/run.sh vall-e-x:/build/backend/python/vall-e-x/run.sh vllm:/build/backend/python/vllm/run.sh] grpcAttempts:20 grpcAttemptsDelay:2 singleActiveBackend:false parallelRequests:false}
1:36PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/piper
1:36PM DBG GRPC Service for en_US-amy-medium.onnx will be running at: '127.0.0.1:42107'
1:36PM DBG GRPC Service state dir: /tmp/go-processmanager3111906216
1:36PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:42107: connect: connection refused"
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr 2024/01/05 13:36:15 gRPC Server listening at 127.0.0.1:42107
1:36PM DBG GRPC Service Ready
1:36PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:en_US-amy-medium.onnx ContextSize:0 Seed:0 NBatch:0 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:0 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/espeak-ng-data RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0 ModelFile:/models/en_US-amy-medium.onnx Device: UseTriton:false ModelBaseName: UseFastTokenizer:false PipelineType: SchedulerType: CUDA:false CFGScale:0 IMG2IMG:false CLIPModel: CLIPSubfolder: CLIPSkip:0 ControlNet: Tokenizer: LoraBase: LoraAdapter: LoraScale:0 NoMulMatQ:false DraftModel: AudioPath: Quantization: MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0}
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr terminate called after throwing an instance of 'nlohmann::json_abi_v3_11_2::detail::parse_error'
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr   what():  [json.exception.parse_error.101] parse error at line 1, column 1: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr SIGABRT: abort
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr PC=0x7f6d0de8cce1 m=4 sigcode=18446744073709551610
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr signal arrived during cgo execution
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr goroutine 34 [syscall]:
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.cgocall(0x84a2a0, 0xc000121840)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/cgocall.go:157 +0x4b fp=0xc000121818 sp=0xc0001217e0 pc=0x41a44b
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr github.com/mudler/go-piper._Cfunc_piper_tts(0x7f6cb0000cb0, 0x7f6cb0000cd0, 0x7f6cb0000d00, 0x7f6cb0000d40, 0x7f6cb0000d60)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  _cgo_gotypes.go:89 +0x4b fp=0xc000121840 sp=0xc000121818 pc=0x84956b
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr github.com/mudler/go-piper.TextToWav({0xc00031a080?, 0x8280a7?}, {0xc000308020, 0x1d}, {0xc000030640, 0x37}, {0x0, 0x0}, {0xc000308040, 0x1e})
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /build/sources/go-piper/piper.go:19 +0xe5 fp=0xc0001218a0 sp=0xc000121840 pc=0x849785
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr main.(*PiperB).TTS(...)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /build/backend/go/tts/piper.go:48
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr main.(*Piper).TTS(0x4e?, 0x4e?)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /build/backend/go/tts/piper.go:31 +0x48 fp=0xc000121900 sp=0xc0001218a0 pc=0x849aa8
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr github.com/go-skynet/LocalAI/pkg/grpc.(*server).TTS(0xc000036ec0, {0xc000326060?, 0x50f506?}, 0x0?)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /build/pkg/grpc/server.go:83 +0xe6 fp=0xc0001219b0 sp=0xc000121900 pc=0x840aa6
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr github.com/go-skynet/LocalAI/pkg/grpc/proto._Backend_TTS_Handler({0x9a6c00?, 0xc000036ec0}, {0xa939d0, 0xc00030c2d0}, 0xc00030e080, 0x0)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /build/pkg/grpc/proto/backend_grpc.pb.go:357 +0x169 fp=0xc000121a08 sp=0xc0001219b0 pc=0x83e5a9
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr google.golang.org/grpc.(*Server).processUnaryRPC(0xc0001be1e0, {0xa939d0, 0xc00030c210}, {0xa96b98, 0xc00019a340}, 0xc00032a000, 0xc0001c0ea0, 0xde76f0, 0x0)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/google.golang.org/grpc@v1.59.0/server.go:1343 +0xe03 fp=0xc000121df0 sp=0xc000121a08 pc=0x8268a3
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr google.golang.org/grpc.(*Server).handleStream(0xc0001be1e0, {0xa96b98, 0xc00019a340}, 0xc00032a000)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/google.golang.org/grpc@v1.59.0/server.go:1737 +0xc4c fp=0xc000121f78 sp=0xc000121df0 pc=0x82b80c
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr google.golang.org/grpc.(*Server).serveStreams.func1.1()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/google.golang.org/grpc@v1.59.0/server.go:986 +0x86 fp=0xc000121fe0 sp=0xc000121f78 pc=0x8247a6
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.goexit()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000121fe8 sp=0xc000121fe0 pc=0x47d7a1
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr created by google.golang.org/grpc.(*Server).serveStreams.func1 in goroutine 22
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/google.golang.org/grpc@v1.59.0/server.go:997 +0x145
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr goroutine 1 [IO wait]:
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.gopark(0x4c99b0?, 0xc0001afb28?, 0x78?, 0xfb?, 0x4e9d5d?)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc0001afb08 sp=0xc0001afae8 pc=0x44ebee
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.netpollblock(0x47b812?, 0x419be6?, 0x0?)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/netpoll.go:564 +0xf7 fp=0xc0001afb40 sp=0xc0001afb08 pc=0x447697
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr internal/poll.runtime_pollWait(0x7f6cc51f26e0, 0x72)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/netpoll.go:343 +0x85 fp=0xc0001afb60 sp=0xc0001afb40 pc=0x4786c5
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr internal/poll.(*pollDesc).wait(0xc000182780?, 0x4?, 0x0)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0001afb88 sp=0xc0001afb60 pc=0x4e29c7
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr internal/poll.(*pollDesc).waitRead(...)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr internal/poll.(*FD).Accept(0xc000182780)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/internal/poll/fd_unix.go:611 +0x2ac fp=0xc0001afc30 sp=0xc0001afb88 pc=0x4e7eac
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr net.(*netFD).accept(0xc000182780)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/net/fd_unix.go:172 +0x29 fp=0xc0001afce8 sp=0xc0001afc30 pc=0x55d9c9
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr net.(*TCPListener).accept(0xc00007a4c0)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/net/tcpsock_posix.go:152 +0x1e fp=0xc0001afd10 sp=0xc0001afce8 pc=0x574b5e
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr net.(*TCPListener).Accept(0xc00007a4c0)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/net/tcpsock.go:315 +0x30 fp=0xc0001afd40 sp=0xc0001afd10 pc=0x573d10
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr google.golang.org/grpc.(*Server).Serve(0xc0001be1e0, {0xa92f88?, 0xc00007a4c0})
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/google.golang.org/grpc@v1.59.0/server.go:852 +0x462 fp=0xc0001afe80 sp=0xc0001afd40 pc=0x823402
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr github.com/go-skynet/LocalAI/pkg/grpc.StartServer({0x7fffaa19da85?, 0xc000024160?}, {0xa97680?, 0xc000036df0})
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /build/pkg/grpc/server.go:178 +0x17d fp=0xc0001aff10 sp=0xc0001afe80 pc=0x841c3d 1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr main.main()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /build/backend/go/tts/main.go:18 +0x85 fp=0xc0001aff40 sp=0xc0001aff10 pc=0x8498a5
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.main()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/proc.go:267 +0x2bb fp=0xc0001affe0 sp=0xc0001aff40 pc=0x44e79b
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.goexit()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0001affe8 sp=0xc0001affe0 pc=0x47d7a1
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr goroutine 2 [force gc (idle)]:
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00005afa8 sp=0xc00005af88 pc=0x44ebee
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.goparkunlock(...)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/proc.go:404
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.forcegchelper()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/proc.go:322 +0xb3 fp=0xc00005afe0 sp=0xc00005afa8 pc=0x44ea73
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.goexit()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00005afe8 sp=0xc00005afe0 pc=0x47d7a1
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr created by runtime.init.6 in goroutine 1
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/proc.go:310 +0x1a
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr goroutine 3 [GC sweep wait]:
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00005b778 sp=0xc00005b758 pc=0x44ebee
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.goparkunlock(...)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/proc.go:404
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.bgsweep(0x0?)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/mgcsweep.go:280 +0x94 fp=0xc00005b7c8 sp=0xc00005b778 pc=0x43ab14
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.gcenable.func1()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/mgc.go:200 +0x25 fp=0xc00005b7e0 sp=0xc00005b7c8 pc=0x42fcc5
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.goexit()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00005b7e8 sp=0xc00005b7e0 pc=0x47d7a1
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr created by runtime.gcenable in goroutine 1
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/mgc.go:200 +0x66
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr goroutine 4 [GC scavenge wait]:
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.gopark(0xc000034070?, 0xa8bcd8?, 0x1?, 0x0?, 0xc0000071e0?)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00005bf70 sp=0xc00005bf50 pc=0x44ebee
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.goparkunlock(...)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/proc.go:404
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.(*scavengerState).park(0xe324e0)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc00005bfa0 sp=0xc00005bf70 pc=0x4383e9
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.bgscavenge(0x0?)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/mgcscavenge.go:653 +0x3c fp=0xc00005bfc8 sp=0xc00005bfa0 pc=0x43897c
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.gcenable.func2()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/mgc.go:201 +0x25 fp=0xc00005bfe0 sp=0xc00005bfc8 pc=0x42fc65
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.goexit()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00005bfe8 sp=0xc00005bfe0 pc=0x47d7a1
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr created by runtime.gcenable in goroutine 1
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/mgc.go:201 +0xa5
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr goroutine 5 [finalizer wait]:
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.gopark(0x198?, 0x9d3280?, 0x1?, 0xfd?, 0x0?)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00005a620 sp=0xc00005a600 pc=0x44ebee
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.runfinq()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/mfinal.go:193 +0x107 fp=0xc00005a7e0 sp=0xc00005a620 pc=0x42ece7
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.goexit()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00005a7e8 sp=0xc00005a7e0 pc=0x47d7a1
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr created by runtime.createfing in goroutine 1
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/mfinal.go:163 +0x3d
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr goroutine 20 [select]:
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.gopark(0xc000317f00?, 0x2?, 0x1e?, 0x0?, 0xc000317ed4?)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000317d80 sp=0xc000317d60 pc=0x44ebee
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.selectgo(0xc000317f00, 0xc000317ed0, 0x7bdb76?, 0x0, 0xc000240000?, 0x1)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000317ea0 sp=0xc000317d80 pc=0x45e645
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc00020c280, 0x1) 1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/google.golang.org/grpc@v1.59.0/internal/transport/controlbuf.go:418 +0x113 fp=0xc000317f30 sp=0xc000317ea0 pc=0x79c9d3
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc000306070)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/google.golang.org/grpc@v1.59.0/internal/transport/controlbuf.go:552 +0x86 fp=0xc000317f90 sp=0xc000317f30 pc=0x79d0e6
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr google.golang.org/grpc/internal/transport.NewServerTransport.func2()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/google.golang.org/grpc@v1.59.0/internal/transport/http2_server.go:336 +0xd5 fp=0xc000317fe0 sp=0xc000317f90 pc=0x7b3935
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.goexit()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000317fe8 sp=0xc000317fe0 pc=0x47d7a1
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr created by google.golang.org/grpc/internal/transport.NewServerTransport in goroutine 19
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/google.golang.org/grpc@v1.59.0/internal/transport/http2_server.go:333 +0x1acc
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr goroutine 21 [select]:
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.gopark(0xc00005cf70?, 0x4?, 0x20?, 0x84?, 0xc00005cec0?)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00005cd28 sp=0xc00005cd08 pc=0x44ebee
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.selectgo(0xc00005cf70, 0xc00005ceb8, 0x0?, 0x0, 0x0?, 0x1)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc00005ce48 sp=0xc00005cd28 pc=0x45e645
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr google.golang.org/grpc/internal/transport.(*http2Server).keepalive(0xc00019a340)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/google.golang.org/grpc@v1.59.0/internal/transport/http2_server.go:1152 +0x225 fp=0xc00005cfc8 sp=0xc00005ce48 pc=0x7babe5
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr google.golang.org/grpc/internal/transport.NewServerTransport.func4()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/google.golang.org/grpc@v1.59.0/internal/transport/http2_server.go:339 +0x25 fp=0xc00005cfe0 sp=0xc00005cfc8 pc=0x7b3825
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.goexit()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00005cfe8 sp=0xc00005cfe0 pc=0x47d7a1
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr created by google.golang.org/grpc/internal/transport.NewServerTransport in goroutine 19
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/google.golang.org/grpc@v1.59.0/internal/transport/http2_server.go:339 +0x1b0e
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr goroutine 22 [IO wait]:
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.gopark(0xe4a640?, 0xb?, 0x0?, 0x0?, 0x6?)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00006baa0 sp=0xc00006ba80 pc=0x44ebee
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.netpollblock(0x4c7c38?, 0x419be6?, 0x0?)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/netpoll.go:564 +0xf7 fp=0xc00006bad8 sp=0xc00006baa0 pc=0x447697
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr internal/poll.runtime_pollWait(0x7f6cc51f25e8, 0x72)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/netpoll.go:343 +0x85 fp=0xc00006baf8 sp=0xc00006bad8 pc=0x4786c5
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr internal/poll.(*pollDesc).wait(0xc00021a080?, 0xc000238000?, 0x0)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc00006bb20 sp=0xc00006baf8 pc=0x4e29c7
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr internal/poll.(*pollDesc).waitRead(...)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr internal/poll.(*FD).Read(0xc00021a080, {0xc000238000, 0x8000, 0x8000})
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc00006bbb8 sp=0xc00006bb20 pc=0x4e3cba
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr net.(*netFD).Read(0xc00021a080, {0xc000238000?, 0x1060100000000?, 0x8?})
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/net/fd_posix.go:55 +0x25 fp=0xc00006bc00 sp=0xc00006bbb8 pc=0x55b9a5
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr net.(*conn).Read(0xc00020e008, {0xc000238000?, 0x0?, 0xc00006bcd0?})
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/net/net.go:179 +0x45 fp=0xc00006bc48 sp=0xc00006bc00 pc=0x56c285
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr net.(*TCPConn).Read(0x0?, {0xc000238000?, 0xc00006bca0?, 0x46caed?})
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  <autogenerated>:1 +0x25 fp=0xc00006bc78 sp=0xc00006bc48 pc=0x57ea25
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr bufio.(*Reader).Read(0xc000208300, {0xc000248040, 0x9, 0xc15e1f547adb90f9?})
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/bufio/bufio.go:244 +0x197 fp=0xc00006bcb0 sp=0xc00006bc78 pc=0x5ac497
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr io.ReadAtLeast({0xa906a0, 0xc000208300}, {0xc000248040, 0x9, 0x9}, 0x9)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/io/io.go:335 +0x90 fp=0xc00006bcf8 sp=0xc00006bcb0 pc=0x4c1d10 1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr io.ReadFull(...)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/io/io.go:354
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr golang.org/x/net/http2.readFrameHeader({0xc000248040, 0x9, 0xc00002a6a8?}, {0xa906a0?, 0xc000208300?})
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/golang.org/x/net@v0.17.0/http2/frame.go:237 +0x65 fp=0xc00006bd48 sp=0xc00006bcf8 pc=0x789445
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr golang.org/x/net/http2.(*Framer).ReadFrame(0xc000248000)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/golang.org/x/net@v0.17.0/http2/frame.go:498 +0x85 fp=0xc00006bdf0 sp=0xc00006bd48 pc=0x789b85
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams(0xc00019a340, 0x1?)
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/google.golang.org/grpc@v1.59.0/internal/transport/http2_server.go:636 +0x145 fp=0xc00006bf00 sp=0xc00006bdf0 pc=0x7b6a85
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr google.golang.org/grpc.(*Server).serveStreams(0xc0001be1e0, {0xa96b98?, 0xc00019a340})
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/google.golang.org/grpc@v1.59.0/server.go:979 +0x1c2 fp=0xc00006bf80 sp=0xc00006bf00 pc=0x824542
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr google.golang.org/grpc.(*Server).handleRawConn.func1()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/google.golang.org/grpc@v1.59.0/server.go:920 +0x45 fp=0xc00006bfe0 sp=0xc00006bf80 pc=0x823da5
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr runtime.goexit()
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00006bfe8 sp=0xc00006bfe0 pc=0x47d7a1
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr created by google.golang.org/grpc.(*Server).handleRawConn in goroutine 19
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr  /go/pkg/mod/google.golang.org/grpc@v1.59.0/server.go:919 +0x185
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr rax    0x0
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr rbx    0x7f6cc6a6e700
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr rcx    0x7f6d0de8cce1
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr rdx    0x0
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr rdi    0x2
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr rsi    0x7f6cc6a6c7f0
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr rbp    0x7f6cb031ce48
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr rsp    0x7f6cc6a6c7f0
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr r8     0x0
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr r9     0x7f6cc6a6c7f0
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr r10    0x8
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr r11    0x246
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr r12    0x7f6cb031cd30
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr r13    0x8c8f70
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr r14    0x7f6cc6a6d088
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr r15    0x1
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr rip    0x7f6d0de8cce1
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr rflags 0x246
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr cs     0x33
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr fs     0x0
1:36PM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:42107): stderr gs     0x0
[172.17.0.1]:52338 500 - POST /tts

Additional context

• I might use the endpoint wrong, but that is what I figured out from https://localai.io/features/text-to-audio/.
• May be a docker image bug.
• Also having this issue with the low quality model from the gallery: https://github.com/go-skynet/model-gallery/blob/main/index.yaml#L132 edit: did not correctly test that (filename changed more than just medium/low)

[dionysius]

Oooooh: According to piper

You will need two files per voice: A .onnx model file, such as en_US-lessac-medium.onnx A .onnx.json config file, such as en_US-lessac-medium.onnx.json

So when downloading the models from huggingface, download both.

Followup: Same issue or new issue?

• [ ] add missing voices in the gallery? e.g. en_US-amy-medium was missing
• [ ] document piper model requirements when manually downloading? Needs the .onnx and the .onnx.json file at least and is nowhere documented inside localai
• [ ] Shouldn't it be intended to use the yaml's name as model instead of the filename & backend for the /tss endpoint? Since I assume the /models endpoint is the intended way to organise the download and configuration of models. This way we could omit the backend parameter from /tts and load it prom the parameters in the yaml file. Related: https://github.com/mudler/LocalAI/issues/1276

Example body payload for /models/apply:

{
  "name": "some-name",
  "url": "github:go-skynet/model-gallery/virtual.yaml",
  "files": [
    {
      "filename": "en_US-amy-medium.onnx",
      "sha256": "",
      "uri": "https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/amy/medium/en_US-amy-medium.onnx"
    },
    {
      "filename": "en_US-amy-medium.onnx.json",
      "sha256": "",
      "uri": "https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/amy/medium/en_US-amy-medium.onnx.json"
    }
  ],
  "overrides": {
    "backend": "piper",
    "parameters": {
      "model": "en_US-amy-medium.onnx"
    }
  }
}

[dionysius]

Examples got added for this as well #1556 and issue itself already resoved