search

mudler / LocalAI

:robot: The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.
https://localai.io
MIT License
21.75k stars 1.66k forks source link

Piper backend no longer working #1936

Closed lunamidori5 closed 3 months ago

lunamidori5 commented 3 months ago

LocalAI version: Master GPU Cuda 12, 11 & Midori AI GPU

Environment, CPU architecture, OS, and Version: Docker VM

Describe the bug Piper Replys with a post 500

To Reproduce Send a piper request to openai on localai

Expected behavior return data to tts program

Logs

9:53AM DBG GRPC Process is not responding: en_US-amy-medium.onnx
9:53AM DBG Loading model in memory from file: /models/en_US-amy-medium.onnx
9:53AM DBG Loading Model en_US-amy-medium.onnx with gRPC (file: /models/en_US-amy-medium.onnx) (backend: piper): {backendString:piper model:en_US-amy-medium.onnx threads:0 assetDir:/tmp/localai/backend_data context:{emptyCtx:{}} gRPCOptions:0xc000038200 externalBackends:map[autogptq:/build/backend/python/autogptq/run.sh bark:/build/backend/python/bark/run.sh coqui:/build/backend/python/coqui/run.sh diffusers:/build/backend/python/diffusers/run.sh exllama:/build/backend/python/exllama/run.sh exllama2:/build/backend/python/exllama2/run.sh huggingface-embeddings:/build/backend/python/sentencetransformers/run.sh mamba:/build/backend/python/mamba/run.sh petals:/build/backend/python/petals/run.sh sentencetransformers:/build/backend/python/sentencetransformers/run.sh transformers:/build/backend/python/transformers/run.sh transformers-musicgen:/build/backend/python/transformers-musicgen/run.sh vall-e-x:/build/backend/python/vall-e-x/run.sh vllm:/build/backend/python/vllm/run.sh] grpcAttempts:20 grpcAttemptsDelay:2 singleActiveBackend:true parallelRequests:false}
9:53AM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/piper
9:53AM DBG GRPC Service for en_US-amy-medium.onnx will be running at: '127.0.0.1:34889'
9:53AM DBG GRPC Service state dir: /tmp/go-processmanager4126426971
9:53AM DBG GRPC Service Started
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr 2024/03/31 09:53:44 gRPC Server listening at 127.0.0.1:34889
9:53AM DBG GRPC Service Ready
9:53AM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:en_US-amy-medium.onnx ContextSize:512 Seed:1803868554 NBatch:512 F16Memory:false MLock:false MMap:true VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:99999999 MainGPU: TensorSplit: Threads:8 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/espeak-ng-data RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0 ModelFile:/models/en_US-amy-medium.onnx Device: UseTriton:false ModelBaseName: UseFastTokenizer:false PipelineType: SchedulerType: CUDA:false CFGScale:0 IMG2IMG:false CLIPModel: CLIPSubfolder: CLIPSkip:0 ControlNet: Tokenizer: LoraBase: LoraAdapter: LoraScale:0 NoMulMatQ:false DraftModel: AudioPath: Quantization: GPUMemoryUtilization:0 TrustRemoteCode:false EnforceEager:false SwapSpace:0 MaxModelLen:0 MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0 Type:}
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr terminate called after throwing an instance of 'nlohmann::json_abi_v3_11_2::detail::parse_error'
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr   what():  [json.exception.parse_error.101] parse error at line 1, column 1: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr SIGABRT: abort
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr PC=0x7f10951199fc m=4 sigcode=18446744073709551610
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr signal arrived during cgo execution
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr 
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr goroutine 23 [syscall]:
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.cgocall(0x7ec520, 0xc0001f3840)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/cgocall.go:157 +0x4b fp=0xc0001f3818 sp=0xc0001f37e0 pc=0x41a9eb
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr github.com/mudler/go-piper._Cfunc_piper_tts(0x7f103c000b70, 0x2d372c0, 0x7f103c000bb0, 0x7f103c000bf0, 0x7f103c000c10)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  _cgo_gotypes.go:89 +0x4b fp=0xc0001f3840 sp=0xc0001f3818 pc=0x7eb56b
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr github.com/mudler/go-piper.TextToWav({0xc0001ac570?, 0x7cd667?}, {0xc0001c0498, 0x15}, {0xc000392040, 0x37}, {0x0, 0x0}, {0xc0001c2440, 0x1e})
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /build/sources/go-piper/piper.go:19 +0xe5 fp=0xc0001f38a0 sp=0xc0001f3840 pc=0x7eb785
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr main.(*PiperB).TTS(...)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /build/backend/go/tts/piper.go:48
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr main.(*Piper).TTS(0x62?, 0x62?)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /build/backend/go/tts/piper.go:31 +0x48 fp=0xc0001f3900 sp=0xc0001f38a0 pc=0x7ebaa8
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr github.com/go-skynet/LocalAI/pkg/grpc.(*server).TTS(0xc000190c90, {0xc000256310?, 0x50eea6?}, 0x0?)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /build/pkg/grpc/server.go:83 +0xe9 fp=0xc0001f39b0 sp=0xc0001f3900 pc=0x7e8349
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr github.com/go-skynet/LocalAI/pkg/grpc/proto._Backend_TTS_Handler({0x950a00?, 0xc000190c90}, {0xa261f0, 0xc000268e70}, 0xc000212680, 0x0)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /build/pkg/grpc/proto/backend_grpc.pb.go:430 +0x169 fp=0xc0001f3a08 sp=0xc0001f39b0 pc=0x7e50c9
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr google.golang.org/grpc.(*Server).processUnaryRPC(0xc0002541e0, {0xa261f0, 0xc000268db0}, {0xa292f8, 0xc000102680}, 0xc0001fb440, 0xc000268960, 0xd47410, 0x0)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/google.golang.org/grpc@v1.59.0/server.go:1343 +0xe03 fp=0xc0001f3df0 sp=0xc0001f3a08 pc=0x7cbe63
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr google.golang.org/grpc.(*Server).handleStream(0xc0002541e0, {0xa292f8, 0xc000102680}, 0xc0001fb440)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/google.golang.org/grpc@v1.59.0/server.go:1737 +0xc4c fp=0xc0001f3f78 sp=0xc0001f3df0 pc=0x7d0dcc
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr google.golang.org/grpc.(*Server).serveStreams.func1.1()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/google.golang.org/grpc@v1.59.0/server.go:986 +0x86 fp=0xc0001f3fe0 sp=0xc0001f3f78 pc=0x7c9d66
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.goexit()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0001f3fe8 sp=0xc0001f3fe0 pc=0x47df61
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr created by google.golang.org/grpc.(*Server).serveStreams.func1 in goroutine 38
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/google.golang.org/grpc@v1.59.0/server.go:997 +0x145
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr 
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr goroutine 1 [IO wait]:
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.gopark(0x4c9fb0?, 0xc00024bb28?, 0x78?, 0xbb?, 0x4e96fd?)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00024bb08 sp=0xc00024bae8 pc=0x44f3ce
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.netpollblock(0x47bfd2?, 0x41a186?, 0x0?)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/netpoll.go:564 +0xf7 fp=0xc00024bb40 sp=0xc00024bb08 pc=0x447e77
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr internal/poll.runtime_pollWait(0x7f104c422e58, 0x72)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/netpoll.go:343 +0x85 fp=0xc00024bb60 sp=0xc00024bb40 pc=0x478e85
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr internal/poll.(*pollDesc).wait(0xc000212500?, 0x4?, 0x0)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc00024bb88 sp=0xc00024bb60 pc=0x4e2367
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr internal/poll.(*pollDesc).waitRead(...)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr internal/poll.(*FD).Accept(0xc000212500)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/internal/poll/fd_unix.go:611 +0x2ac fp=0xc00024bc30 sp=0xc00024bb88 pc=0x4e784c
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr net.(*netFD).accept(0xc000212500)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/net/fd_unix.go:172 +0x29 fp=0xc00024bce8 sp=0xc00024bc30 pc=0x5677a9
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr net.(*TCPListener).accept(0xc0001d03e0)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/net/tcpsock_posix.go:152 +0x1e fp=0xc00024bd10 sp=0xc00024bce8 pc=0x57e77e
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr net.(*TCPListener).Accept(0xc0001d03e0)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/net/tcpsock.go:315 +0x30 fp=0xc00024bd40 sp=0xc00024bd10 pc=0x57d930
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr google.golang.org/grpc.(*Server).Serve(0xc0002541e0, {0xa257e0?, 0xc0001d03e0})
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/google.golang.org/grpc@v1.59.0/server.go:852 +0x462 fp=0xc00024be80 sp=0xc00024bd40 pc=0x7c89c2
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr github.com/go-skynet/LocalAI/pkg/grpc.StartServer({0x7ffc903e1899?, 0xc000194100?}, {0xa2b618?, 0xc000190bc0})
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /build/pkg/grpc/server.go:226 +0x17d fp=0xc00024bf10 sp=0xc00024be80 pc=0x7ea1fd
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr main.main()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /build/backend/go/tts/main.go:18 +0x85 fp=0xc00024bf40 sp=0xc00024bf10 pc=0x7eb8a5
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.main()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/proc.go:267 +0x2bb fp=0xc00024bfe0 sp=0xc00024bf40 pc=0x44ef7b
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.goexit()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00024bfe8 sp=0xc00024bfe0 pc=0x47df61
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr 
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr goroutine 2 [force gc (idle)]:
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000094fa8 sp=0xc000094f88 pc=0x44f3ce
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.goparkunlock(...)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/proc.go:404
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.forcegchelper()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/proc.go:322 +0xb3 fp=0xc000094fe0 sp=0xc000094fa8 pc=0x44f253
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.goexit()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000094fe8 sp=0xc000094fe0 pc=0x47df61
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr created by runtime.init.6 in goroutine 1
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/proc.go:310 +0x1a
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr 
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr goroutine 3 [GC sweep wait]:
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000095778 sp=0xc000095758 pc=0x44f3ce
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.goparkunlock(...)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/proc.go:404
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.bgsweep(0x0?)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/mgcsweep.go:280 +0x94 fp=0xc0000957c8 sp=0xc000095778 pc=0x43b2f4
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.gcenable.func1()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/mgc.go:200 +0x25 fp=0xc0000957e0 sp=0xc0000957c8 pc=0x4304a5
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.goexit()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000957e8 sp=0xc0000957e0 pc=0x47df61
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr created by runtime.gcenable in goroutine 1
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/mgc.go:200 +0x66
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr 
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr goroutine 4 [GC scavenge wait]:
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.gopark(0xc0000be000?, 0xa1eb00?, 0x1?, 0x0?, 0xc0000071e0?)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000095f70 sp=0xc000095f50 pc=0x44f3ce
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.goparkunlock(...)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/proc.go:404
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.(*scavengerState).park(0xd90760)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000095fa0 sp=0xc000095f70 pc=0x438bc9
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.bgscavenge(0x0?)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/mgcscavenge.go:653 +0x3c fp=0xc000095fc8 sp=0xc000095fa0 pc=0x43915c
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.gcenable.func2()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/mgc.go:201 +0x25 fp=0xc000095fe0 sp=0xc000095fc8 pc=0x430445
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.goexit()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000095fe8 sp=0xc000095fe0 pc=0x47df61
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr created by runtime.gcenable in goroutine 1
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/mgc.go:201 +0xa5
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr 
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr goroutine 18 [finalizer wait]:
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.gopark(0x198?, 0x96f160?, 0x1?, 0x5?, 0x0?)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000094620 sp=0xc000094600 pc=0x44f3ce
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.runfinq()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/mfinal.go:193 +0x107 fp=0xc0000947e0 sp=0xc000094620 pc=0x42f4c7
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.goexit()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000947e8 sp=0xc0000947e0 pc=0x47df61
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr created by runtime.createfing in goroutine 1
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/mfinal.go:163 +0x3d
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr 
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr goroutine 36 [select]:
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.gopark(0xc0003e3f00?, 0x2?, 0x1e?, 0x0?, 0xc0003e3ed4?)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc0003e3d80 sp=0xc0003e3d60 pc=0x44f3ce
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.selectgo(0xc0003e3f00, 0xc0003e3ed0, 0x763136?, 0x0, 0xc000324000?, 0x1)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0003e3ea0 sp=0xc0003e3d80 pc=0x45ede5
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0001265a0, 0x1)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/google.golang.org/grpc@v1.59.0/internal/transport/controlbuf.go:418 +0x113 fp=0xc0003e3f30 sp=0xc0003e3ea0 pc=0x741f93
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc000380460)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/google.golang.org/grpc@v1.59.0/internal/transport/controlbuf.go:552 +0x86 fp=0xc0003e3f90 sp=0xc0003e3f30 pc=0x7426a6
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr google.golang.org/grpc/internal/transport.NewServerTransport.func2()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/google.golang.org/grpc@v1.59.0/internal/transport/http2_server.go:336 +0xd5 fp=0xc0003e3fe0 sp=0xc0003e3f90 pc=0x758ef5
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.goexit()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0003e3fe8 sp=0xc0003e3fe0 pc=0x47df61
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr created by google.golang.org/grpc/internal/transport.NewServerTransport in goroutine 35
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/google.golang.org/grpc@v1.59.0/internal/transport/http2_server.go:333 +0x1acc
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr 
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr goroutine 37 [select]:
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.gopark(0xc000090770?, 0x4?, 0xa0?, 0xc5?, 0xc0000906c0?)
[172.24.0.1]:37032 500 - POST /tts
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000090528 sp=0xc000090508 pc=0x44f3ce
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.selectgo(0xc000090770, 0xc0000906b8, 0xf?, 0x0, 0xc000090690?, 0x1)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000090648 sp=0xc000090528 pc=0x45ede5
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr google.golang.org/grpc/internal/transport.(*http2Server).keepalive(0xc000102680)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/google.golang.org/grpc@v1.59.0/internal/transport/http2_server.go:1152 +0x225 fp=0xc0000907c8 sp=0xc000090648 pc=0x7601a5
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr google.golang.org/grpc/internal/transport.NewServerTransport.func4()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/google.golang.org/grpc@v1.59.0/internal/transport/http2_server.go:339 +0x25 fp=0xc0000907e0 sp=0xc0000907c8 pc=0x758de5
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.goexit()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000907e8 sp=0xc0000907e0 pc=0x47df61
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr created by google.golang.org/grpc/internal/transport.NewServerTransport in goroutine 35
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/google.golang.org/grpc@v1.59.0/internal/transport/http2_server.go:339 +0x1b0e
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr 
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr goroutine 38 [IO wait]:
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.gopark(0x100000000?, 0xb?, 0x0?, 0x0?, 0x6?)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc0000a4aa0 sp=0xc0000a4a80 pc=0x44f3ce
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.netpollblock(0x4c8238?, 0x41a186?, 0x0?)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/netpoll.go:564 +0xf7 fp=0xc0000a4ad8 sp=0xc0000a4aa0 pc=0x447e77
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr internal/poll.runtime_pollWait(0x7f104c422d60, 0x72)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/netpoll.go:343 +0x85 fp=0xc0000a4af8 sp=0xc0000a4ad8 pc=0x478e85
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr internal/poll.(*pollDesc).wait(0xc000116380?, 0xc00031c000?, 0x0)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0000a4b20 sp=0xc0000a4af8 pc=0x4e2367
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr internal/poll.(*pollDesc).waitRead(...)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr internal/poll.(*FD).Read(0xc000116380, {0xc00031c000, 0x8000, 0x8000})
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc0000a4bb8 sp=0xc0000a4b20 pc=0x4e365a
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr net.(*netFD).Read(0xc000116380, {0xc00031c000?, 0x1060100000000?, 0x8?})
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/net/fd_posix.go:55 +0x25 fp=0xc0000a4c00 sp=0xc0000a4bb8 pc=0x565785
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr net.(*conn).Read(0xc000144050, {0xc00031c000?, 0x0?, 0xc0000a4cd0?})
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/net/net.go:179 +0x45 fp=0xc0000a4c48 sp=0xc0000a4c00 pc=0x575ea5
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr net.(*TCPConn).Read(0x0?, {0xc00031c000?, 0xc0000a4ca0?, 0x46d28d?})
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  <autogenerated>:1 +0x25 fp=0xc0000a4c78 sp=0xc0000a4c48 pc=0x588645
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr bufio.(*Reader).Read(0xc00010e6c0, {0xc00032c040, 0x9, 0xc17a6a8a9dcb4709?})
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/bufio/bufio.go:244 +0x197 fp=0xc0000a4cb0 sp=0xc0000a4c78 pc=0x52c0f7
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr io.ReadAtLeast({0xa232c0, 0xc00010e6c0}, {0xc00032c040, 0x9, 0x9}, 0x9)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/io/io.go:335 +0x90 fp=0xc0000a4cf8 sp=0xc0000a4cb0 pc=0x4c23f0
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr io.ReadFull(...)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/io/io.go:354
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr golang.org/x/net/http2.readFrameHeader({0xc00032c040, 0x9, 0xc0003a2060?}, {0xa232c0?, 0xc00010e6c0?})
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/golang.org/x/net@v0.22.0/http2/frame.go:237 +0x65 fp=0xc0000a4d48 sp=0xc0000a4cf8 pc=0x72e9e5
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr golang.org/x/net/http2.(*Framer).ReadFrame(0xc00032c000)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/golang.org/x/net@v0.22.0/http2/frame.go:498 +0x85 fp=0xc0000a4df0 sp=0xc0000a4d48 pc=0x72f125
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams(0xc000102680, 0x1?)
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/google.golang.org/grpc@v1.59.0/internal/transport/http2_server.go:636 +0x145 fp=0xc0000a4f00 sp=0xc0000a4df0 pc=0x75c045
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr google.golang.org/grpc.(*Server).serveStreams(0xc0002541e0, {0xa292f8?, 0xc000102680})
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/google.golang.org/grpc@v1.59.0/server.go:979 +0x1c2 fp=0xc0000a4f80 sp=0xc0000a4f00 pc=0x7c9b02
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr google.golang.org/grpc.(*Server).handleRawConn.func1()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/google.golang.org/grpc@v1.59.0/server.go:920 +0x45 fp=0xc0000a4fe0 sp=0xc0000a4f80 pc=0x7c9365
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr runtime.goexit()
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000a4fe8 sp=0xc0000a4fe0 pc=0x47df61
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr created by google.golang.org/grpc.(*Server).handleRawConn in goroutine 35
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr  /root/go/pkg/mod/google.golang.org/grpc@v1.59.0/server.go:919 +0x185
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr 
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr rax    0x0
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr rbx    0x7f104d49b600
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr rcx    0x7f10951199fc
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr rdx    0x6
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr rdi    0x4abc
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr rsi    0x4ac0
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr rbp    0x4ac0
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr rsp    0x7f104d499730
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr r8     0x7f104d499800
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr r9     0x7f104d499340
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr r10    0x8
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr r11    0x246
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr r12    0x6
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr r13    0x16
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr r14    0x7f104d499df0
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr r15    0x1
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr rip    0x7f10951199fc
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr rflags 0x246
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr cs     0x33
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr fs     0x0
9:53AM DBG GRPC(en_US-amy-medium.onnx-127.0.0.1:34889): stderr gs     0x0

Additional context

lunamidori5 commented 3 months ago

have updated every 4 hours and still not fixed

lunamidori5 commented 3 months ago

@mudler