search

mudler / LocalAI

:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed inference
https://localai.io
MIT License
24.37k stars 1.87k forks source link

When using the bert embedding model, the process crashes when passing a certain string #780

Open zhengxiang5965 opened 1 year ago

zhengxiang5965 commented 1 year ago

LocalAI version:

V1.20.1 Environment, CPU architecture, OS, and Version:

Not limited to specific CPUs Describe the bug

When using the bert embeddings model, when passing a certain string, the process will flash back and no error message will be seen To Reproduce

1. curl --location 'http://localhost:8080/models/apply' \ --header 'Content-Type: application/json' \ --data '{ "url": "github:go-skynet/model-gallery/bert-embeddings.yaml" }' 2. curl --location 'http://localhost:8080/v1/embeddings' \ --header 'Content-Type: application/json' \ --data '{"model":"bert-embeddings","input":["A2A2A2A2A2A2A21221222F2o2c2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2V2321202A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A222022212A21212A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A22212V2-2F2O2C2-2M2O2-2020232A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A","A2A2A2A2A2A2A22221222F2o2c2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A2A212A2A2A2A2V2121202V2221202A2A2A2A222021292A292A222A2A2A2A2A2A2A2A2A2A2A2A2A2A2a2A2A2A2A2A2A2A22A2A2A2A2A2A2A2A2A2A2A2A2A2V2221202V2321202A2A2A2A222022212A21212AA","A2A2A2A2A2A2A23221222F2o2c2A2A2A2A2A2A2A2A21212A2A212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212422212A2A2A2A21212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212423212A2A2A2A2A2A212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212121212a"]}'

Expected behavior

Return the correct result Logs

2023-07-20 16:51:35 bert_load_from_file: loading model from '/opt/bert-MiniLM-L6-v2q4_0.bin' - please wait ... 2023-07-20 16:51:35 bert_load_from_file: n_vocab = 30522 2023-07-20 16:51:35 bert_load_from_file: n_max_tokens = 512 2023-07-20 16:51:35 bert_load_from_file: n_embd = 384 2023-07-20 16:51:35 bert_load_from_file: n_intermediate = 1536 2023-07-20 16:51:35 bert_load_from_file: n_head = 12 2023-07-20 16:51:35 bert_load_from_file: n_layer = 6 2023-07-20 16:51:35 bert_load_from_file: f16 = 2 2023-07-20 16:51:35 bert_load_from_file: ggml ctx size = 13.57 MB 2023-07-20 16:51:35 fatal error: unexpected signal during runtime execution 2023-07-20 16:51:35 [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xc03248] 2023-07-20 16:51:35 2023-07-20 16:51:35 runtime stack: 2023-07-20 16:51:35 runtime.throw({0xe1c3d8?, 0x3500000014?}) 2023-07-20 16:51:35 /usr/local/go/src/runtime/panic.go:1047 +0x5d fp=0x7f4254fe0858 sp=0x7f4254fe0828 pc=0x47ba7d 2023-07-20 16:51:35 runtime.sigpanic() 2023-07-20 16:51:35 /usr/local/go/src/runtime/signal_unix.go:825 +0x3e9 fp=0x7f4254fe08b8 sp=0x7f4254fe0858 pc=0x491f29 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 27 [syscall]: 2023-07-20 16:51:35 runtime.cgocall(0xb15010, 0xc00012ce48) 2023-07-20 16:51:35 /usr/local/go/src/runtime/cgocall.go:157 +0x5c fp=0xc00012ce20 sp=0xc00012cde8 pc=0x44a85c 2023-07-20 16:51:35 github.com/go-skynet/go-bert%2ecpp._Cfunc_bert_embeddings(0x7f4230001ef0, 0x7f4230000b60, 0xc0004c4000) 2023-07-20 16:51:35 _cgo_gotypes.go:127 +0x4c fp=0xc00012ce48 sp=0xc00012ce20 pc=0x8e1e6c 2023-07-20 16:51:35 github.com/go-skynet/go-bert%2ecpp.(Bert).Embeddings.func1(0x7f4230595300?, 0x4?, 0x7f4230001ef0?) 2023-07-20 16:51:35 /build/go-bert/gobert.go:38 +0x74 fp=0xc00012ce90 sp=0xc00012ce48 pc=0x8e2634 2023-07-20 16:51:35 github.com/go-skynet/go-bert%2ecpp.(Bert).Embeddings(0x0?, {0xc000164000, 0x356}, {0xc00012cf50, 0x1, 0x0?}) 2023-07-20 16:51:35 /build/go-bert/gobert.go:38 +0x115 fp=0xc00012cef8 sp=0xc00012ce90 pc=0x8e2435 2023-07-20 16:51:35 github.com/go-skynet/LocalAI/api.ModelEmbedding.func2() 2023-07-20 16:51:35 /build/api/prediction.go:137 +0xd5 fp=0xc00012cf68 sp=0xc00012cef8 pc=0xaa02d5 2023-07-20 16:51:35 github.com/go-skynet/LocalAI/api.ModelEmbedding.func4() 2023-07-20 16:51:35 /build/api/prediction.go:158 +0x18b fp=0xc00012d000 sp=0xc00012cf68 pc=0xaa006b 2023-07-20 16:51:35 github.com/go-skynet/LocalAI/api.embeddingsEndpoint.func1(0xc0000ee580?) 2023-07-20 16:51:35 /build/api/openai.go:326 +0x4f6 fp=0xc00012d9f0 sp=0xc00012d000 pc=0xa9ad96 2023-07-20 16:51:35 github.com/gofiber/fiber/v2.(App).next(0xc0000f3680, 0xc0000ee580) 2023-07-20 16:51:35 /go/pkg/mod/github.com/gofiber/fiber/v2@v2.47.0/router.go:144 +0x1bf fp=0xc00012da98 sp=0xc00012d9f0 pc=0x8aef9f 2023-07-20 16:51:35 github.com/gofiber/fiber/v2.(Ctx).Next(0xe?) 2023-07-20 16:51:35 /go/pkg/mod/github.com/gofiber/fiber/v2@v2.47.0/ctx.go:913 +0x53 fp=0xc00012dab8 sp=0xc00012da98 pc=0x8999f3 2023-07-20 16:51:35 github.com/gofiber/fiber/v2/middleware/recover.New.func1(0xd474a0?) 2023-07-20 16:51:35 /go/pkg/mod/github.com/gofiber/fiber/v2@v2.47.0/middleware/recover/recover.go:43 +0xcb fp=0xc00012db30 sp=0xc00012dab8 pc=0x8bbe0b 2023-07-20 16:51:35 github.com/gofiber/fiber/v2.(App).next(0xc0000f3680, 0xc0000ee580) 2023-07-20 16:51:35 /go/pkg/mod/github.com/gofiber/fiber/v2@v2.47.0/router.go:144 +0x1bf fp=0xc00012dbd8 sp=0xc00012db30 pc=0x8aef9f 2023-07-20 16:51:35 github.com/gofiber/fiber/v2.(App).handler(0xc0000f3680, 0x4d2e57?) 2023-07-20 16:51:35 /go/pkg/mod/github.com/gofiber/fiber/v2@v2.47.0/router.go:171 +0x87 fp=0xc00012dc38 sp=0xc00012dbd8 pc=0x8af1e7 2023-07-20 16:51:35 github.com/gofiber/fiber/v2.(App).handler-fm(0xc000142000?) 2023-07-20 16:51:35 :1 +0x2c fp=0xc00012dc58 sp=0xc00012dc38 pc=0x8b440c 2023-07-20 16:51:35 github.com/valyala/fasthttp.(Server).serveConn(0xc000360200, {0x15eff18?, 0xc000014048}) 2023-07-20 16:51:35 /go/pkg/mod/github.com/valyala/fasthttp@v1.48.0/server.go:2363 +0x11d3 fp=0xc00012dec8 sp=0xc00012dc58 pc=0x811633 2023-07-20 16:51:35 github.com/valyala/fasthttp.(Server).serveConn-fm({0x15eff18?, 0xc000014048?}) 2023-07-20 16:51:35 :1 +0x39 fp=0xc00012def0 sp=0xc00012dec8 pc=0x820ef9 2023-07-20 16:51:35 github.com/valyala/fasthttp.(workerPool).workerFunc(0xc000040d20, 0xc000140000) 2023-07-20 16:51:35 /go/pkg/mod/github.com/valyala/fasthttp@v1.48.0/workerpool.go:224 +0xa9 fp=0xc00012dfa0 sp=0xc00012def0 pc=0x81d129 2023-07-20 16:51:35 github.com/valyala/fasthttp.(workerPool).getCh.func1() 2023-07-20 16:51:35 /go/pkg/mod/github.com/valyala/fasthttp@v1.48.0/workerpool.go:196 +0x38 fp=0xc00012dfe0 sp=0xc00012dfa0 pc=0x81ce98 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00012dfe8 sp=0xc00012dfe0 pc=0x4ad9a1 2023-07-20 16:51:35 created by github.com/valyala/fasthttp.(workerPool).getCh 2023-07-20 16:51:35 /go/pkg/mod/github.com/valyala/fasthttp@v1.48.0/workerpool.go:195 +0x1b0 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 1 [IO wait]: 2023-07-20 16:51:35 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc0001311d8 sp=0xc0001311b8 pc=0x47e7d6 2023-07-20 16:51:35 runtime.netpollblock(0x45d653?, 0x449eef?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc000131210 sp=0xc0001311d8 pc=0x476fd7 2023-07-20 16:51:35 internal/poll.runtime_pollWait(0x7f425c33e618, 0x72) 2023-07-20 16:51:35 /usr/local/go/src/runtime/netpoll.go:306 +0x89 fp=0xc000131230 sp=0xc000131210 pc=0x4a8249 2023-07-20 16:51:35 internal/poll.(pollDesc).wait(0xc000190480?, 0x46da20?, 0x0) 2023-07-20 16:51:35 /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc000131258 sp=0xc000131230 pc=0x525a92 2023-07-20 16:51:35 internal/poll.(pollDesc).waitRead(...) 2023-07-20 16:51:35 /usr/local/go/src/internal/poll/fd_poll_runtime.go:89 2023-07-20 16:51:35 internal/poll.(FD).Accept(0xc000190480) 2023-07-20 16:51:35 /usr/local/go/src/internal/poll/fd_unix.go:614 +0x2bd fp=0xc000131300 sp=0xc000131258 pc=0x52b39d 2023-07-20 16:51:35 net.(netFD).accept(0xc000190480) 2023-07-20 16:51:35 /usr/local/go/src/net/fd_unix.go:172 +0x35 fp=0xc0001313b8 sp=0xc000131300 pc=0x5ad855 2023-07-20 16:51:35 net.(TCPListener).accept(0xc000012660) 2023-07-20 16:51:35 /usr/local/go/src/net/tcpsock_posix.go:148 +0x25 fp=0xc0001313e0 sp=0xc0001313b8 pc=0x5c3c05 2023-07-20 16:51:35 net.(TCPListener).Accept(0xc000012660) 2023-07-20 16:51:35 /usr/local/go/src/net/tcpsock.go:297 +0x3d fp=0xc000131410 sp=0xc0001313e0 pc=0x5c2cfd 2023-07-20 16:51:35 github.com/valyala/fasthttp.acceptConn(0xc000360200, {0x15ec830, 0xc000012660}, 0xc000131608) 2023-07-20 16:51:35 /go/pkg/mod/github.com/valyala/fasthttp@v1.48.0/server.go:1928 +0x62 fp=0xc0001314f0 sp=0xc000131410 pc=0x80fb02 2023-07-20 16:51:35 github.com/valyala/fasthttp.(Server).Serve(0xc000360200, {0x15ec830?, 0xc000012660}) 2023-07-20 16:51:35 /go/pkg/mod/github.com/valyala/fasthttp@v1.48.0/server.go:1821 +0x4f4 fp=0xc000131638 sp=0xc0001314f0 pc=0x80f114 2023-07-20 16:51:35 github.com/gofiber/fiber/v2.(App).Listen(0xc0000f3680, {0xdf9ccd?, 0x7?}) 2023-07-20 16:51:35 /go/pkg/mod/github.com/gofiber/fiber/v2@v2.47.0/listen.go:88 +0x11d fp=0xc000131698 sp=0xc000131638 pc=0x8a5ffd 2023-07-20 16:51:35 main.main.func1(0xc000362160?) 2023-07-20 16:51:35 /build/main.go:161 +0x825 fp=0xc000131950 sp=0xc000131698 pc=0xad4de5 2023-07-20 16:51:35 github.com/urfave/cli/v2.(Command).Run(0xc000362160, 0xc0000aaf40, {0xc000036180, 0x3, 0x3}) 2023-07-20 16:51:35 /go/pkg/mod/github.com/urfave/cli/v2@v2.25.7/command.go:274 +0x9eb fp=0xc000131bf0 sp=0xc000131950 pc=0xac1eab 2023-07-20 16:51:35 github.com/urfave/cli/v2.(App).RunContext(0xc00035e000, {0x15ecd38?, 0xc00003a088}, {0xc000036180, 0x3, 0x3}) 2023-07-20 16:51:35 /go/pkg/mod/github.com/urfave/cli/v2@v2.25.7/app.go:332 +0x616 fp=0xc000131c60 sp=0xc000131bf0 pc=0xabe7d6 2023-07-20 16:51:35 github.com/urfave/cli/v2.(App).Run(...) 2023-07-20 16:51:35 /go/pkg/mod/github.com/urfave/cli/v2@v2.25.7/app.go:309 2023-07-20 16:51:35 main.main() 2023-07-20 16:51:35 /build/main.go:165 +0x12b6 fp=0xc000131f80 sp=0xc000131c60 pc=0xad44f6 2023-07-20 16:51:35 runtime.main() 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:250 +0x207 fp=0xc000131fe0 sp=0xc000131f80 pc=0x47e3a7 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000131fe8 sp=0xc000131fe0 pc=0x4ad9a1 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 2 [force gc (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000084fb0 sp=0xc000084f90 pc=0x47e7d6 2023-07-20 16:51:35 runtime.goparkunlock(...) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:387 2023-07-20 16:51:35 runtime.forcegchelper() 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:305 +0xb0 fp=0xc000084fe0 sp=0xc000084fb0 pc=0x47e610 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000084fe8 sp=0xc000084fe0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.init.6 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:293 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 3 [GC sweep wait]: 2023-07-20 16:51:35 runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000085780 sp=0xc000085760 pc=0x47e7d6 2023-07-20 16:51:35 runtime.goparkunlock(...) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:387 2023-07-20 16:51:35 runtime.bgsweep(0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgcsweep.go:319 +0xde fp=0xc0000857c8 sp=0xc000085780 pc=0x46a8de 2023-07-20 16:51:35 runtime.gcenable.func1() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:178 +0x26 fp=0xc0000857e0 sp=0xc0000857c8 pc=0x45fb26 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000857e8 sp=0xc0000857e0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcenable 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:178 +0x6b 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 4 [GC scavenge wait]: 2023-07-20 16:51:35 runtime.gopark(0x2371834184286?, 0x70972?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000085f70 sp=0xc000085f50 pc=0x47e7d6 2023-07-20 16:51:35 runtime.goparkunlock(...) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:387 2023-07-20 16:51:35 runtime.(scavengerState).park(0x1b11220) 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgcscavenge.go:400 +0x53 fp=0xc000085fa0 sp=0xc000085f70 pc=0x468793 2023-07-20 16:51:35 runtime.bgscavenge(0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgcscavenge.go:633 +0x65 fp=0xc000085fc8 sp=0xc000085fa0 pc=0x468d85 2023-07-20 16:51:35 runtime.gcenable.func2() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:179 +0x26 fp=0xc000085fe0 sp=0xc000085fc8 pc=0x45fac6 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000085fe8 sp=0xc000085fe0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcenable 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:179 +0xaa 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 5 [finalizer wait]: 2023-07-20 16:51:35 runtime.gopark(0x1a0?, 0x1b11f00?, 0x60?, 0x78?, 0xc000084770?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000084628 sp=0xc000084608 pc=0x47e7d6 2023-07-20 16:51:35 runtime.runfinq() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mfinal.go:193 +0x107 fp=0xc0000847e0 sp=0xc000084628 pc=0x45eb67 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000847e8 sp=0xc0000847e0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.createfing 2023-07-20 16:51:35 /usr/local/go/src/runtime/mfinal.go:163 +0x45 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 6 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000086750 sp=0xc000086730 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc0000867e0 sp=0xc000086750 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000867e8 sp=0xc0000867e0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 18 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000080750 sp=0xc000080730 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc0000807e0 sp=0xc000080750 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000807e8 sp=0xc0000807e0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 34 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00053e750 sp=0xc00053e730 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc00053e7e0 sp=0xc00053e750 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00053e7e8 sp=0xc00053e7e0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 7 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000086f50 sp=0xc000086f30 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc000086fe0 sp=0xc000086f50 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000086fe8 sp=0xc000086fe0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 19 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000080f50 sp=0xc000080f30 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc000080fe0 sp=0xc000080f50 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000080fe8 sp=0xc000080fe0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 8 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000087750 sp=0xc000087730 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc0000877e0 sp=0xc000087750 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000877e8 sp=0xc0000877e0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 35 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00053ef50 sp=0xc00053ef30 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc00053efe0 sp=0xc00053ef50 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00053efe8 sp=0xc00053efe0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 20 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000081750 sp=0xc000081730 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc0000817e0 sp=0xc000081750 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000817e8 sp=0xc0000817e0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 36 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00053f750 sp=0xc00053f730 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc00053f7e0 sp=0xc00053f750 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00053f7e8 sp=0xc00053f7e0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 21 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000081f50 sp=0xc000081f30 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc000081fe0 sp=0xc000081f50 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000081fe8 sp=0xc000081fe0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 37 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00053ff50 sp=0xc00053ff30 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc00053ffe0 sp=0xc00053ff50 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00053ffe8 sp=0xc00053ffe0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 22 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000082750 sp=0xc000082730 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc0000827e0 sp=0xc000082750 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000827e8 sp=0xc0000827e0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 9 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000087f50 sp=0xc000087f30 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc000087fe0 sp=0xc000087f50 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000087fe8 sp=0xc000087fe0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 23 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000082f50 sp=0xc000082f30 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc000082fe0 sp=0xc000082f50 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000082fe8 sp=0xc000082fe0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 38 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000540750 sp=0xc000540730 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc0005407e0 sp=0xc000540750 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0005407e8 sp=0xc0005407e0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 10 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x237183414cdaa?, 0x0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00053a750 sp=0xc00053a730 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc00053a7e0 sp=0xc00053a750 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00053a7e8 sp=0xc00053a7e0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 24 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x1f70ce0?, 0x1?, 0xff?, 0x7a?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000083750 sp=0xc000083730 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc0000837e0 sp=0xc000083750 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000837e8 sp=0xc0000837e0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 25 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x23718341540b1?, 0x1?, 0xe8?, 0x77?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000083f50 sp=0xc000083f30 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc000083fe0 sp=0xc000083f50 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000083fe8 sp=0xc000083fe0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 11 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x237183412b4a6?, 0x1?, 0x23?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00053af50 sp=0xc00053af30 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc00053afe0 sp=0xc00053af50 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00053afe8 sp=0xc00053afe0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 26 [GC worker (idle)]: 2023-07-20 16:51:35 runtime.gopark(0x2371834148545?, 0x1?, 0x1a?, 0xc7?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000114750 sp=0xc000114730 pc=0x47e7d6 2023-07-20 16:51:35 runtime.gcBgMarkWorker() 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc0001147e0 sp=0xc000114750 pc=0x461891 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0001147e8 sp=0xc0001147e0 pc=0x4ad9a1 2023-07-20 16:51:35 created by runtime.gcBgMarkStartWorkers 2023-07-20 16:51:35 /usr/local/go/src/runtime/mgc.go:1199 +0x25 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 39 [select]: 2023-07-20 16:51:35 runtime.gopark(0xc00053beb0?, 0x2?, 0x0?, 0x0?, 0xc00053be74?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000093c20 sp=0xc000093c00 pc=0x47e7d6 2023-07-20 16:51:35 runtime.selectgo(0xc000093eb0, 0xc00053be70, 0x0?, 0x0, 0x0?, 0x1) 2023-07-20 16:51:35 /usr/local/go/src/runtime/select.go:327 +0x7be fp=0xc000093d60 sp=0xc000093c20 pc=0x48e35e 2023-07-20 16:51:35 github.com/go-skynet/LocalAI/api.(galleryApplier).start.func1() 2023-07-20 16:51:35 /build/api/gallery.go:78 +0xee fp=0xc000093fe0 sp=0xc000093d60 pc=0xa9772e 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000093fe8 sp=0xc000093fe0 pc=0x4ad9a1 2023-07-20 16:51:35 created by github.com/go-skynet/LocalAI/api.(galleryApplier).start 2023-07-20 16:51:35 /build/api/gallery.go:76 +0xaa 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 40 [sleep]: 2023-07-20 16:51:35 runtime.gopark(0x237195e64acef?, 0xc0000cc810?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00053c700 sp=0xc00053c6e0 pc=0x47e7d6 2023-07-20 16:51:35 time.Sleep(0x12a05f200) 2023-07-20 16:51:35 /usr/local/go/src/runtime/time.go:195 +0x135 fp=0xc00053c740 sp=0xc00053c700 pc=0x4aa815 2023-07-20 16:51:35 github.com/valyala/fasthttp.(FS).initRequestHandler.func1() 2023-07-20 16:51:35 /go/pkg/mod/github.com/valyala/fasthttp@v1.48.0/fs.go:482 +0x13c fp=0xc00053c7e0 sp=0xc00053c740 pc=0x7dacfc 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00053c7e8 sp=0xc00053c7e0 pc=0x4ad9a1 2023-07-20 16:51:35 created by github.com/valyala/fasthttp.(FS).initRequestHandler 2023-07-20 16:51:35 /go/pkg/mod/github.com/valyala/fasthttp@v1.48.0/fs.go:459 +0x4d6 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 41 [sleep]: 2023-07-20 16:51:35 runtime.gopark(0x237195e649434?, 0xc0000cc8c0?, 0x0?, 0x0?, 0x0?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00053cf00 sp=0xc00053cee0 pc=0x47e7d6 2023-07-20 16:51:35 time.Sleep(0x12a05f200) 2023-07-20 16:51:35 /usr/local/go/src/runtime/time.go:195 +0x135 fp=0xc00053cf40 sp=0xc00053cf00 pc=0x4aa815 2023-07-20 16:51:35 github.com/valyala/fasthttp.(FS).initRequestHandler.func1() 2023-07-20 16:51:35 /go/pkg/mod/github.com/valyala/fasthttp@v1.48.0/fs.go:482 +0x13c fp=0xc00053cfe0 sp=0xc00053cf40 pc=0x7dacfc 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00053cfe8 sp=0xc00053cfe0 pc=0x4ad9a1 2023-07-20 16:51:35 created by github.com/valyala/fasthttp.(FS).initRequestHandler 2023-07-20 16:51:35 /go/pkg/mod/github.com/valyala/fasthttp@v1.48.0/fs.go:459 +0x4d6 2023-07-20 16:51:35 2023-07-20 16:51:35 goroutine 42 [sleep]: 2023-07-20 16:51:35 runtime.gopark(0x2371834223736?, 0xc00053d788?, 0x65?, 0xdb?, 0xc000040d50?) 2023-07-20 16:51:35 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00053d758 sp=0xc00053d738 pc=0x47e7d6 2023-07-20 16:51:35 time.Sleep(0x2540be400) 2023-07-20 16:51:35 /usr/local/go/src/runtime/time.go:195 +0x135 fp=0xc00053d798 sp=0xc00053d758 pc=0x4aa815 2023-07-20 16:51:35 github.com/valyala/fasthttp.(workerPool).Start.func2() 2023-07-20 16:51:35 /go/pkg/mod/github.com/valyala/fasthttp@v1.48.0/workerpool.go:67 +0x56 fp=0xc00053d7e0 sp=0xc00053d798 pc=0x81c5f6 2023-07-20 16:51:35 runtime.goexit() 2023-07-20 16:51:35 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00053d7e8 sp=0xc00053d7e0 pc=0x4ad9a1 2023-07-20 16:51:35 created by github.com/valyala/fasthttp.(workerPool).Start 2023-07-20 16:51:35 /go/pkg/mod/github.com/valyala/fasthttp@v1.48.0/workerpool.go:59 +0xdd 2023-07-20T08:51:35.885177720Z bert_load_from_file: ............

Additional context

If you delete the last character 'a' in the content of the input field in the request body, the program will not crash

zhengxiang5965 commented 1 year ago

@mudler #715

localai-bot commented 1 year ago

:warning::warning::warning::warning::warning:

Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!

_but.... I can also be funny or helpful :smilecat: and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.

Don't engage in conversation with me, I don't support (yet) replying!

:warning::warning::warning::warning::warning:

Based on the provided context, it seems like there is a bug in the GitHub assistant for the LocalAI open source project related to the handling of requests. The issue may be caused by a race condition or a programming error in the code that handles incoming requests. It's recommended to investigate further and debug the issue to identify the root cause and fix it as soon as possible to avoid any potential security vulnerabilities or performance issues.

Sources:

mudler commented 1 year ago

looks like you are exceeding the prompt size. Did you try with another embedding backend, such as huggingface? https://localai.io/features/embeddings/index.html#huggingface-embeddings

netandreus commented 1 year ago

Hello, @mudler ! I tried it at my MacBook M2 (compiled with "metal").

./models/text-embedding-ada-002.yaml

f16: true
gpu_layers: 1
name: text-embedding-ada-002
backend: huggingface-embeddings
embeddings: true
parameters:
  model: all-MiniLM-L6-v2

Here is output from client side:

(base) andrey@m2 ~ % curl http://localhost:8080/v1/embeddings -H "Content-Type: application/json" -d '{
    "model": "text-embedding-ada-002",
    "input": "Test",
}'
{"error":{"code":500,"message":"failed reading parameters from request:invalid character '}' looking for beginning of object key string","type":""}}%
(base) andrey@m2 ~ % curl http://localhost:8080/v1/embeddings -H "Content-Type: application/json" -d '{
    "model": "text-embedding-ada-002",
    "input": "Test"
}'
{"error":{"code":500,"message":"backend unsupported: huggingface-embeddings","type":""}}%

and from server (localAI) side:

5:49PM DBG Request received:
5:49PM DBG Parameter Config: &{PredictionOptions:{Model:all-MiniLM-L6-v2 Language: N:0 TopP:0 TopK:0 Temperature:0 Maxtokens:0 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:text-embedding-ada-002 F16:true Threads:4 Debug:true Roles:map[] Embeddings:true Backend:huggingface-embeddings TemplateConfig:{Chat: ChatMessage: Completion: Edit: Functions:} PromptStrings:[] InputStrings:[Test] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:1 MMap:false MMlock:false LowVRAM:false Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:0 NUMA:false LoraAdapter: LoraBase: NoMulMatQ:false DraftModel: NDraft:0 Quantization:} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{PipelineType: SchedulerType: CUDA:false EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0} VallE:{AudioPath:}}
5:49PM DBG Loading model huggingface-embeddings from all-MiniLM-L6-v2
[127.0.0.1]:55086 500 - POST /v1/embeddings

I try to make this one:

The huggingface backend is an optional backend of LocalAI and uses Python. If you are running LocalAI from the containers you are good to go and should be already configured for use. If you are running LocalAI manually you must install the python dependencies (pip install -r /path/to/LocalAI/extra/requirements) and specify the extra backend in the EXTERNAL_GRPC_BACKENDS environment variable ( EXTERNAL_GRPC_BACKENDS="huggingface-embeddings:/path/to/LocalAI/extra/grpc/huggingface/huggingface.py" ) .
The huggingface backend does support only embeddings of text, and not of tokens. If you need to embed tokens you can use the bert backend or llama.cpp.
No models are required to be downloaded before using the huggingface backend. The models will be downloaded automatically the first time the API is used.

Now result in server log changed:

6:06PM DBG Request received:
6:06PM DBG Parameter Config: &{PredictionOptions:{Model:all-MiniLM-L6-v2 Language: N:0 TopP:0 TopK:0 Temperature:0 Maxtokens:0 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:text-embedding-ada-002 F16:true Threads:4 Debug:true Roles:map[] Embeddings:true Backend:huggingface-embeddings TemplateConfig:{Chat: ChatMessage: Completion: Edit: Functions:} PromptStrings:[] InputStrings:[Test] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:1 MMap:false MMlock:false LowVRAM:false Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:0 NUMA:false LoraAdapter: LoraBase: NoMulMatQ:false DraftModel: NDraft:0 Quantization:} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{PipelineType: SchedulerType: CUDA:false EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0} VallE:{AudioPath:}}
6:06PM DBG Loading model huggingface-embeddings from all-MiniLM-L6-v2
6:06PM DBG Loading model in memory from file: /Users/andrey/sandbox/llm/current/models/all-MiniLM-L6-v2
6:06PM DBG Loading GRPC Model huggingface-embeddings: {backendString:huggingface-embeddings model:all-MiniLM-L6-v2 threads:4 assetDir:/tmp/localai/backend_data context:{emptyCtx:{}} gRPCOptions:0x140003b89c0 externalBackends:map[huggingface-embeddings:/Users/andrey/sandbox/llm/current/build/local_ai/extra/grpc/huggingface/huggingface.py] grpcAttempts:20 grpcAttemptsDelay:2 singleActiveBackend:false}
6:06PM DBG Loading external backend: /Users/andrey/sandbox/llm/current/build/local_ai/extra/grpc/huggingface/huggingface.py
6:06PM DBG Loading GRPC Process: /Users/andrey/sandbox/llm/current/build/local_ai/extra/grpc/huggingface/huggingface.py
6:06PM DBG GRPC Service for all-MiniLM-L6-v2 will be running at: '127.0.0.1:57865'
6:06PM DBG GRPC Service state dir: /var/folders/f9/1b1jz83s4ysfn9zfncbsb8y40000gn/T/go-processmanager1166167290
6:06PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:57865: connect: connection refused"
6:06PM DBG GRPC(all-MiniLM-L6-v2-127.0.0.1:57865): stderr Traceback (most recent call last):
6:06PM DBG GRPC(all-MiniLM-L6-v2-127.0.0.1:57865): stderr   File "/Users/andrey/sandbox/llm/current/build/local_ai/extra/grpc/huggingface/huggingface.py", line 2, in <module>
6:06PM DBG GRPC(all-MiniLM-L6-v2-127.0.0.1:57865): stderr     import grpc
6:06PM DBG GRPC(all-MiniLM-L6-v2-127.0.0.1:57865): stderr   File "/Users/andrey/miniconda3/lib/python3.11/site-packages/grpc/__init__.py", line 22, in <module>
6:06PM DBG GRPC(all-MiniLM-L6-v2-127.0.0.1:57865): stderr     from grpc import _compression
6:06PM DBG GRPC(all-MiniLM-L6-v2-127.0.0.1:57865): stderr   File "/Users/andrey/miniconda3/lib/python3.11/site-packages/grpc/_compression.py", line 20, in <module>
6:06PM DBG GRPC(all-MiniLM-L6-v2-127.0.0.1:57865): stderr     from grpc._cython import cygrpc
6:06PM DBG GRPC(all-MiniLM-L6-v2-127.0.0.1:57865): stderr ImportError: dlopen(/Users/andrey/miniconda3/lib/python3.11/site-packages/grpc/_cython/cygrpc.cpython-311-darwin.so, 0x0002): symbol not found in flat namespace '_kCFStreamPropertySocketNativeHandle'
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:57865: connect: connection refused"
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:57865: connect: connection refused"
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:57865: connect: connection refused"
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:57865: connect: connection refused"
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:57865: connect: connection refused"
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:57865: connect: connection refused"
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:57865: connect: connection refused"
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:57865: connect: connection refused"
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:57865: connect: connection refused"
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:57865: connect: connection refused"
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:57865: connect: connection refused"
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:57865: connect: connection refused"
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:57865: connect: connection refused"
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:57865: connect: connection refused"

What can we do with this error?

jurkov commented 11 months ago

I got it somehow working by using the older version of all-MiniLM-L6-v2/ggml-model-q4_0.bin at https://huggingface.co/skeskinen/ggml/tree/f2fc4b47cde74a0865a15caa67ff3dd11b1e4ccc/all-MiniLM-L6-v2 (it's the older commit) But the answer is still incorrect.

yiranleng commented 10 months ago

I got it somehow working by using the older version of all-MiniLM-L6-v2/ggml-model-q4_0.bin at https://huggingface.co/skeskinen/ggml/tree/f2fc4b47cde74a0865a15caa67ff3dd11b1e4ccc/all-MiniLM-L6-v2 (it's the older commit) But the answer is still incorrect.

This version has solved the problem