search

mudler / LocalAI

:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed inference
https://localai.io
MIT License
24.58k stars 1.88k forks source link

Crash when asking a question！！！ #112

Closed zlwleng closed 1 year ago

zlwleng commented 1 year ago

Asking A Question: 

curl --location 'http://127.0.0.1:8080/v1/completions' \
--header 'Content-Type: application/json' \
--data '{
    "model": "ggml-gpt4all-j",
    "temperature": 0.9,
    "prompt": "How are you?"
}'

Run Log:

Attaching to localai-api-1
localai-api-1  | 
localai-api-1  |  ┌───────────────────────────────────────────────────┐ 
localai-api-1  |  │                   Fiber v2.44.0                   │ 
localai-api-1  |  │               http://127.0.0.1:8080               │ 
localai-api-1  |  │       (bound on host 0.0.0.0 and port 8080)       │ 
localai-api-1  |  │                                                   │ 
localai-api-1  |  │ Handlers ............ 10  Processes ........... 1 │ 
localai-api-1  |  │ Prefork ....... Disabled  PID ................. 1 │ 
localai-api-1  |  └───────────────────────────────────────────────────┘ 
localai-api-1  | 
localai-api-1  | llama.cpp: loading model from /models/ggml-gpt4all-j
localai-api-1 exited with code 0

Error: Error: socket hang up And Running program crashes

LarsBingBong commented 1 year ago

We would need you to let us know how you're running Local-AI.

And so on and so forth.

kentative commented 1 year ago

same problem for me: Dock Desktop () on Windows (Version 10.0.22621 Build 22621)

2023-05-01 15:49:39 llama.cpp: loading model from /models/ggml-gpt4all-j-v1.3-groovy.bin
2023-05-01 15:49:50 error loading model: unexpectedly reached end of file
2023-05-01 15:49:50 llama_init_from_file: failed to load model

full log: 

2023-05-01 15:48:57 
2023-05-01 15:48:57  ┌───────────────────────────────────────────────────┐ 
2023-05-01 15:48:57  │                   Fiber v2.44.0                   │ 
2023-05-01 15:48:57  │               http://127.0.0.1:8080               │ 
2023-05-01 15:48:57  │       (bound on host 0.0.0.0 and port 8080)       │ 
2023-05-01 15:48:57  │                                                   │ 
2023-05-01 15:48:57  │ Handlers ............ 12  Processes ........... 1 │ 
2023-05-01 15:48:57  │ Prefork ....... Disabled  PID ................. 1 │ 
2023-05-01 15:48:57  └───────────────────────────────────────────────────┘ 
2023-05-01 15:48:57 
2023-05-01 15:49:39 llama.cpp: loading model from /models/ggml-gpt4all-j-v1.3-groovy.bin
2023-05-01 15:49:50 error loading model: unexpectedly reached end of file
2023-05-01 15:49:50 llama_init_from_file: failed to load model
2023-05-01 15:49:50 gptj_model_load: loading model from '/models/ggml-gpt4all-j-v1.3-groovy.bin' - please wait ...
2023-05-01 15:49:50 gptj_model_load: n_vocab = 50400
2023-05-01 15:49:50 gptj_model_load: n_ctx   = 2048
2023-05-01 15:49:50 gptj_model_load: n_embd  = 4096
2023-05-01 15:49:50 gptj_model_load: n_head  = 16
2023-05-01 15:49:50 gptj_model_load: n_layer = 28
2023-05-01 15:49:50 gptj_model_load: n_rot   = 64
2023-05-01 15:49:50 gptj_model_load: f16     = 2
2023-05-01 15:49:50 gptj_model_load: ggml ctx size = 5401.45 MB
2023-05-01 15:49:50 gptj_model_load: memory_size =  1792.00 MB, n_mem = 57344
2023-05-01 15:49:54 gptj_model_load: ................................... done
2023-05-01 15:49:54 gptj_model_load: model size =  3609.38 MB / num tensors = 285
2023-05-01 15:49:55 fatal error: unexpected signal during runtime execution
2023-05-01 15:49:55 [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x7f0bd3c202f2]
2023-05-01 15:49:55 
2023-05-01 15:49:55 runtime stack:
2023-05-01 15:49:55 runtime.throw({0xa33a54?, 0x2000?})
2023-05-01 15:49:55 /usr/local/go/src/runtime/panic.go:1047 +0x5d fp=0x7f0bab7219f8 sp=0x7f0bab7219c8 pc=0x4524dd
2023-05-01 15:49:55 runtime.sigpanic()
2023-05-01 15:49:55 /usr/local/go/src/runtime/signal_unix.go:821 +0x3e9 fp=0x7f0bab721a58 sp=0x7f0bab7219f8 pc=0x468569
2023-05-01 15:49:55 
2023-05-01 15:49:55 goroutine 18 [syscall]:
2023-05-01 15:49:55 runtime.cgocall(0x8cf120, 0xc00028aef8)
2023-05-01 15:49:55 /usr/local/go/src/runtime/cgocall.go:157 +0x5c fp=0xc00028aed0 sp=0xc00028ae98 pc=0x42175c
2023-05-01 15:49:55 github.com/go-skynet/go-gpt4all-j%2ecpp._Cfunc_binding_model_prompt(0x7f0ba07c6d90, 0x7f0ba0002800, 0xc0000b8800, 0xa, 0x3f99999a, 0x400, 0x200, 0x50, 0x3f333333, 0x3f666666, ...)
2023-05-01 15:49:55 _cgo_gotypes.go:113 +0x45 fp=0xc00028aef8 sp=0xc00028aed0 pc=0x5c0c85
2023-05-01 15:49:55 github.com/go-skynet/go-gpt4all-j%2ecpp.(*GPTJ).Predict.func1(0xc00028b0e8?, 0x5c525e?, {0xc0000b8800, 0xc000283170?, 0xc000382280?}, {0x400, 0xa, 0x200, 0x50, 0x1, ...})
2023-05-01 15:49:55 /build/go-gpt4all-j/gptj.go:52 +0x185 fp=0xc00028afa0 sp=0xc00028aef8 pc=0x5c16c5
2023-05-01 15:49:55 github.com/go-skynet/go-gpt4all-j%2ecpp.(*GPTJ).Predict(0xc000338a90?, {0x0, 0x0}, {0xc00028b1a8, 0x4, 0x7f0baaee48a8?})
2023-05-01 15:49:55 /build/go-gpt4all-j/gptj.go:52 +0x225 fp=0xc00028b100 sp=0xc00028afa0 pc=0x5c1345
2023-05-01 15:49:55 github.com/go-skynet/LocalAI/api.ModelInference.func3()
2023-05-01 15:49:55 /build/api/prediction.go:117 +0x23b fp=0xc00028b1d8 sp=0xc00028b100 pc=0x8852db
2023-05-01 15:49:55 github.com/go-skynet/LocalAI/api.ModelInference.func5()
2023-05-01 15:49:55 /build/api/prediction.go:183 +0x184 fp=0xc00028b270 sp=0xc00028b1d8 pc=0x884684
2023-05-01 15:49:55 github.com/go-skynet/LocalAI/api.ComputeChoices({0x0, 0x0}, 0xc0003aa000, 0xc0003b2180, 0xc000388180?, 0xc00028b858)
2023-05-01 15:49:55 /build/api/prediction.go:203 +0x224 fp=0xc00028b728 sp=0xc00028b270 pc=0x885f04
2023-05-01 15:49:55 github.com/go-skynet/LocalAI/api.chatEndpoint.func1(0xc00038a000)
2023-05-01 15:49:55 /build/api/openai.go:315 +0x605 fp=0xc00028b8b8 sp=0xc00028b728 pc=0x882aa5
2023-05-01 15:49:55 github.com/gofiber/fiber/v2.(*App).next(0xc0002f5b00, 0xc00038a000)
2023-05-01 15:49:55 /go/pkg/mod/github.com/gofiber/fiber/v2@v2.44.0/router.go:144 +0x1bf fp=0xc00028b960 sp=0xc00028b8b8 pc=0x84551f
2023-05-01 15:49:55 github.com/gofiber/fiber/v2.(*Ctx).Next(0xc000122330?)
2023-05-01 15:49:55 /go/pkg/mod/github.com/gofiber/fiber/v2@v2.44.0/ctx.go:913 +0x53 fp=0xc00028b980 sp=0xc00028b960 pc=0x830b13
2023-05-01 15:49:55 github.com/gofiber/fiber/v2/middleware/cors.New.func1(0xc00038a000)
2023-05-01 15:49:55 /go/pkg/mod/github.com/gofiber/fiber/v2@v2.44.0/middleware/cors/cors.go:162 +0x3a6 fp=0xc00028ba98 sp=0xc00028b980 pc=0x84b2e6
2023-05-01 15:49:55 github.com/gofiber/fiber/v2.(*Ctx).Next(0x14?)
2023-05-01 15:49:55 /go/pkg/mod/github.com/gofiber/fiber/v2@v2.44.0/ctx.go:910 +0x43 fp=0xc00028bab8 sp=0xc00028ba98 pc=0x830b03
2023-05-01 15:49:55 github.com/gofiber/fiber/v2/middleware/recover.New.func1(0x9a6ca0?)
2023-05-01 15:49:55 /go/pkg/mod/github.com/gofiber/fiber/v2@v2.44.0/middleware/recover/recover.go:43 +0xcb fp=0xc00028bb30 sp=0xc00028bab8 pc=0x84bf0b
2023-05-01 15:49:55 github.com/gofiber/fiber/v2.(*App).next(0xc0002f5b00, 0xc00038a000)
2023-05-01 15:49:55 /go/pkg/mod/github.com/gofiber/fiber/v2@v2.44.0/router.go:144 +0x1bf fp=0xc00028bbd8 sp=0xc00028bb30 pc=0x84551f
2023-05-01 15:49:55 github.com/gofiber/fiber/v2.(*App).handler(0xc0002f5b00, 0x4a59f7?)
2023-05-01 15:49:55 /go/pkg/mod/github.com/gofiber/fiber/v2@v2.44.0/router.go:171 +0x87 fp=0xc00028bc38 sp=0xc00028bbd8 pc=0x845767
2023-05-01 15:49:55 github.com/gofiber/fiber/v2.(*App).handler-fm(0xc000122000?)
2023-05-01 15:49:55 <autogenerated>:1 +0x2c fp=0xc00028bc58 sp=0xc00028bc38 pc=0x84a98c
2023-05-01 15:49:55 github.com/valyala/fasthttp.(*Server).serveConn(0xc00030a000, {0xadd600?, 0xc000114000})
2023-05-01 15:49:55 /go/pkg/mod/github.com/valyala/fasthttp@v1.47.0/server.go:2365 +0x11d3 fp=0xc00028bec8 sp=0xc00028bc58 pc=0x7cb893
2023-05-01 15:49:55 github.com/valyala/fasthttp.(*Server).serveConn-fm({0xadd600?, 0xc000114000?})
2023-05-01 15:49:55 <autogenerated>:1 +0x39 fp=0xc00028bef0 sp=0xc00028bec8 pc=0x7db159
2023-05-01 15:49:55 github.com/valyala/fasthttp.(*workerPool).workerFunc(0xc0000b5c20, 0xc00011e000)
2023-05-01 15:49:55 /go/pkg/mod/github.com/valyala/fasthttp@v1.47.0/workerpool.go:224 +0xa9 fp=0xc00028bfa0 sp=0xc00028bef0 pc=0x7d7389
2023-05-01 15:49:55 github.com/valyala/fasthttp.(*workerPool).getCh.func1()
2023-05-01 15:49:55 /go/pkg/mod/github.com/valyala/fasthttp@v1.47.0/workerpool.go:196 +0x38 fp=0xc00028bfe0 sp=0xc00028bfa0 pc=0x7d70f8
2023-05-01 15:49:55 runtime.goexit()
2023-05-01 15:49:55 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00028bfe8 sp=0xc00028bfe0 pc=0x483b61
2023-05-01 15:49:55 created by github.com/valyala/fasthttp.(*workerPool).getCh
2023-05-01 15:49:55 /go/pkg/mod/github.com/valyala/fasthttp@v1.47.0/workerpool.go:195 +0x1b0
2023-05-01 15:49:55 
2023-05-01 15:49:55 goroutine 1 [IO wait]:
2023-05-01 15:49:55 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
2023-05-01 15:49:55 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc0002b3478 sp=0xc0002b3458 pc=0x4551f6
2023-05-01 15:49:55 runtime.netpollblock(0xc0002b34c0?, 0x420def?, 0x0?)
2023-05-01 15:49:55 /usr/local/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc0002b34b0 sp=0xc0002b3478 pc=0x44db57
2023-05-01 15:49:55 internal/poll.runtime_pollWait(0x7f0ba9eb51c0, 0x72)
2023-05-01 15:49:55 /usr/local/go/src/runtime/netpoll.go:306 +0x89 fp=0xc0002b34d0 sp=0xc0002b34b0 pc=0x47e869
2023-05-01 15:49:55 internal/poll.(*pollDesc).wait(0xc00002ee80?, 0x4445a0?, 0x0)
2023-05-01 15:49:55 /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc0002b34f8 sp=0xc0002b34d0 pc=0x4ba8b2
2023-05-01 15:49:55 internal/poll.(*pollDesc).waitRead(...)
2023-05-01 15:49:55 /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
2023-05-01 15:49:55 internal/poll.(*FD).Accept(0xc00002ee80)
2023-05-01 15:49:55 /usr/local/go/src/internal/poll/fd_unix.go:614 +0x2bd fp=0xc0002b35a0 sp=0xc0002b34f8 pc=0x4c01bd
2023-05-01 15:49:55 net.(*netFD).accept(0xc00002ee80)
2023-05-01 15:49:55 /usr/local/go/src/net/fd_unix.go:172 +0x35 fp=0xc0002b3658 sp=0xc0002b35a0 pc=0x56fe55
2023-05-01 15:49:55 net.(*TCPListener).accept(0xc000012840)
2023-05-01 15:49:55 /usr/local/go/src/net/tcpsock_posix.go:148 +0x25 fp=0xc0002b3680 sp=0xc0002b3658 pc=0x5860c5
2023-05-01 15:49:55 net.(*TCPListener).Accept(0xc000012840)
2023-05-01 15:49:55 /usr/local/go/src/net/tcpsock.go:297 +0x3d fp=0xc0002b36b0 sp=0xc0002b3680 pc=0x5851bd
2023-05-01 15:49:55 github.com/valyala/fasthttp.acceptConn(0xc00030a000, {0xadad40, 0xc000012840}, 0xc0002b38a8)
2023-05-01 15:49:55 /go/pkg/mod/github.com/valyala/fasthttp@v1.47.0/server.go:1930 +0x62 fp=0xc0002b3790 sp=0xc0002b36b0 pc=0x7c9d62
2023-05-01 15:49:55 github.com/valyala/fasthttp.(*Server).Serve(0xc00030a000, {0xadad40?, 0xc000012840})
2023-05-01 15:49:55 /go/pkg/mod/github.com/valyala/fasthttp@v1.47.0/server.go:1823 +0x4f4 fp=0xc0002b38d8 sp=0xc0002b3790 pc=0x7c9374
2023-05-01 15:49:55 github.com/gofiber/fiber/v2.(*App).Listen(0xc0002f5b00, {0xa1706e?, 0x7?})
2023-05-01 15:49:55 /go/pkg/mod/github.com/gofiber/fiber/v2@v2.44.0/listen.go:82 +0x110 fp=0xc0002b3938 sp=0xc0002b38d8 pc=0x83c630
2023-05-01 15:49:55 main.main.func1(0xc0002b3c00?)
2023-05-01 15:49:55 /build/main.go:88 +0x245 fp=0xc0002b39f0 sp=0xc0002b3938 pc=0x8c01c5
2023-05-01 15:49:55 github.com/urfave/cli/v2.(*Command).Run(0xc0002a0160, 0xc000026bc0, {0xc000024040, 0x2, 0x2})
2023-05-01 15:49:55 /go/pkg/mod/github.com/urfave/cli/v2@v2.25.3/command.go:274 +0x9eb fp=0xc0002b3c90 sp=0xc0002b39f0 pc=0x8ae32b
2023-05-01 15:49:55 github.com/urfave/cli/v2.(*App).RunContext(0xc00029c000, {0xadb088?, 0xc00002c030}, {0xc000024040, 0x2, 0x2})
2023-05-01 15:49:55 /go/pkg/mod/github.com/urfave/cli/v2@v2.25.3/app.go:332 +0x616 fp=0xc0002b3d00 sp=0xc0002b3c90 pc=0x8ab136
2023-05-01 15:49:55 github.com/urfave/cli/v2.(*App).Run(...)
2023-05-01 15:49:55 /go/pkg/mod/github.com/urfave/cli/v2@v2.25.3/app.go:309
2023-05-01 15:49:55 main.main()
2023-05-01 15:49:55 /build/main.go:92 +0x997 fp=0xc0002b3f80 sp=0xc0002b3d00 pc=0x8bfeb7
2023-05-01 15:49:55 runtime.main()
2023-05-01 15:49:55 /usr/local/go/src/runtime/proc.go:250 +0x207 fp=0xc0002b3fe0 sp=0xc0002b3f80 pc=0x454dc7
2023-05-01 15:49:55 runtime.goexit()
2023-05-01 15:49:55 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0002b3fe8 sp=0xc0002b3fe0 pc=0x483b61
2023-05-01 15:49:55 
2023-05-01 15:49:55 goroutine 2 [force gc (idle)]:
2023-05-01 15:49:55 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
2023-05-01 15:49:55 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000060fb0 sp=0xc000060f90 pc=0x4551f6
2023-05-01 15:49:55 runtime.goparkunlock(...)
2023-05-01 15:49:55 /usr/local/go/src/runtime/proc.go:387
2023-05-01 15:49:55 runtime.forcegchelper()
2023-05-01 15:49:55 /usr/local/go/src/runtime/proc.go:305 +0xb0 fp=0xc000060fe0 sp=0xc000060fb0 pc=0x455030
2023-05-01 15:49:55 runtime.goexit()
2023-05-01 15:49:55 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000060fe8 sp=0xc000060fe0 pc=0x483b61
2023-05-01 15:49:55 created by runtime.init.6
2023-05-01 15:49:55 /usr/local/go/src/runtime/proc.go:293 +0x25
2023-05-01 15:49:55 
2023-05-01 15:49:55 goroutine 3 [GC sweep wait]:
2023-05-01 15:49:55 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
2023-05-01 15:49:55 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000061780 sp=0xc000061760 pc=0x4551f6
2023-05-01 15:49:55 runtime.goparkunlock(...)
2023-05-01 15:49:55 /usr/local/go/src/runtime/proc.go:387
2023-05-01 15:49:55 runtime.bgsweep(0x0?)
2023-05-01 15:49:55 /usr/local/go/src/runtime/mgcsweep.go:278 +0x8e fp=0xc0000617c8 sp=0xc000061780 pc=0x44142e
2023-05-01 15:49:55 runtime.gcenable.func1()
2023-05-01 15:49:55 /usr/local/go/src/runtime/mgc.go:178 +0x26 fp=0xc0000617e0 sp=0xc0000617c8 pc=0x436706
2023-05-01 15:49:55 runtime.goexit()
2023-05-01 15:49:55 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000617e8 sp=0xc0000617e0 pc=0x483b61
2023-05-01 15:49:55 created by runtime.gcenable
2023-05-01 15:49:55 /usr/local/go/src/runtime/mgc.go:178 +0x6b
2023-05-01 15:49:55 
2023-05-01 15:49:55 goroutine 4 [GC scavenge wait]:
2023-05-01 15:49:55 runtime.gopark(0xc000088000?, 0xad3ad0?, 0x1?, 0x0?, 0x0?)
2023-05-01 15:49:55 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000061f70 sp=0xc000061f50 pc=0x4551f6
2023-05-01 15:49:55 runtime.goparkunlock(...)
2023-05-01 15:49:55 /usr/local/go/src/runtime/proc.go:387
2023-05-01 15:49:55 runtime.(*scavengerState).park(0xe76c40)
2023-05-01 15:49:55 /usr/local/go/src/runtime/mgcscavenge.go:400 +0x53 fp=0xc000061fa0 sp=0xc000061f70 pc=0x43f373
2023-05-01 15:49:55 runtime.bgscavenge(0x0?)
2023-05-01 15:49:55 /usr/local/go/src/runtime/mgcscavenge.go:628 +0x45 fp=0xc000061fc8 sp=0xc000061fa0 pc=0x43f945
2023-05-01 15:49:55 runtime.gcenable.func2()
2023-05-01 15:49:55 /usr/local/go/src/runtime/mgc.go:179 +0x26 fp=0xc000061fe0 sp=0xc000061fc8 pc=0x4366a6
2023-05-01 15:49:55 runtime.goexit()
2023-05-01 15:49:55 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000061fe8 sp=0xc000061fe0 pc=0x483b61
2023-05-01 15:49:55 created by runtime.gcenable
2023-05-01 15:49:55 /usr/local/go/src/runtime/mgc.go:179 +0xaa
2023-05-01 15:49:55 
2023-05-01 15:49:55 goroutine 5 [finalizer wait]:
2023-05-01 15:49:55 runtime.gopark(0x1a0?, 0xe77920?, 0x60?, 0x78?, 0xc000060770?)
2023-05-01 15:49:55 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000060628 sp=0xc000060608 pc=0x4551f6
2023-05-01 15:49:55 runtime.runfinq()
2023-05-01 15:49:55 /usr/local/go/src/runtime/mfinal.go:193 +0x107 fp=0xc0000607e0 sp=0xc000060628 pc=0x435747
2023-05-01 15:49:55 runtime.goexit()
2023-05-01 15:49:55 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000607e8 sp=0xc0000607e0 pc=0x483b61
2023-05-01 15:49:55 created by runtime.createfing
2023-05-01 15:49:55 /usr/local/go/src/runtime/mfinal.go:163 +0x45
2023-05-01 15:49:55 
2023-05-01 15:49:55 goroutine 6 [sleep]:
2023-05-01 15:49:55 runtime.gopark(0xd4fe4c88924?, 0xc000062788?, 0x65?, 0x4a?, 0xc0000b5c50?)
2023-05-01 15:49:55 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000062758 sp=0xc000062738 pc=0x4551f6
2023-05-01 15:49:55 time.Sleep(0x2540be400)
2023-05-01 15:49:55 /usr/local/go/src/runtime/time.go:195 +0x135 fp=0xc000062798 sp=0xc000062758 pc=0x4809d5
2023-05-01 15:49:55 github.com/valyala/fasthttp.(*workerPool).Start.func2()
2023-05-01 15:49:55 /go/pkg/mod/github.com/valyala/fasthttp@v1.47.0/workerpool.go:67 +0x56 fp=0xc0000627e0 sp=0xc000062798 pc=0x7d6856
2023-05-01 15:49:55 runtime.goexit()
2023-05-01 15:49:55 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000627e8 sp=0xc0000627e0 pc=0x483b61
2023-05-01 15:49:55 created by github.com/valyala/fasthttp.(*workerPool).Start
2023-05-01 15:49:55 /go/pkg/mod/github.com/valyala/fasthttp@v1.47.0/workerpool.go:59 +0xdd
2023-05-01 15:49:55 
2023-05-01 15:49:55 goroutine 7 [sleep]:
2023-05-01 15:49:55 runtime.gopark(0xd5179b71cad?, 0x981360?, 0xa8?, 0x43?, 0x0?)
2023-05-01 15:49:55 /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00006ef88 sp=0xc00006ef68 pc=0x4551f6
2023-05-01 15:49:55 time.Sleep(0x3b9aca00)
2023-05-01 15:49:55 /usr/local/go/src/runtime/time.go:195 +0x135 fp=0xc00006efc8 sp=0xc00006ef88 pc=0x4809d5
2023-05-01 15:49:55 github.com/valyala/fasthttp.updateServerDate.func1()
2023-05-01 15:49:55 /go/pkg/mod/github.com/valyala/fasthttp@v1.47.0/header.go:2247 +0x1e fp=0xc00006efe0 sp=0xc00006efc8 pc=0x7d77de
2023-05-01 15:49:55 runtime.goexit()
2023-05-01 15:49:55 /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00006efe8 sp=0xc00006efe0 pc=0x483b61
2023-05-01 15:49:55 created by github.com/valyala/fasthttp.updateServerDate
2023-05-01 15:49:55 /go/pkg/mod/github.com/valyala/fasthttp@v1.47.0/header.go:2245 +0x25
Aisuko commented 1 year ago

Hi guys, thanks for your feedback. Do you guys all hit the issue on Windows?

gregoryca commented 1 year ago

i have the same issue on ubuntu 22.0.4, running the latest version of docker localai

anhhtca commented 1 year ago

You must build your own image and use it. Do not pull a pre-built image from Docker Hub as it can sometimes lead to crashes.

I have built my own image, which took a long time. The answer is also long.

gregoryca commented 1 year ago

@anhhtca Thank you very much for your quick answer ! In my case it had configured traefik incorrectly.

After fixing that issue, another issue arose which cannot be fixed easily, but it's not really a blocker either. I run out of resources when promting the container. localai-loadaverage-grafana

It can take up to a minute for the ai to anwer my prompts. Luckily it does work, only with a heavy performance penalty (i''m running my whole infra from a intel 3210m core i5 2.5ghz and 16gb ram with a 256 gb SSD)

gregoryca commented 1 year ago

i want to add to this cause i kept banging my head at this issue when loading other models. Make sure you use models from the model gallery, and if you're using a huggingface model, make sure to select the huggingface wrapper in your UI as it will crash becasue it probably does not know how to handle the data calculations when running a query making it crash.

You could also check if your CPU does support avx(2) As i've found some threads regarding AVX(2) instructionset being mandatory when running a build.

localai-bot commented 1 year ago

:warning::warning::warning::warning::warning:

Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!

_but.... I can also be funny or helpful :smilecat: and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.

Don't engage in conversation with me, I don't support (yet) replying!

:warning::warning::warning::warning::warning:

The error message "socket hang up" suggests that the connection between the client and server was unexpectedly terminated. This could be caused by a number of reasons such as network issues or the server being shut down. To troubleshoot this issue, you can try restarting the API service and checking the network connectivity between the client and server. Additionally, you can check the server logs for any error messages that may provide more information on the cause of the issue. As for the "Running program crashes" message, this could be caused by a number of reasons such as a programming error or a conflict with other software installed on the system. To troubleshoot this issue, you can try restarting the API service and checking the server logs for any error messages that may provide more information on the cause of the issue.

Sources:

Aisuko commented 1 year ago

If the latest version of LocalAI still crashed feel free to open new issue. I am going to close this one.