search

mudler / LocalAI

:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed inference
https://localai.io
MIT License
24.53k stars 1.88k forks source link

Docker container loading model fails - Unexepected End of File #87

Closed ChaseDDevelopment closed 1 year ago

ChaseDDevelopment commented 1 year ago

Hey there! I am trying to use this on a docker "server" within my local network, and on that specific machine I get hit with the following error:

local-ai  | 2023-04-26T00:13:47.432284170Z error loading model: unexpectedly reached end of file
local-ai  | 2023-04-26T00:13:47.432317663Z llama_init_from_file: failed to load model
local-ai  | 2023-04-26T00:13:47.524068627Z gptj_model_load: loading model from '/models/ggml-gpt4all-j' - please wait ...
local-ai  | 2023-04-26T00:13:47.524100675Z gptj_model_load: n_vocab = 50400
local-ai  | 2023-04-26T00:13:47.524104579Z gptj_model_load: n_ctx   = 2048
local-ai  | 2023-04-26T00:13:47.524107130Z gptj_model_load: n_embd  = 4096
local-ai  | 2023-04-26T00:13:47.524110309Z gptj_model_load: n_head  = 16
local-ai  | 2023-04-26T00:13:47.524135455Z gptj_model_load: n_layer = 28
local-ai  | 2023-04-26T00:13:47.524137975Z gptj_model_load: n_rot   = 64
local-ai  | 2023-04-26T00:13:47.524140167Z gptj_model_load: f16     = 2
local-ai  | 2023-04-26T00:13:47.524142430Z gptj_model_load: ggml ctx size = 5401.45 MB
local-ai  | 2023-04-26T00:13:47.524144654Z gptj_model_load: memory_size =  1792.00 MB, n_mem = 57344
local-ai  | 2023-04-26T00:13:48.630473827Z SIGILL: illegal instruction
local-ai  | 2023-04-26T00:13:48.630521560Z PC=0x8da2a9 m=0 sigcode=2
local-ai  | 2023-04-26T00:13:48.630525275Z signal arrived during cgo execution
local-ai  | 2023-04-26T00:13:48.630527908Z instruction bytes: 0x62 0xf2 0xfd 0x8 0x7c 0xc0 0x49 0x89 0x45 0x0 0x48 0x89 0x83 0x68 0x1 0x0
local-ai  | 2023-04-26T00:13:48.630530427Z 
local-ai  | 2023-04-26T00:13:48.630532762Z goroutine 7 [syscall]:
local-ai  | 2023-04-26T00:13:48.630535146Z runtime.cgocall(0x894bf0, 0xc0002871b0)
local-ai  | 2023-04-26T00:13:48.630537480Z  /usr/local/go/src/runtime/cgocall.go:157 +0x5c fp=0xc000287188 sp=0xc000287150 pc=0x4204bc
local-ai  | 2023-04-26T00:13:48.630540122Z github.com/go-skynet/go-gpt4all-j%2ecpp._Cfunc_gptj_predict(0x2cba890, 0x2c9f490, 0xc00011a200)
local-ai  | 2023-04-26T00:13:48.630542517Z  _cgo_gotypes.go:158 +0x4c fp=0xc0002871b0 sp=0xc000287188 pc=0x5bfb8c
local-ai  | 2023-04-26T00:13:48.630544960Z github.com/go-skynet/go-gpt4all-j%2ecpp.(*GPTJ).Predict.func1(0x2ca0ea0?, 0x10ffffffff?, {0xc00011a200, 0x3f6666663f333333?, 0xc000000009?})
local-ai  | 2023-04-26T00:13:48.630547417Z  /build/go-gpt4all-j/gptj.go:43 +0x7e fp=0xc0002871f0 sp=0xc0002871b0 pc=0x5c037e
local-ai  | 2023-04-26T00:13:48.630549880Z github.com/go-skynet/go-gpt4all-j%2ecpp.(*GPTJ).Predict(0x100c0002872e0?, {0xc0001220c0, 0xb6}, {0xc0002873d0, 0x5, 0xc0001220c0?})
local-ai  | 2023-04-26T00:13:48.630555133Z  /build/go-gpt4all-j/gptj.go:43 +0x225 fp=0xc0002872f8 sp=0xc0002871f0 pc=0x5c0045
local-ai  | 2023-04-26T00:13:48.630557757Z github.com/go-skynet/LocalAI/api.openAIEndpoint.func1.3()
local-ai  | 2023-04-26T00:13:48.630560143Z  /build/api/api.go:285 +0x33e fp=0xc000287408 sp=0xc0002872f8 pc=0x84b1de
local-ai  | 2023-04-26T00:13:48.630562497Z github.com/go-skynet/LocalAI/api.openAIEndpoint.func1(0x98a500?)
local-ai  | 2023-04-26T00:13:48.630564835Z  /build/api/api.go:341 +0xebc fp=0xc0002878c8 sp=0xc000287408 pc=0x849f9c
local-ai  | 2023-04-26T00:13:48.630567217Z github.com/gofiber/fiber/v2.(*App).next(0xc0002ef680, 0xc0000d4840)
local-ai  | 2023-04-26T00:13:48.630571963Z  /go/pkg/mod/github.com/gofiber/fiber/v2@v2.42.0/router.go:134 +0x1b6 fp=0xc000287970 sp=0xc0002878c8 pc=0x8424b6
local-ai  | 2023-04-26T00:13:48.630574843Z github.com/gofiber/fiber/v2.(*Ctx).Next(0xc0002fa930?)
local-ai  | 2023-04-26T00:13:48.630578591Z  /go/pkg/mod/github.com/gofiber/fiber/v2@v2.42.0/ctx.go:964 +0x53 fp=0xc000287990 sp=0xc000287970 pc=0x82e733
local-ai  | 2023-04-26T00:13:48.630581401Z github.com/gofiber/fiber/v2/middleware/cors.New.func1(0xc0000d4840)
local-ai  | 2023-04-26T00:13:48.630584290Z  /go/pkg/mod/github.com/gofiber/fiber/v2@v2.42.0/middleware/cors/cors.go:140 +0x385 fp=0xc000287a98 sp=0xc000287990 pc=0x848185
local-ai  | 2023-04-26T00:13:48.630586814Z github.com/gofiber/fiber/v2.(*Ctx).Next(0x14?)
local-ai  | 2023-04-26T00:13:48.630597212Z  /go/pkg/mod/github.com/gofiber/fiber/v2@v2.42.0/ctx.go:961 +0x43 fp=0xc000287ab8 sp=0xc000287a98 pc=0x82e723
local-ai  | 2023-04-26T00:13:48.630599982Z github.com/gofiber/fiber/v2/middleware/recover.New.func1(0x98a500?)
local-ai  | 2023-04-26T00:13:48.630603910Z  /go/pkg/mod/github.com/gofiber/fiber/v2@v2.42.0/middleware/recover/recover.go:43 +0xcb fp=0xc000287b30 sp=0xc000287ab8 pc=0x848dab
local-ai  | 2023-04-26T00:13:48.630606610Z github.com/gofiber/fiber/v2.(*App).next(0xc0002ef680, 0xc0000d4840)
local-ai  | 2023-04-26T00:13:48.630620795Z  /go/pkg/mod/github.com/gofiber/fiber/v2@v2.42.0/router.go:134 +0x1b6 fp=0xc000287bd8 sp=0xc000287b30 pc=0x8424b6
local-ai  | 2023-04-26T00:13:48.630638618Z github.com/gofiber/fiber/v2.(*App).handler(0xc0002ef680, 0x4a4697?)
local-ai  | 2023-04-26T00:13:48.630646665Z  /go/pkg/mod/github.com/gofiber/fiber/v2@v2.42.0/router.go:160 +0x87 fp=0xc000287c38 sp=0xc000287bd8 pc=0x8426e7
local-ai  | 2023-04-26T00:13:48.630653028Z github.com/gofiber/fiber/v2.(*App).handler-fm(0xc0002fa600?)
local-ai  | 2023-04-26T00:13:48.630658799Z  <autogenerated>:1 +0x2c fp=0xc000287c58 sp=0xc000287c38 pc=0x84786c
local-ai  | 2023-04-26T00:13:48.630670537Z github.com/valyala/fasthttp.(*Server).serveConn(0xc000306000, {0xab69c0?, 0xc0000143c0})
local-ai  | 2023-04-26T00:13:48.630677246Z  /go/pkg/mod/github.com/valyala/fasthttp@v1.44.0/server.go:2372 +0x11d3 fp=0xc000287ec8 sp=0xc000287c58 pc=0x7c9393
local-ai  | 2023-04-26T00:13:48.630691410Z github.com/valyala/fasthttp.(*Server).serveConn-fm({0xab69c0?, 0xc0000143c0?})
local-ai  | 2023-04-26T00:13:48.630697411Z  <autogenerated>:1 +0x39 fp=0xc000287ef0 sp=0xc000287ec8 pc=0x7d8a59
local-ai  | 2023-04-26T00:13:48.630702961Z github.com/valyala/fasthttp.(*workerPool).workerFunc(0xc0000b7900, 0xc00030a180)
local-ai  | 2023-04-26T00:13:48.630715544Z  /go/pkg/mod/github.com/valyala/fasthttp@v1.44.0/workerpool.go:224 +0xa9 fp=0xc000287fa0 sp=0xc000287ef0 pc=0x7d4d29
local-ai  | 2023-04-26T00:13:48.630722293Z github.com/valyala/fasthttp.(*workerPool).getCh.func1()
local-ai  | 2023-04-26T00:13:48.630728071Z  /go/pkg/mod/github.com/valyala/fasthttp@v1.44.0/workerpool.go:196 +0x38 fp=0xc000287fe0 sp=0xc000287fa0 pc=0x7d4a98
local-ai  | 2023-04-26T00:13:48.630733370Z runtime.goexit()
local-ai  | 2023-04-26T00:13:48.630745332Z  /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000287fe8 sp=0xc000287fe0 pc=0x482821
local-ai  | 2023-04-26T00:13:48.630752732Z created by github.com/valyala/fasthttp.(*workerPool).getCh
local-ai  | 2023-04-26T00:13:48.630758305Z  /go/pkg/mod/github.com/valyala/fasthttp@v1.44.0/workerpool.go:195 +0x1b0
local-ai  | 2023-04-26T00:13:48.630763584Z 
local-ai  | 2023-04-26T00:13:48.630768662Z goroutine 1 [IO wait]:
local-ai  | 2023-04-26T00:13:48.630781621Z runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
local-ai  | 2023-04-26T00:13:48.630788024Z  /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc0002b34a0 sp=0xc0002b3480 pc=0x453eb6
local-ai  | 2023-04-26T00:13:48.630793761Z runtime.netpollblock(0x7f5a540e36a8?, 0x41fb4f?, 0x0?)
local-ai  | 2023-04-26T00:13:48.630799030Z  /usr/local/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc0002b34d8 sp=0xc0002b34a0 pc=0x44c8b7
local-ai  | 2023-04-26T00:13:48.630873165Z internal/poll.runtime_pollWait(0x7f5a2a513980, 0x72)
local-ai  | 2023-04-26T00:13:48.630891371Z  /usr/local/go/src/runtime/netpoll.go:306 +0x89 fp=0xc0002b34f8 sp=0xc0002b34d8 pc=0x47d529
local-ai  | 2023-04-26T00:13:48.630927780Z internal/poll.(*pollDesc).wait(0xc00002ed80?, 0x4?, 0x0)
local-ai  | 2023-04-26T00:13:48.630948702Z  /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc0002b3520 sp=0xc0002b34f8 pc=0x4b9552
local-ai  | 2023-04-26T00:13:48.630956640Z internal/poll.(*pollDesc).waitRead(...)
local-ai  | 2023-04-26T00:13:48.630962491Z  /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
local-ai  | 2023-04-26T00:13:48.630967926Z internal/poll.(*FD).Accept(0xc00002ed80)
local-ai  | 2023-04-26T00:13:48.630973063Z  /usr/local/go/src/internal/poll/fd_unix.go:614 +0x2bd fp=0xc0002b35c8 sp=0xc0002b3520 pc=0x4bee5d
local-ai  | 2023-04-26T00:13:48.630978247Z net.(*netFD).accept(0xc00002ed80)
local-ai  | 2023-04-26T00:13:48.630983588Z  /usr/local/go/src/net/fd_unix.go:172 +0x35 fp=0xc0002b3680 sp=0xc0002b35c8 pc=0x585e95
local-ai  | 2023-04-26T00:13:48.630991617Z net.(*TCPListener).accept(0xc000012708)
local-ai  | 2023-04-26T00:13:48.630996294Z  /usr/local/go/src/net/tcpsock_posix.go:148 +0x25 fp=0xc0002b36a8 sp=0xc0002b3680 pc=0x59c105
local-ai  | 2023-04-26T00:13:48.631001114Z net.(*TCPListener).Accept(0xc000012708)
local-ai  | 2023-04-26T00:13:48.631005265Z  /usr/local/go/src/net/tcpsock.go:297 +0x3d fp=0xc0002b36d8 sp=0xc0002b36a8 pc=0x59b1fd
local-ai  | 2023-04-26T00:13:48.631010413Z github.com/valyala/fasthttp.acceptConn(0xc000306000, {0xab40c0, 0xc000012708}, 0xc0002b38d0)
local-ai  | 2023-04-26T00:13:48.631016848Z  /go/pkg/mod/github.com/valyala/fasthttp@v1.44.0/server.go:1931 +0x62 fp=0xc0002b37b8 sp=0xc0002b36d8 pc=0x7c7822
local-ai  | 2023-04-26T00:13:48.631022467Z github.com/valyala/fasthttp.(*Server).Serve(0xc000306000, {0xab40c0?, 0xc000012708})
local-ai  | 2023-04-26T00:13:48.631027695Z  /go/pkg/mod/github.com/valyala/fasthttp@v1.44.0/server.go:1824 +0x4f4 fp=0xc0002b3900 sp=0xc0002b37b8 pc=0x7c6e34
local-ai  | 2023-04-26T00:13:48.631037966Z github.com/gofiber/fiber/v2.(*App).Listen(0xc0002ef680, {0x9f6919?, 0x7?})
local-ai  | 2023-04-26T00:13:48.631048021Z  /go/pkg/mod/github.com/gofiber/fiber/v2@v2.42.0/listen.go:82 +0x110 fp=0xc0002b3960 sp=0xc0002b3900 pc=0x839cb0
local-ai  | 2023-04-26T00:13:48.631051706Z main.main.func1(0xc0002fe160?)
local-ai  | 2023-04-26T00:13:48.631054660Z  /build/main.go:83 +0x1f5 fp=0xc0002b39f8 sp=0xc0002b3960 pc=0x885c75
local-ai  | 2023-04-26T00:13:48.631061855Z github.com/urfave/cli/v2.(*Command).Run(0xc0002fe160, 0xc000026a80, {0xc000024220, 0x1, 0x1})
local-ai  | 2023-04-26T00:13:48.631074097Z  /go/pkg/mod/github.com/urfave/cli/v2@v2.25.1/command.go:274 +0x9eb fp=0xc0002b3c98 sp=0xc0002b39f8 pc=0x873eeb
local-ai  | 2023-04-26T00:13:48.631102273Z github.com/urfave/cli/v2.(*App).RunContext(0xc0002fc000, {0xab43d8?, 0xc00002c050}, {0xc000024220, 0x1, 0x1})
local-ai  | 2023-04-26T00:13:48.631113809Z  /go/pkg/mod/github.com/urfave/cli/v2@v2.25.1/app.go:332 +0x616 fp=0xc0002b3d08 sp=0xc0002b3c98 pc=0x870cf6
local-ai  | 2023-04-26T00:13:48.631138996Z github.com/urfave/cli/v2.(*App).Run(...)
local-ai  | 2023-04-26T00:13:48.631152233Z  /go/pkg/mod/github.com/urfave/cli/v2@v2.25.1/app.go:309
local-ai  | 2023-04-26T00:13:48.631171211Z main.main()
local-ai  | 2023-04-26T00:13:48.631177099Z  /build/main.go:87 +0x8c9 fp=0xc0002b3f80 sp=0xc0002b3d08 pc=0x8859a9
local-ai  | 2023-04-26T00:13:48.631181761Z runtime.main()
local-ai  | 2023-04-26T00:13:48.631184564Z  /usr/local/go/src/runtime/proc.go:250 +0x207 fp=0xc0002b3fe0 sp=0xc0002b3f80 pc=0x453a87
local-ai  | 2023-04-26T00:13:48.631187366Z runtime.goexit()
local-ai  | 2023-04-26T00:13:48.631189955Z  /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0002b3fe8 sp=0xc0002b3fe0 pc=0x482821
local-ai  | 2023-04-26T00:13:48.631192711Z 
local-ai  | 2023-04-26T00:13:48.631195264Z goroutine 2 [force gc (idle)]:
local-ai  | 2023-04-26T00:13:48.631197913Z runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
local-ai  | 2023-04-26T00:13:48.631200946Z  /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000060fb0 sp=0xc000060f90 pc=0x453eb6
local-ai  | 2023-04-26T00:13:48.631209657Z runtime.goparkunlock(...)
local-ai  | 2023-04-26T00:13:48.631212893Z  /usr/local/go/src/runtime/proc.go:387
local-ai  | 2023-04-26T00:13:48.631215688Z runtime.forcegchelper()
local-ai  | 2023-04-26T00:13:48.631219185Z  /usr/local/go/src/runtime/proc.go:305 +0xb0 fp=0xc000060fe0 sp=0xc000060fb0 pc=0x453cf0
local-ai  | 2023-04-26T00:13:48.631222067Z runtime.goexit()
local-ai  | 2023-04-26T00:13:48.631228769Z  /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000060fe8 sp=0xc000060fe0 pc=0x482821
local-ai  | 2023-04-26T00:13:48.631235542Z created by runtime.init.6
local-ai  | 2023-04-26T00:13:48.631240250Z  /usr/local/go/src/runtime/proc.go:293 +0x25
local-ai  | 2023-04-26T00:13:48.631245478Z 
local-ai  | 2023-04-26T00:13:48.631249997Z goroutine 3 [GC sweep wait]:
local-ai  | 2023-04-26T00:13:48.631260540Z runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
local-ai  | 2023-04-26T00:13:48.631270600Z  /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000061780 sp=0xc000061760 pc=0x453eb6
local-ai  | 2023-04-26T00:13:48.631288597Z runtime.goparkunlock(...)
local-ai  | 2023-04-26T00:13:48.631291756Z  /usr/local/go/src/runtime/proc.go:387
local-ai  | 2023-04-26T00:13:48.631297449Z runtime.bgsweep(0x0?)
local-ai  | 2023-04-26T00:13:48.631304029Z  /usr/local/go/src/runtime/mgcsweep.go:278 +0x8e fp=0xc0000617c8 sp=0xc000061780 pc=0x44018e
local-ai  | 2023-04-26T00:13:48.631307082Z runtime.gcenable.func1()
local-ai  | 2023-04-26T00:13:48.631316122Z  /usr/local/go/src/runtime/mgc.go:178 +0x26 fp=0xc0000617e0 sp=0xc0000617c8 pc=0x435466
local-ai  | 2023-04-26T00:13:48.631321955Z runtime.goexit()
local-ai  | 2023-04-26T00:13:48.631333627Z  /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000617e8 sp=0xc0000617e0 pc=0x482821
local-ai  | 2023-04-26T00:13:48.631338069Z created by runtime.gcenable
local-ai  | 2023-04-26T00:13:48.631340877Z  /usr/local/go/src/runtime/mgc.go:178 +0x6b
local-ai  | 2023-04-26T00:13:48.631343645Z 
local-ai  | 2023-04-26T00:13:48.631346219Z goroutine 4 [GC scavenge wait]:
local-ai  | 2023-04-26T00:13:48.631352052Z runtime.gopark(0xc000088000?, 0xaacfb8?, 0x1?, 0x0?, 0x0?)
local-ai  | 2023-04-26T00:13:48.631363424Z  /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000061f70 sp=0xc000061f50 pc=0x453eb6
local-ai  | 2023-04-26T00:13:48.631366783Z runtime.goparkunlock(...)
local-ai  | 2023-04-26T00:13:48.631372288Z  /usr/local/go/src/runtime/proc.go:387
local-ai  | 2023-04-26T00:13:48.631375320Z runtime.(*scavengerState).park(0xe36bc0)
local-ai  | 2023-04-26T00:13:48.631381090Z  /usr/local/go/src/runtime/mgcscavenge.go:400 +0x53 fp=0xc000061fa0 sp=0xc000061f70 pc=0x43e0d3
local-ai  | 2023-04-26T00:13:48.631384196Z runtime.bgscavenge(0x0?)
local-ai  | 2023-04-26T00:13:48.631387177Z  /usr/local/go/src/runtime/mgcscavenge.go:628 +0x45 fp=0xc000061fc8 sp=0xc000061fa0 pc=0x43e6a5
local-ai  | 2023-04-26T00:13:48.631390226Z runtime.gcenable.func2()
local-ai  | 2023-04-26T00:13:48.631402767Z  /usr/local/go/src/runtime/mgc.go:179 +0x26 fp=0xc000061fe0 sp=0xc000061fc8 pc=0x435406
local-ai  | 2023-04-26T00:13:48.631408944Z runtime.goexit()
local-ai  | 2023-04-26T00:13:48.631421482Z  /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000061fe8 sp=0xc000061fe0 pc=0x482821
local-ai  | 2023-04-26T00:13:48.631427638Z created by runtime.gcenable
local-ai  | 2023-04-26T00:13:48.631430572Z  /usr/local/go/src/runtime/mgc.go:179 +0xaa
local-ai  | 2023-04-26T00:13:48.631436909Z 
local-ai  | 2023-04-26T00:13:48.631439812Z goroutine 5 [finalizer wait]:
local-ai  | 2023-04-26T00:13:48.631445451Z runtime.gopark(0x1a0?, 0xe378a0?, 0x60?, 0x78?, 0xc000060770?)
local-ai  | 2023-04-26T00:13:48.631525149Z  /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000060628 sp=0xc000060608 pc=0x453eb6
local-ai  | 2023-04-26T00:13:48.631537876Z runtime.runfinq()
local-ai  | 2023-04-26T00:13:48.631541428Z  /usr/local/go/src/runtime/mfinal.go:193 +0x107 fp=0xc0000607e0 sp=0xc000060628 pc=0x4344a7
local-ai  | 2023-04-26T00:13:48.631548354Z runtime.goexit()
local-ai  | 2023-04-26T00:13:48.631552373Z  /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000607e8 sp=0xc0000607e0 pc=0x482821
local-ai  | 2023-04-26T00:13:48.631556795Z created by runtime.createfing
local-ai  | 2023-04-26T00:13:48.631566661Z  /usr/local/go/src/runtime/mfinal.go:163 +0x45
local-ai  | 2023-04-26T00:13:48.631571688Z 
local-ai  | 2023-04-26T00:13:48.631582007Z goroutine 6 [sleep]:
local-ai  | 2023-04-26T00:13:48.631586602Z runtime.gopark(0x1014f278b3c0?, 0xc000062788?, 0xc5?, 0x37?, 0xc0000b7930?)
local-ai  | 2023-04-26T00:13:48.631596872Z  /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000062758 sp=0xc000062738 pc=0x453eb6
local-ai  | 2023-04-26T00:13:48.631600397Z time.Sleep(0x2540be400)
local-ai  | 2023-04-26T00:13:48.631603157Z  /usr/local/go/src/runtime/time.go:195 +0x135 fp=0xc000062798 sp=0xc000062758 pc=0x47f695
local-ai  | 2023-04-26T00:13:48.631609759Z github.com/valyala/fasthttp.(*workerPool).Start.func2()
local-ai  | 2023-04-26T00:13:48.631612639Z  /go/pkg/mod/github.com/valyala/fasthttp@v1.44.0/workerpool.go:67 +0x56 fp=0xc0000627e0 sp=0xc000062798 pc=0x7d41f6
local-ai  | 2023-04-26T00:13:48.631615507Z runtime.goexit()
local-ai  | 2023-04-26T00:13:48.631618180Z  /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000627e8 sp=0xc0000627e0 pc=0x482821
local-ai  | 2023-04-26T00:13:48.631643792Z created by github.com/valyala/fasthttp.(*workerPool).Start
local-ai  | 2023-04-26T00:13:48.631650041Z  /go/pkg/mod/github.com/valyala/fasthttp@v1.44.0/workerpool.go:59 +0xdd
local-ai  | 2023-04-26T00:13:48.631654705Z 
local-ai  | 2023-04-26T00:13:48.631659179Z rax    0x43e0100
local-ai  | 2023-04-26T00:13:48.631663561Z rbx    0x7ffd4e6d1630
local-ai  | 2023-04-26T00:13:48.631667460Z rcx    0x43e02f0
local-ai  | 2023-04-26T00:13:48.631671803Z rdx    0x43e02f8
local-ai  | 2023-04-26T00:13:48.631676353Z rdi    0x43e0100
local-ai  | 2023-04-26T00:13:48.631680512Z rsi    0x1f8
local-ai  | 2023-04-26T00:13:48.631684735Z rbp    0x7ffd4e6d14d0
local-ai  | 2023-04-26T00:13:48.631689321Z rsp    0x7ffd4e6d1400
local-ai  | 2023-04-26T00:13:48.631693725Z r8     0x43e0100
local-ai  | 2023-04-26T00:13:48.631697789Z r9     0x7f5a542eebe0
local-ai  | 2023-04-26T00:13:48.631701977Z r10    0xfffffffffffff327
local-ai  | 2023-04-26T00:13:48.631706132Z r11    0x200
local-ai  | 2023-04-26T00:13:48.631710458Z r12    0x7ffd4e6d1470
local-ai  | 2023-04-26T00:13:48.631714481Z r13    0x43e00c8
local-ai  | 2023-04-26T00:13:48.631717749Z r14    0x7ffd4e6d1440
local-ai  | 2023-04-26T00:13:48.631720182Z r15    0x7ffd4e6d1618
local-ai  | 2023-04-26T00:13:48.631722494Z rip    0x8da2a9
local-ai  | 2023-04-26T00:13:48.631724802Z rflags 0x10206
local-ai  | 2023-04-26T00:13:48.631727733Z cs     0x33
local-ai  | 2023-04-26T00:13:48.631731851Z fs     0x0
local-ai  | 2023-04-26T00:13:48.631735934Z gs     0x0

I don't quite know what the issue is, and am unfamiliar with go programming, so I don't quite know where to start. I can run it locally, on my 13900K and it works without issue. The server I am trying to run it on is a proxmox host with a Intel 9900K as the processor. I don't quite know what else I'd need to provide to re-create this, but I'm hoping the logs will help.

On another side note, I tried running this on my k3s cluster, that is running off some old Xeons, and I couldn't create the container at all. After googling I guess llama.cpp needs avx2 which my xeons don't have, is this accurate for local-ai as well (would assume so, but confirmation never hurts).

Please let me know If I can provide anything else!

mudler commented 1 year ago

Hey @ChaseDDevelopment , thanks for the report!

I think this is a dup of #88 . I've added a comment with more details there, closing this one.