search

cztomsik / ava

All-in-one desktop app for running LLMs locally.
https://avapls.com
Other
400 stars 15 forks source link

Cannot download model #16

Closed lming closed 7 months ago

lming commented 7 months ago

Got Unexpected Error when clicking "Download" of a model, same error when clicking "Import" button. Console shows error

01:17:31 debug default: Missing dependency: src.api.download.POST /download__struct_18238
01:17:31 debug server: POST /api/download 500

Both headless web app and mac app behaviors the same.

image
cztomsik commented 7 months ago

Hey, thanks for reporting, should be fixed now. I did some refactoring yesterday (extracting the server-code to its own repo) and looks like I messed up something, sorry. :)

Just wait for a new build if you are not building yourself.

lming commented 7 months ago

Hey - thanks for the quick turn around - it still doesn't work for me

lming@mmax ava % zig build
Fetch Packages [3/3] tokamak... /Users/lming/work/ava/build.zig.zon:13:21: error: hash mismatch: expected: 12206b21956424fa8d78b5c959fac09247de583ace30e7440aebeba936e098d8fb3a, found: 12201735ec280c5444883ea131682f2eda70be65b120b661a7839dada3636f962959
            .hash = "12206b21956424fa8d78b5c959fac09247de583ace30e7440aebeba936e098d8fb3a",

If I modified the hash to the found value, it complains another error:

lming@mmax ava % zig build
warning: FileNotFound: /Users/lming/.cache/zig/p/1220491e02c1f645f41d46803d6fc039cbeef125a1d106f90a9cc718f6782df28a08/build.zig
error: FileNotFound

I am new to Zig so I don't know how to fix this error...

cztomsik commented 7 months ago

You best bet is to just grab a copy from github actions https://github.com/cztomsik/ava/actions/runs/7774124705

Zig build system is a new thing, it might also have some bugs.

Cached files are in ~/.cache/zig on macos and probably something similar on linux. Also, always try with the newest zig version. I am updating every week at least.

EDIT: For macos builds, you also need ggml-metal.metal file to be in the same directory. You can find it in the llama.cpp repo. If you have time, you can try to figure out why the ./src/macos/create-dmg.sh does not work in the github pipeline, because that would the problem entirely and you could then just grab dmg there.

cztomsik commented 7 months ago

The original issue was fixed. Pipelines are also passing so I think either it works now or it has to be something specific to your machine.

MartinBspheroid commented 7 months ago

I'd like to give this project a go, but I'm stuck on this as well. Grabbed a brand new headless build from pipeline but this is what I've got. image

cztomsik commented 7 months ago

Can you check the Network tab in google chrome? I don't have linux machine here to check it out but macos headless build works fine (download starts and looks it goes as expected). Also, there is a tab with system logs in the Settings screen.

MartinBspheroid commented 7 months ago

Sure, it's responding with {"error":"MissingField"} in response of POST: http://127.0.0.1:3002/api/models

here are logs from System tab: 


16:17:02 debug server: GET /app.js 200
16:17:03 debug server: GET /api/chat 200
16:17:03 debug server: GET /api/models 200
16:17:07 debug server: GET / 200
16:17:07 debug server: GET /app.js 200
16:17:07 debug server: GET /api/models 200
16:17:07 debug server: GET /api/chat 200
16:17:14 debug server: GET /api/models 200
16:17:14 debug server: GET /api/system-info 200
16:18:50 debug server: POST /api/download 200
16:18:50 debug server: POST /api/models 500
16:18:50 debug server: GET /api/models 200
cztomsik commented 7 months ago

Apologies, there was one small bug but I think it will not build currently for linux because there's a problem in the llama.cpp repo. Here's what we are waiting for to get merged https://github.com/ggerganov/llama.cpp/issues/5537