Closed verhovsky closed 9 months ago
vercel[bot]
commented
9 months ago The latest updates on your projects. Learn more about Vercel for Git ↗︎
| Name | Status | Preview | Comments | Updated (UTC) |
|---|---|---|---|---|
| free-chat | ✅ Ready (Inspect) | Visit Preview | 💬 Add feedback | Dec 19, 2023 1:22am |
psugihara
commented
9 months ago cool, makes sense. better than crashing! it would be great to somehow capture the actual errors or server logs so users could report and/or understand them.
verhovsky
commented
9 months ago It seems like it crashes starting the server here because the llama process crashes:
If you load a model that's bigger than your RAM (starling-lm-7b-alpha.Q6_K.gguf on my 8GB Mac), and ask it to generate text, it hangs for a while then crashes the app. This way it should at least show an error message to the user (don't mind the tab bar, idk how that got there. It somehow appeared when I was trying to modify the tab hiding code and now I can't get rid of it, but it's not from this change):