fix: bundle CUDA DLL into the release

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

| Name | Status | Preview | Comments | Updated (UTC) | | :--- | :----- | :------ | :------- | :------ | | **local-ai-web** | ⬜️ Ignored ([Inspect](https://vercel.com/louisgv/local-ai-web/9sVVEV9m49u9KmB3hyecNpwGvYj1)) | [Visit Preview](https://local-ai-web-git-61-bug-cuda-dlls-louisgv.vercel.app) | | Sep 16, 2023 11:49pm |

[louisgv]

https://github.com/bytecodealliance/wasmtime/blob/c19c729214e2237902eb177609643cb6523b7f2b/cranelift/native/src/lib.rs#L184

[LLukas22]

https://github.com/louisgv/local.ai/assets/65088241/6173b693-d020-4098-9f35-517a720f1044

@louisgv The callback to the UI of the fed tokens seams to slowdown the feeding process significantly. (Should be instant)

[louisgv]

The callback to the UI of the fed tokens seams to slowdown the feeding process significantly

Yes haha - there's a 42ms artificial lag that I introduced to make the UI a bit more smooth:

ref: https://github.com/louisgv/local.ai/blob/main/apps/desktop/src/providers/thread.ts#L160-L161

For non-accelerated machine and models, this is needed to have something showing :d.......

[LLukas22]

Currently this copies the cuda dlls next to the local.ai executable if the cargo tauri dev or cargo tauri build command is executed with the --features cublas flag. @louisgv Is this enought to include the dlls into the bundle?

[LLukas22]

@louisgv Whats the plan going forward on this? Can you take over and handle the auto update stuff?

[louisgv]

@LLukas22 yup, I'm on it now!

[louisgv]

Kinda want to wait for the Metal fix to land. My main fear with this PR is that the build seems flaky :d.... (OOM?...)

Perhaps we should remove some of the flakyness by building for either cuda or CL only?...

[louisgv]

Per the docs, there's still no metal chips on github runner VM yet: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources

So we will still need a metal self-hosted runner I think :d

[louisgv]

The last piece missing from this PR is a pipeline to upload the content of each zip artifact into a release.

1. Extract the zip artifact of the 3 jobs (mac, linux, window)
2. Have a 4th job called release, that's in charge of making a release and the update.json
3. Take the path and the signature to form a Tauri update package: https://tauri.app/v1/guides/distribution/updater/#static-json-file
4. Create the draft release