search

SciSharp / LLamaSharp

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.
https://scisharp.github.io/LLamaSharp
MIT License
2.71k stars 350 forks source link

Split platform-specific binaries for NuGet backends #957

Closed m0nsky closed 1 month ago

m0nsky commented 1 month ago

This PR refactors the NuGet package structure for the LLamaSharp.Backend.Cuda11, LLamaSharp.Backend.Cuda12 and LLamaSharp.Backend.Vulkan packages by splitting the platform-specific binaries (Windows and Linux) into separate packages, and adds them as dependencies on the main packages. This way we can work around the 250MB limit for NuGet which broke the latest release.

Package Name File Size Before (MB) File Size After (MB)
LLamaSharp.Backend.Cuda11 286.36 MB 0.032 MB
LLamaSharp.Backend.Cuda11.Windows N/A 142.56 MB
LLamaSharp.Backend.Cuda11.Linux N/A 143.82 MB
LLamaSharp.Backend.Cuda12 284.33 MB 0.032 MB
LLamaSharp.Backend.Cuda12.Windows N/A 141.62 MB
LLamaSharp.Backend.Cuda12.Linux N/A 142.74 MB
LLamaSharp.Backend.Vulkan 3.49 MB 0.032 MB
LLamaSharp.Backend.Vulkan.Windows N/A 1.54 MB
LLamaSharp.Backend.Vulkan.Linux N/A 1.98 MB
m0nsky commented 1 month ago

I tested installing the CUDA & Vulkan packages today and it seems to be working fine.