search

ngxson / wllama

WebAssembly binding for llama.cpp - Enabling on-browser LLM inference
https://huggingface.co/spaces/ngxson/wllama
MIT License
444 stars 23 forks source link

Feature: list the available local models from the cache #112

Open synw opened 2 months ago

synw commented 2 months ago

I would like to get a models list from what is in the cache, to implement an "available local models" feature.

I saw that there is a list function in the cache manager but it is not very convenient to get the models names, as this is a list of files and that the file names are not very usable: for example: 2f29f4de9810c95c9c4676f6939af5afc48396b1_Phi-3-mini-4k-instruct-q4-00001-of-00010.gguf.

ngxson commented 2 months ago

There is also cacheEntry.metadata object, which holds the original URL of the model. Is this what you want?

https://github.com/ngxson/wllama/blob/e7fe626/src/cache-manager.ts#L8

Example usage: https://github.com/ngxson/wllama/blob/e7fe626e1bb45844af075728739799511d867786/examples/main/src/utils/wllama.context.tsx#L68

synw commented 2 months ago

Yes thanks for pointing me to this example. I used your function with removing references to external types and list of models and it works well.

Would it be possible to have a generic high level getManageModels function integrated in the library?