Closed 0wwafa closed 5 days ago
@0wwafa There are UIs based on llama.cpp that are doing that already, ollama, llama.cui just to name a few https://github.com/dspasyuk/llama.cui Look at the list of UIs that are based/have support for the llama.cpp in Readme.
thanks.. I just said it would be nice to have it in llama.cpp.
@0wwafa If you just want to save prompts and generation you can set --prompt-cache file.txt and --prompt-cache-all in llama-server it will save it to a file.
I'm not familiar with the new server UI, but probably POST /slots/:id_slot
is not yet implemented on the UI.
Please note that this endpoint is to save/restore state file saved on server side. We don't provide solution for downloading/uploading state via API due to security and stability concern (uploading big file is quite challenge)
I'm not familiar with the new server UI, but probably
POST /slots/:id_slot
is not yet implemented on the UI.Please note that this endpoint is to save/restore state file saved on server side. We don't provide solution for downloading/uploading state via API due to security and stability concern (uploading big file is quite challenge)
Nope. I was saying to save the "state" on the client, by downloading/uploading a file.
@0wwafa If you just want to save prompts and generation you can set --prompt-cache file.txt and --prompt-cache-all in llama-server it will save it to a file.
hmm.. no.. I wish the session object to be saved on the client by clicking a button and uploaded at the start is you want to resume that session.
@0wwafa Resuming the state is possible using prompt embedding but it gets VRAM expensive very quickly. I have not introduced it in llama.cui but it is doable. Llama.cui just loads the previous prompts and responses but does not feed back to the model.
@0wwafa Resuming the state is possible using prompt embedding but it gets VRAM expensive very quickly. I have not introduced it in llama.cui but it is doable. Llama.cui just loads the previous prompts and responses but does not feed back to the model.
hmm there must be a "quick" way to do it like loading/saving the session and the token cache/status like from the llama binary but from the webapp.
This issue was closed because it has been inactive for 14 days since being marked as stale.
It would be useful to be able to import/export the full session from the UI. In this way a session started with a model could be continued with another or reimported with the same model to continue it some other time. I noticed in the javascript console there is an object containing everything. It could be enough to save and restore that. What do you think?