search

lmstudio-ai / lmstudio-bug-tracker

Bug tracking for the LM Studio desktop application
10 stars 3 forks source link

[BUG] "Log Prompts and Responses" does not work #81

Open dgreensp opened 3 months ago

dgreensp commented 3 months ago

For me, on macOS, LM Studio 0.2.31, changing "Verbose Server Logs" from ON to OFF does not seem to have any effect, even though it prints "Verbose server logs are DISABLED". It seems I am forced to log the entire contents of every request and response.

dgreensp commented 2 months ago

@yagil You told me on Hacker News to "at" you.

I just tried 0.3.1 (mac) and this is still happening. Regardless of the "Verbose Logging" or "Log Prompts and Responses" settings, every request is logged, and every response is logged. That's too much logging for me.

I'm experimenting with calling LM Studio programmatically (via the chat API) to do batch processing of short pieces of data, and eventually I might have to use something that has a cache, rather than feeding in the prompt each time (I know this is possible with llama.cpp, which LM Studio uses internally), but at the very least I don't want to write a copy of the prompt to disk for every single request.

yagil commented 2 months ago

Thanks for the ping @dgreensp. Are you able to share a screenshot (with any sensitive information blocked off) that shows the logs you're hoping to not appear?

dgreensp commented 2 months ago

Sure. It's this kind of thing (where the generated output here, in case it's hard to spot, is just the word "ok"):

image
dgreensp commented 1 month ago

Has anyone looked into this yet?

yagil commented 1 month ago

@dgreensp want to make sure I understand. Is the ask to not log any incoming request content? Is it just the content or the entire thing?

Also, is the motivation data privacy or "cost" of disk writes, or other?

dgreensp commented 1 month ago

@yagil This is not a feature request, it is a bug report; you guys literally created a switch in the UI labeled "Log Prompts and Responses" that doesn't seem to work.

There are two on/off switches:

If both are turned off, the server should not log the entire request body or the entire response body.

A web server, for developers to use, that regularly handles requests in the 10s of KB, that always logs the entirety of every request to the console, no matter what flags you give it is... not a normal thing. It's about the cost and the principle. You can imagine generating a gigabyte of logs by, for example, doing some batch processing that involves making 200,000 5K requests. You can say, well who doesn't have a gigabyte of extra disk lying around, but it's also displaying that gigabyte of text in a UI panel. That could crash the app or take up a lot of RAM. If you leave the server running for days, the log will just get longer. On disk, a new file is started every day, apparently, but the UI panel does not clear.

Anyway, I don't think the feature requires too much justification, it makes sense, and there's already a switch in the UI for it. I'm just asking for it to work (or for someone to investigate whether it has a bug).

yagil commented 1 month ago

Log Prompts and Responses indeed seems to not work as intended, this is a bug