search

ohmplatform / FreedomGPT

This codebase is for a React and Electron-based app that executes the FreedomGPT LLM locally (offline and private) on Mac and Windows using a chat-based interface
http://www.freedomgpt.com
GNU General Public License v3.0
2.61k stars 358 forks source link

[BUG REPORT] If I instruct it to write a story, it starts repeating itself half way through. #15

Open gaveitatry2 opened 1 year ago

gaveitatry2 commented 1 year ago

I used this Windows build of Freedom GPT: https://github.com/ohmplatform/freedom-gpt-electron-app/issues/13

90% or more of the time, if I instruct the AI to write a story, the story starts off great and interesting but halfway through it will start repeating the same few lines of text over and over and over again. I'd rather it just stop writing.

Is this something that you are aware of and is there anything you can do to make this behavior stop?

TheRealTeaLeaf commented 1 year ago

I experienced the same on windows and mac too. I also don't think that it can build longer output than 250 words or so and it seems that if the input gets a bit longer, then it doesn't understand the complete request - there might be a limit around 250 characters or so.

cooperdk commented 1 year ago

I experienced the same on windows and mac too. I also don't think that it can build longer output than 250 words or so and it seems that if the input gets a bit longer, then it doesn't understand the complete request - there might be a limit around 250 characters or so.

There is definitely not. I asked it to write a Python/Flask application for scraping a web page and storing the data in a database. It did. The response was well over 3 KB.

Also, using resources\models\windows\chat.exe it NEVER repeated itself, and I made it write stories up to 3 KB in size.

I experience the loop too, but ONLY in the GUI. There must be something wrong with their commandline or something.

(EDIT: I actually just made it loop in the CLI program)

It seems that this primarily occurs after you have prompted the AI a couple of times. It is likely due to memory issues. AI has a tendency to f**k up when run on CPU while it doesn't on GPU. CUDA is specifically designed for AI - a regular CPU is not.

TheRealTeaLeaf commented 1 year ago

I experienced the same on windows and mac too. I also don't think that it can build longer output than 250 words or so and it seems that if the input gets a bit longer, then it doesn't understand the complete request - there might be a limit around 250 characters or so.

There is definitely not. I asked it to write a Python/Flask application for scraping a web page and storing the data in a database. It did. The response was well over 3 KB.

Also, using resources\models\windows\chat.exe it NEVER repeated itself, and I made it write stories up to 3 KB in size.

I experience the loop too, but ONLY in the GUI. There must be something wrong with their commandline or something.

(EDIT: I actually just made it loop in the CLI program)

It seems that this primarily occurs after you have prompted the AI a couple of times. It is likely due to memory issues. AI has a tendency to f**k up when run on CPU while it doesn't on GPU. CUDA is specifically designed for AI - a regular CPU is not.

thanks for the idea - i only tried with the GUI, i will have another try. However not only the looping issue exist in the GUI, just to give you an example: i copy-pasted an article and asked to summarize it and then it always only summarizes the first few lines, like the rest of the text wouldn't be there.

ItsPi3141 commented 1 year ago

That's not related to the app. Its just the model. I think freedomGPT will implement a settings page soon so once that is released, you can change the temp settings and find the right setting.