TheR1D / shell_gpt

A command-line productivity tool powered by AI large language models like GPT-4, will help you accomplish your tasks faster and more efficiently.
MIT License
9.43k stars 746 forks source link

fix: trim markdown style code annotations from shell output #381

Closed hamidzr closed 9 months ago

hamidzr commented 10 months ago

With some more recent models whenever I use the shell output format the response includes something like this and we take those out here. We might be able to achieve this with better role descriptions aka prompts but I couldn't get that to reliably work and this is easier to reason about.

\```zsh
cowsay hi
\```

Tested on the following model


MODEL | DESCRIPTION | CONTEXT WINDOW | TRAINING DATA
-- | -- | -- | --
gpt-4-1106-preview | GPT-4 TurboNewThe latest GPT-4 model with improved instruction following, JSON mode, reproducible outputs, parallel function calling, and more. Returns a maximum of 4,096 output tokens. This preview model is not yet suited for production traffic. Learn more. | 128,000 tokens | Up to Apr 2023

MODEL   DESCRIPTION CONTEXT WINDOW  TRAINING DATA
gpt-4-1106-preview  GPT-4 TurboNew
The latest GPT-4 model with improved instruction following, JSON mode, reproducible outputs, parallel function calling, and more. Returns a maximum of 4,096 output tokens. This preview model is not yet suited for production traffic. [Learn more](https://openai.com/blog/new-models-and-developer-products-announced-at-devday).   128,000 tokens  Up to Apr 2023
TheR1D commented 9 months ago

Hi @hamidzr Thank you for interest and effort. I agree that this issue should be fixed with proper roles. Since we can't adapt ShellGPT prompts to every LLM OpenAI has, I think we will change default model of ShellGPT to one of GPT-4 in comming updates.

Sobsz commented 6 months ago

I've just stumbled upon the same issue with Mistral 7B Instruct (which is provided as an example in the Ollama guide), and it's unfortunately much more stubborn in adding Markdown code blocks. I see no harm in having this feature, since it's extremely unlikely for an actual shell command to start with ```bash. (Although, one could argue that a model unable to refrain from using Markdown is unlikely to be useful...)

EmVee381 commented 1 week ago

I've just stumbled upon the same issue with Mistral 7B Instruct (which is provided as an example in the Ollama guide), and it's unfortunately much more stubborn in adding Markdown code blocks. I see no harm in having this feature, since it's extremely unlikely for an actual shell command to start with ```bash. (Although, one could argue that a model unable to refrain from using Markdown is unlikely to be useful...)

Same here, I use GPT-4 Free with ChatGPT authenticated API through 'HAR' files, so it is like being forwarded directly to free ChatGPT. It also cannot strip markdowns from replies. :) Regarding your note in parentheses, this model is definitely useful; it's GPT-4. I would expect that the Hamidzr version would be merged, as I think that would resolve the issue.