search

BuilderIO / ai-shell

A CLI that converts natural language to shell commands.
MIT License
4.11k stars 264 forks source link

Worked a few times, then it stopped #105

Open jkolyer opened 6 months ago

jkolyer commented 6 months ago

I installed it and ran it for a few queries which worked. Then suddenly it stopped working, and now interprets all input as script. It gives an explanation about creating a script, which is not what was asked.

$ ai 'list all log files'


┌  AI Shell
│
◇  Your script:

script
molove commented 5 months ago

Same here

steve8708 commented 5 months ago

ah, i think this might be issues with GPT 3. could you try using GPT 4 (e.g. gpt 4 turbo) via ai config command and lmk if this improves?

molove commented 5 months ago

Same here

It seems to have started working again for me now. Unfortunately I can't remember the exact working that I had previously used that just produced "script"

jkolyer commented 5 months ago

i only have gpt3 access. please update the documentation accordingly.

On Wed, Apr 10, 2024 at 1:22 PM molove @.***> wrote:

Same here

It seems to have started working again for me now. Unfortunately I can't remember the exact working that I had previously used that just produced "script"

— Reply to this email directly, view it on GitHub https://github.com/BuilderIO/ai-shell/issues/105#issuecomment-2048372880, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADWXLERXSYQLFE3TUIEKCLY4WNQZAVCNFSM6AAAAABETQ32S6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBYGM3TEOBYGA . You are receiving this because you authored the thread.Message ID: @.***>

--

@.*** (c) 415 699 2175

molove commented 5 months ago

I only have GPT 3.5 too. It's sometimes working and sometimes not now for me

steve8708 commented 5 months ago

oh wow I thought GPT 4 was GA. you mean in openai dashboard/API you don't have access to any GPT 4 models?

molove commented 5 months ago

I just have a pay as you go account credited with a fixed amount rather than a subscription which limits me to GPT 3.5. I'm just a user who uses Linux as my main OS who occasionally need to tinker under the hood. I can't justify $20/month for something I rarely use

steve8708 commented 5 months ago

darn, I see. the problem we're finding is that GPT 3.5 has seemingly gotten worse with time at following (sometimes basic) instructions, which throws off our parsing logic.

I tried to address it previously but the lack of determinism around GPT 3.5 has been a challenge. GPT 4 handles this much better and more consistently

I would suggest in the short term, a PR to make modifications/improvements to the logic about parsing the command out of the response from GPT would be the best solution