Open amirshawn opened 1 month ago
Hello there and thanks for trying OpenHands!
Installing php. You can try two things:
For your editing issue, which model are you using? Are you using gpt-4o? There is an open editing issue with gpt-4o currently: https://github.com/All-Hands-AI/OpenHands/issues/3679 There is work being done on improving the editing of files so hopefully this gets fixed in the near future.
Hi Mamoodi! I have been mostly using claude sonnet 3.5 since it has the context caching working. I can't wait for Gemini to work because my project is using up all of Claude's context.
I've found that it struggles to edit files consistently. I'm not sure if it's a permissions issue but it's very unpredictable. I'm not a very advanced programmer so it's hard for me to debug it but what I seem to find is that after a long conversation it starts not being able to edit or overwrite files. What it's trying to insert ends up at the end of the file creating a duplicate. The only solution I've found so far is to go into docker and delete the image and then rebuild the whole thing and then it works again for a while. My only guess since it happens mid conversation is that it might have to do with the prompt caching and possibly once the ttl of the cache has expired and the context refreshes then the ability to edit files is disrupted. That's just a guess though and I haven't dug into your code to see how any of this works.
Another thing that's caused me a lot of issues is when the model seems to go rogue. I ask it to do something and then it starts doing something completely different. It's almost like it reads some instructions in a file and starts going on it's own path and the pause button doesn't seem to stop it in time. I like the mode where you can approve every action but that slows everything down significantly. I wonder if there's a way to group a set of actions and then approve the set and then have it do the set as opposed to every single thing it does. Also not being able to toggle the setting to turn confirmation mode on and off during a conversation is not ideal.
Another bug I've found is that when I write a response in the middle of an action during confirmation mode, the whole thing stalls and the next action doesn't have an option to approve it or deny it. I have to reload the page and resume the session to get it to work again.
I appologize I'm not able to help with the codebase but I hope I can be helpful in finding bugs that become evident while using the system.
Thank you! Amir
Hi Amir, thanks for your detailed write up.
Regarding the editing issue, there is definitely improvements that can be done on the OpenHands side. I definitely know there is work being done but not sure when it will be ready. Here is the mega issue that is relevant to editing: https://github.com/All-Hands-AI/OpenHands/issues/3231
The going rogue issue. I have run into this myself where I give it something vague and it starts going on a completely different path. For this, the solution has been to give it more detailed instructions. For example I might say "Create this file that does X but do not run it" or "You already have a project setup in your workspace....". Otherwise it finds the files and starts creating an app haha.
My number one suggestion right now is when you give it a prompt and it doesn't behave correctly, press the Thumbs Down
button on the UI and share it if you are able to. This will give feedback to the devs to be able to make things better:
https://docs.all-hands.dev/modules/usage/feedback
Hopefully OpenHands can get better on these issues over time.
Hi Mamoodi,
Quick question, I've noticed that the best way to avoid the errors during the editing of files is if the file is deleted and replaced. I have to ask it each time to do that but then the next time it tries to edit again. Is there a way to give it instructions that it will retain in it's context?
That's a good question. I'm not an expert in how LLMs behave but you could try giving it a prompt like: "From now on, whenever you need to edit a file, just replace the file entirely with the new content". See how that goes. This might work if your file has little content. Unfortunate this happens though. Hopefully this gets fixed soon.
Hi @mamoodi I was curious if you know of a way to give custom instructions to the model that will persist longer than a few turns? I find that it keep repeating the same mistakes and it would be nice to have some instructions that stick with it like you can in the ui of claude and chatgpt.
OpenHands doesn't specifically provide that functionality. If you tell the LLM to "Never do X from now on" and it does do it... then maybe you can open a bug specifying your instructions and logs of when it goes against your ask. I don't know if this would be a OpenHands bug or what but if you open one with more information, at least someone can take a look.
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.
Hi,
I've been playing around with this package and I am highly impressed! I'm trying to use this to work on my Laravel application and a problem I've run across is not having access to composer or PHP in the docker container this is running on. How would I go about installing that?
Another issue I'm having is sometimes I run into an issue where it's unable to edit files correctly. It ends up appending the new data to the end of the file, then tries correcting itself and gets into a loop doing that. I assume it must be some sort of permissions issue but it's weird because it doesn't happen all the time. Do you have any solutions for this?
Thank you very much for this awesome package! I appreciate your help :)