search

plandex-ai / plandex

AI driven development in your terminal. Designed for large, real-world tasks.
https://plandex.ai
GNU Affero General Public License v3.0
10.74k stars 750 forks source link

Weird code merging from deepseek #184

Open viraptor opened 2 months ago

viraptor commented 2 months ago

I've tried using plandex with deepseek. The responses I see in plandex convo seem completely fine and include the whole file being changed. Unfortunately plandex is spinning for a very long time processing those answers and finally injects a lot of copied code - the new code gets pasted in the middle of the file being edited. (python code)

I'd love for those edits to be handled better.

viraptor commented 2 months ago

Aider seems to ask for a search/replace pair when working with deepseek and for any issues applying it it just complains to the LLM and makes it try again. Seems to work quite well in practice.

danenania commented 2 months ago

Are you using deepseek for all the roles? You can check with plandex models.

There's a syntax check and a verification step as well, but it may be that deepseek is having trouble with Plandex's line-number based editing. There are pros and cons to this approach vs. aider's search/replace pair strategy... the solution will likely be to change strategies depending on which model is being used, which we are working towards. The prerequisite is a good set of evals that lets us compare different prompting/editing strategies against different models, and we are close to having this in place. So stay tuned :)

viraptor commented 2 months ago

Yup, all the roles. It seems like everything works fine, apart from the change merging.

For the editing format, yes, aider uses diff (as in a search/replace pair) instead for deepseek coder. They already did that homework here so it can save you some time 😉 https://aider.chat/docs/leaderboards/ (includes % reliability for the best method)