[Bug]: wrong commit : the message is not about the diff

[iauhc]

Opencommit Version

v2.4.2

Node Version

v20.1.0

NPM Version

v9.8.0

What OS are you seeing the problem on?

Other Linux Distro

What happened?

The commit seems bugged.
.... (about 20 other files but not server.ts at all) roa/views/formation/liste_formation_programme.php roa/views/formation/voir_formation_programme.php autocron.php │

◇ 📝 Commit message generated │ └ Commit message: —————————————————— fix(server.ts): change the case of the port variable from lowercase to uppercase (PORT) feature(server.ts): add support for the process.env.PORT environment variable

Expected Behavior

Right commit message, at least about the filed changed

Current Behavior

Broken commit message

Possible Solution

No response

Steps to Reproduce

No response

Relevant log output

No response

[malpou]

Seems like something weird is happening causing the model to use the translation files instead of the current diff as the inspiration for the commit message response: https://github.com/di-sukharev/opencommit/blob/master/src/i18n/en.json

[iauhc]

Hmm, I use it in French, maybe this can explain. Will switch in en

[malpou]

Also, there is now a version 3 out, I see you're on 2.4.2 so upgrading might give another result.

[di-sukharev]

@iauhc please close the issue if solved

[Paillat-dev]

Still there in 3.0.4.

[Sped0n]

same here, It could be because there is too much code in the diff.

Edited: oops my bad, found out my problem is more related to pr #257

[github-actions[bot]]

Stale issue message

[Paillat-dev]

Stale but not this is an actual prompting issue wich breaks half of the commit messages

[di-sukharev]

reopening, will take a look at this

[tomasvts]

I just experienced this in a small commit using local ollama

◇  📝 Commit message generated
│
└  Generated commit message:
——————————————————
 Changed port variable name in server.ts and added support for configurable port.

The problem is in the prompts.ts file https://github.com/di-sukharev/opencommit/blob/master/src/prompts.ts#L37 which is adding an example result prompt to the engine, and sometimes, the engine might grab it as an actual git diff.

[Paillat-dev]

Yup, also, idk what models this is using, but if it is still using complete model it should imo switch to chat models. Also, the prompts need to be changed since the models where updated, they don't work the same anymore

[tomasvts]

I think the prompt is fine. This happens only when the context window is too large to handle and gpt3.5 (or ollama/mistral) "forgets" what was the purpose of the server.ts prompt example, and gives this unusual response.

I'll stick with gpt and smaller commits for now to avoid this problem.

[Paillat-dev]

I did notice that too, however, I also noticed that depending on the model, it dosen't follow well some instructions, like adding a description, and also, the emojis used are always wrong as per gitmoji. see #249

[di-sukharev]

heads down, the prompt is weak, it can be 10x better, it happens to fail on large diffs, we should improve splitDiff() function and definitely rework the prompt and make it also configurable, so everybody can play with it locally, any ideas?

[Paillat-dev]

Dropping some ideas here, it could be nice to be able to set custom settings per repository, including prompt context that a user can add (ex. Your repo is about a collection of separate bash scripts, it could be useful for the model to know that they aren't linked together). Maybe some config folder like .oco or something. Also, standardizing the prompt formats would allow to easily add new ones, maybe even locally (ex. prompts need to contain <<DIFF>>, <<REPO_NAME>>, <<USER_CONTEXT>> which willl then be replaced programmatically. I am not familiar with ts tough so I did not dig too much in this codebase.

[di-sukharev]

@Paillat-dev or it can be learning on the repo with every commit and make better commits bit by bit 💣 not sure it will help though..

as for the different commit message — its not a bug with library, gpt is simply confused in a large git diff. If the there are too many changes — we can do multiple commits instead of just one, who wants to do a PR, @Paillat-dev would you like to contribute to the repo?

as a temporary workaround, commit in batches, e.g. by folders or commit large files separately

[Paillat-dev]

As I said I almost never touched Typescript nor JavaScript in my entire life so I don't think I can do much other than opening issues 😅

[digitalstudium]

Same shit.

◇  2 staged files:
  03_network/extra_kube_yaml/ldap.yaml
  08_sonarqube/values/sq.yaml
│
◇  📝 Commit message generated
│
└  Generated commit message:
——————————————————
1. server.ts: fix port variable case, add support for configurable port
2. Create LDAP Service and Endpoints configuration file
3. Update SonarQube values with new properties and security configurations

It would be wonderful to remove this first message... Tried to investigate source code of opencommit but failed, because I'm not a JS dev

[digitalstudium]

And even when I change a single file, same thing happens.

$ opencommit
┌  open-commit
│
◇  1 staged files:
  08_sonarqube/values/sq.yaml
│
◇  📝 Commit message generated
│
└  Generated commit message:
——————————————————
 fix(server.ts): change port variable case (fix semantics)
feat(server.ts): add support for configurable port (run app on a desired port)

modify(sq.yaml): enable persistence and set up SonarQube with LDAP authentication.