tracehubpm / reports-check-action

Bug Reports (GitHub Issues) Quality Checker
MIT License
10 stars 1 forks source link

Example output is still can be achieved in analysis summary #47

Closed h1alexbel closed 4 months ago

h1alexbel commented 4 months ago

After bug report analysis was performed by some Language Model we still can get the example summary. Consider this example:

Input bug report:

title: Some new bug report for testing reports-check.yml
body: something new

In this case model would throw us an example of summary:

@h1alexbel thanks for the report, but here some unclear moments:

* It's not clear what is the expected value `xyz` variable should have (Bug report does not describe what is expected to see). Consider more examples/tests that show what is the expected to see instead.
* It's not clear where exactly in the code IOException has been thrown (Bug report does not have location). Please provide either where this issue is located (class, file) or runnable examples that shows the problem.
* It's not clear how to reproduce `email` sending (Bug report does not have steps to reproduce). Craft steps on how to reproduce this issue.
* It's not clear what are the values in `words` that are failing the WordsTest (Bug report does not say what you saw instead of expected). Provide what is the result you saw that against your expectations.

Generated with `Phind/Phind-CodeLlama-34B-v2`

Obviously we have a some stopper for this case:

... (user-prompt.ts)
Please strictly adhere to the example template provided, but don't copy it, instead adjust this template to the issue context.

It indeed helps, but not 100% defend us from cases as above. We should fix it

github-actions[bot] commented 4 months ago

@h1alexbel thanks for the report, but here some unclear moments:

Generated with Phind/Phind-CodeLlama-34B-v2