Inconsistency between reported grades and workflow output

[rlalik]

I am running a autograder workflow. The reported result is correct as shown here: however when looking into autograder report, the value is different: The linter here should be 0.8490566037735854

So I am developing my own grader for python which uses pylint to rate the code quality. As pylint rates in range 0-10, by grader takes the pylint score and calculates pylint_score/10.0 * max_score.

My grader is sending following string:

eyJ2ZXJzaW9uIjozLCJzdGF0dXMiOiJwYXNzIiwidGVzdHMiOlt7Im5hbWUiOiJjb21wbGV4X251bS5weSIsInN0YXR1cyI6InBhc3MiLCJzY29yZSI6MC44NDkwNTY2MDM3NzM1ODQ5fV0sIm1heF9zY29yZSI6MX0K

which decodes to

{"version":3,"status":"pass","tests":[{"name":"complex_num.py","status":"pass","score":0.8490566037735849}],"max_score":1}

So from the first picture we see that the value is correct, 3 (from pytest) plus 0.8490566037735849 from linter.

But the report still shows 1. By quick inspection of the code, does it count only tests status value and if pass then counts 1?

My grader provides following status:

• failed if pylint returns fatal error
• error if pylint returns at least on error
• pass in any other case

For me even if there are some issues, the result is still positive. I expected that the reporter will use the score value and not status flag. At least this is not documented in the reporter's README.

Can you please clarify the proper behaviour?

[panjd123]

Hi, I also encountered this issue. I provided a possible solution in this PR #11 , and it works well for me.

[tatsy]

Sorry to interrupt the discussion, but I have encountered a similar issue.

When I use this "autograding-grading-reporter" action, the grade is not displayed either the "Annotations" (as posted by @rlalik) or the GitHub classroom home.

I cannot resolve the problem and have not figured out what the problem is. If someone knows the solution, I'd appreciate for it if they let me know.

Here is the screen shot for my Github Actions.

And that for the result.

Also, that in the GitHub classroom.

Thank you.

[tatsy]

Excuse me answering my own question, but the point is that I should set the name of the check as "run-autograding-tests". This point must be documented on README.md as well, I think.

[alessandropellegrini]

Excuse me answering my own question, but the point is that I should set the name of the check as "run-autograding-tests". This point must be documented on README.md as well, I think.

This has been a brainkiller for me, thanks!