Closed XuGW-Kevin closed 2 months ago
Thank you for the feedback! We have taken this into consideration. That is the reason why we have defined all of the scientific constants in the problem. For example, as you can see in question 35.1, we stated "Given the free electron mass is 9.109e-31kg, speed of light is 3e8m/s and the Planck constant is 6.626e-34J*s.". When we are doing error analysis for LM generation on SciCode, we conclude this kind of error as "not following instructions".
Thank you for the clarification! Using predefined scientific constants indeed helps ensure consistency in the problem solutions.
It seems that the code is using
atol=1e-8, rtol=1e-5
to judge whether two results are close enough. I personally find it too strict sometimes. For example, in problem 35.1, the following code is judged as correct:However, the following code is considered to be wrong:
These two codes appear to have the same functionality. It seems unfair to give them a different judgement. Maybe it would be better to change the threshold to 1e-5 or 1e-4.