incorrect grading for governor question

[lswainemoore]

(This is after a bunch of JSON errors.)

[lswainemoore]

also

[lswainemoore]

something is fishy!

[lswainemoore]

Stepping back: I think this is a neat idea and well-intentioned, but I'm skeptical of the use of LLMs for grading all of these questions, many of which have a single correct answer, or a set of discrete correct answers that could be handled with some pretty simple pattern matching. I do think that for some of them, it could be an appropriate choice (e.g. "What did Susan B. Anthony do?"), but on the whole, I think if I was actually using this to study for the exam, the LLM's grading probably would've left me confused.

Other little issues I think caused by using LLMs:

• "POTUS" is sometimes accepted by the questions to which the answer is the president, but other times not.
• I had the answer 1781 accepted when 1787 is the correct answer:
• It asserted to me about only Maine borders Canada: . (When I hit this question again and answered the same way, I got a different, still incorrect, response. I think that the temperature should at least be turned down.)
• It didn't accept a tribe that is listed in the answer key for the test:

I recognize that I'm sniping from the sidelines here, and I'm not exactly offering to make these changes myself ;), so I want to reiterate that I respect the intentions behind this project, and I think it has the potential to be a useful tool.