Open nmstoker opened 2 years ago
Also, by using the .py header it adds another limitation:
It means that Parsons problems that incorporate a blank section in the def/class section can end up passing no matter what people enter, since it's ignoring the solutions lines for that part and instead getting lines from the header.
Yes, they do! That has to do with our original implementation with the Berkeley OKPy grader. I think we could get away with only the YAML if we were quite strict with how the doctests were declared in the YAML. Perhaps a YAML checker would be in order? Initial PR would be great. So sorry I missed these issues, I apparently wasn't on full notifications for this repo. So s
I may be missing some sensible reason, but it seems like the Parsons problem files repeat various details and thus aren't "DRY".
There's a risk that the tests explained in the problem description in the .yaml file end up differing from those included in the .py header.
I wonder if it might be feasible to simply do away completely with the .py header - after all, it isn't even a complete stand-alone Python script - and then simply rely on the details in the .yaml file.
I see there are slight (intentional) differences in the problem description from the .yaml file and the .py docstrings anyway, and the ability to add text/formatting you wouldn't likely put into a docstring seems useful, so those details seem like they should remain in the .yaml file along with the doctest text they contain and then it's simply a matter of using that for feeding the doctest process.
Hope that makes sense and that I haven't completely misinterpreted how the code is intended to be 🙂 Happy to have a go at this with an initial PR to show how this could be achieved, if you like?