search

JMU-CS / less-java

Language and compiler designed for teaching introductory programming
GNU General Public License v3.0
6 stars 7 forks source link

Remove timing info from test results #47

Closed lam2mo closed 5 years ago

lam2mo commented 5 years ago

Unfortunately, the new test output summary introduced by #46 has timing information (e.g., finished after XXX ms), meaning that a simple text-based diff (as done by dotests.sh) isn't sufficient. I'm not seeing a built-in option to disable this so we may need to strip it out or just remove the entire line using sed in test.sh like we're already doing for container and Vintage output. Regardless of what we do, all of the expected outputs will need to be regenerated.

lam2mo commented 5 years ago

I suppose another alternative would be to implement a custom diff test but it seems easier just to strip the output. Because we're not terribly concerned with performance with Less-Java I don't think the runtime info is really useful anyway.

laurelmay commented 5 years ago

Oh yeah, I neglected to remove that line as part of #46. Honestly, I didn't even notice it. Would adding -e 'Test run finished after' to the grep -v line be a satisfactory fix?

Are there any other lines that should be removed/stripped?

lam2mo commented 5 years ago

Yeah, I think that'd probably be sufficient to close this issue, but maybe @Zamua has a different thought.

A larger issue is that the test names don't really mean anything right now (test0, test1, etc.) but that's probably something that can't be addressed until #12 is implemented.

laurelmay commented 5 years ago

I'll get a PR together for that.

As far as test names (and maybe this would be more appropriate to mention in another issue), there is the @DisplayName annotation for a @Test, which could potentially be used to set the name to the expression after the test keyword? JUnit's ConsoleLauncher would then print that instead of test0, etc. This wouldn't totally solve #12 but could be helpful for making the output look friendlier.

lam2mo commented 5 years ago

Yeah I was thinking something like that. Again, a lot of this will depend on whether we decide to stick with using the JUnit output directly or do our own wrapper around it (e.g., a permanent fix for #12). If the latter, then none of these JUnit output details really matters much in the long term because we'll be able to decide exactly what sort of output we want.