search

Daniel-Diaz / HaTeX

The Haskell LaTeX library.
BSD 3-Clause "New" or "Revised" License
199 stars 46 forks source link

HaTeX-3.21.0.0 test suite failure #144

Closed peti closed 4 years ago

peti commented 4 years ago

Citing from https://hydra.nixos.org/build/107455907:

Test suite hatex-test: RUNNING...
HaTeX
  LaTeX
    LaTeX mempty:        OK
      +++ OK, passed 100 tests.
    LaTeX mappend:       OK (0.02s)
      +++ OK, passed 100 tests.
  Parser
    render . parse = id: FAIL
      *** Failed! Falsified (after 32 tests):
      TeXMath Dollar (TeXSeq (TeXRaw "gvi>flJua/FCd!dbZxP") (TeXMath Dollar (TeXCommS "aTvBLTDYif")))
      Use --quickcheck-replay=67509 to reproduce.

1 out of 3 tests failed (0.03s)
Test suite hatex-test: FAIL
leftaroundabout commented 4 years ago

Of course this is the same rough issue as https://github.com/Daniel-Diaz/HaTeX/issues/115.

I tend to say we should just remove that pesky test. Randomly-generated LaTeX is just too crazy. I'm all for QuickCheck, but in this case the test cases have no semblance to any real LaTeX code. It would be a lot of work to get the parser working reliably with the auto-generated cases, but even then I doubt it would really improve the real-world reliability.

Daniel-Diaz commented 4 years ago

The error here is that parsing

$gvi>flJua/FCd!dbZxP$\aTvBLTDYif$$

gives you this error:

Left "parseLaTeX input" (line 1, column 35):
unexpected end of input
expecting "$$", text, inline math ($), comment, text2, environment or command
text: Empty input.

@leftaroundabout Yes, I see your point. And we already have the other parsing test in which several hand-written LaTeX files are parsed.

Maybe the problem in this case is a $-math expression inside another $-math expression, which is something that shouldn't happen in valid LaTeX code (unless switching to text mode inside the math expression). So the Arbitrary instance is generating invalid LaTeX code and the parser is complaining about it. Maybe if the LaTeX type itself didn't allow for this kind of mistake, this wouldn't be a problem. But that's an entirely different topic.

In any case, something should be done about this test, because it keeps failing randomly. Maybe I'll just comment it out.

bidigo commented 4 years ago

We've been hit by this problem as well, our CI pipeline fails once a week or so. Would be nice if something could be done about it.

leftaroundabout commented 4 years ago

@bidigo well, this should be ok since 9cb9d3d8b817364def0b75a0aeb8fd3c91c23c33, but I guess this is no help for you since it's not yet on Hackage?

@Daniel-Diaz perhaps we should just make a new release.

bidigo commented 4 years ago

Yes, a new release would be perfect. We are building with Nix so we could alternatively add a source dependency to a version that includes the commit...

Daniel-Diaz commented 4 years ago

@bidigo @leftaroundabout

I just released HaTeX-3.22.1.0, including the mentioned commit.

bidigo commented 4 years ago

@Daniel-Diaz Thank you very much!