scalameta / munit

Scala testing library with actionable errors and extensible APIs
https://scalameta.org/munit
Apache License 2.0
433 stars 90 forks source link

Wrong line numbers using assertEquals when helper method declared in seperate file #855

Open AugustNagro opened 6 days ago

AugustNagro commented 6 days ago

When using a MUnit test inside a helper function that lives in a different file than the FunSuite subtype, the error output is very unhelpful.

I've made a minimal reproducer here: https://github.com/AugustNagro/munit-helper-bad-trace

MySuite:
+ this works 0.005s
  ==> X MySuite.this fails  0.021s munit.ComparisonFailException: /home/augustnagro/prog/test-projects/munit-helper-bad-trace/src/test/scala/MySuite.scala:4
  3:class MySuite extends munit.FunSuite {
  4:  mySharedTest(this, "Postgres")
  5:}
  values are not the same
  => Obtained
  abc
  => Diff (- obtained, + expected)
  -abc
  +abd
  at munit.Assertions.failComparison(Assertions.scala:278)
  [error] Failed: Total 2, Failed 1, Errors 0, Passed 1
  [error] Failed tests:
  [error]   MySuite
  [error] (Test / test) sbt.TestsFailedException: Tests unsuccessful
  [error] Total time: 1 s, completed Nov 26, 2024, 12:09:19 AM

If assertEquals printed the stacktrace (like assert() and as suggested in https://github.com/scalameta/munit/issues/782 ), that would make things much better.

tgodzik commented 5 days ago

I've been wondering whether hiding the stack trace is that useful. I think it was done to avoid having traces from munit, but this behaviour is flaky as seen here.

AugustNagro commented 5 days ago

Yeah I also understand that I'm trying to do something against the grain here. If it were configurable at least that would be great.

tgodzik commented 3 days ago

Ok, it looks like it would never print the stack trace and that was on purpose, fixing it in https://github.com/scalameta/munit/pull/856

Should help with your case too or at least shed more light on it