Integration test fails even in clean checkout

SethTisue commented 7 months ago

sbt:scala-yaml> integration/test
org.virtuslab.yaml.K8sConfigSpec:
==> X org.virtuslab.yaml.K8sConfigSpec.initializationError  0.001s java.lang.NullPointerException: null
    at org.virtuslab.yaml.YamlRunnerSpec.<init>(YamlRunnerSpec.scala:14)
    at org.virtuslab.yaml.K8sConfigSpec.<init>(K8sConfigSpec.scala:3)
Error in test - /Users/tisue/scala-yaml/integration-tests/target/scala-2.13/test-classes/yaml/test-suite/9DXL.tml
Events differ - /Users/tisue/scala-yaml/integration-tests/target/scala-2.13/test-classes/yaml/test-suite/RXY3.tml
Error in test - /Users/tisue/scala-yaml/integration-tests/target/scala-2.13/test-classes/yaml/test-suite/M5DY.tml
Error in test - /Users/tisue/scala-yaml/integration-tests/target/scala-2.13/test-classes/yaml/test-suite/SF5V.tml
Error in test - /Users/tisue/scala-yaml/integration-tests/target/scala-2.13/test-classes/yaml/test-suite/Q9WF.tml
...

this came up in the context of https://github.com/scala/community-build/pull/1713

SethTisue commented 7 months ago

I think this was missed until now because the GitHub Actions config in this repo only runs core/test and does not run integration/test cc @tgodzik

tgodzik commented 7 months ago

I will take a look, thanks!

lbialy commented 5 months ago

@kpodsiad hey Kamil, while you're available - what was the highest percent of green tests in this suite, ever? I've seen 44% green in your older PR but current main is around 40%.

kpodsiad commented 5 months ago

@lbialy sorry, I don't remember. But some heads up - they yaml that almost everyone on daily basis is only a small fraction of yaml spec, spec itself is enormous and contains things that no sane person would use like https://github.com/VirtusLab/scala-yaml/issues/24. It might be good idea to determine subset of currently available tests / cases which needs to supported and leave the rest as a fun fact, something maybe nice to have, but absolutely non necessary for 99% of usecases.

lbialy commented 3 months ago

We should track the percent covered by the suite I guess.

lbialy commented 2 months ago

Uh, main today is at 36.65% :/

lbialy commented 2 months ago

And k8s spec is at 62.60%

lbialy commented 2 months ago

no improvement after merge of all bugfixes 😞 😭

lbialy commented 2 months ago

I have added outputYaml checking in official yaml test suite and now it's 14.60% from 36.65% ...

VirtusLab / scala-yaml

Integration test fails even in clean checkout #274