search

opalj / opal

https://www.opal-project.de
Other
51 stars 27 forks source link

Adapt FPCF IT Tests to new Reachable Methods #238

Closed johannesduesing closed 1 week ago

johannesduesing commented 2 weeks ago

PRs #237 and #229 lead to callgraphs capturing more methods, which broke the FPCF integration tests. I checked this locally and got a total of 20 failures for the FPCFAnalysesIntegrationTest. I went through all of them and confirmed that they are due to new methods being reachable, not old method not being found. I regenerated expected values for these tests and confirmed that the integration tests are working. I'd like to re-run them here on GitHub before merging because i had some resource problems running them locally for two large targets ... 😅

johannesduesing commented 2 weeks ago

/it:test

errt commented 2 weeks ago

Something seems to be going on here. You submitted 20 updated files, but the first it:test with the problem had only 14 failing tests. The other it:test runs here then, however, had 22. Either there is something non-deterministic or other factors influenced how many of the tests fail.

errt commented 2 weeks ago

Also, it looks like your /it:test has been skipped, but I don't know why.

errt commented 2 weeks ago

/it:test

johannesduesing commented 2 weeks ago

I think both our improved handling for Thread.start and the bugfix for the static initializers lead to new reachable methods - on distinct test cases. That's why it was 14 failing tests after one and then 22 total after merging the other PR.

The 22 vs 20 difference is easily explained - i couldn't run the IT test locally for two JAR files, the scala runtime and OPAl with dependencies due to resource issues 😄 I'm pretty sure they are also failing and need new actual values, i was hoping to confirm that with the runners executing the IT tests. I'll move to our own servers to regenerate them and update the PR!

johannesduesing commented 2 weeks ago

/it:test

johannesduesing commented 1 week ago

/it:test