hpi-swa / smalltalkCI

Framework for testing Smalltalk projects with GitHub Actions, GitLab CI, Travis CI, AppVeyor, and others.
MIT License
94 stars 68 forks source link

Use platform aware class in tests #619

Closed theseion closed 8 months ago

theseion commented 8 months ago

Fixes #618

fniephaus commented 8 months ago

Thanks for the PR, Max. I had already adjusted the tests via https://github.com/hpi-swa/smalltalkCI/commit/3f277b9f730c0d38f3be3aebd9ecd1957232659e, but cherry-pick the relevant bits from this PR via https://github.com/hpi-swa/smalltalkCI/commit/e03bd965a8561714b5bbb4e4a23dc826f1a77cf5.

Nonetheless, the Pharo 12 builds still fail:

#########################
# 9 tests did not pass: #
#########################

SCIPharoMonticelloLoadSpecTest
 ✗ #testAddLoadedClassesFrom (4ms)
TestFailure: Denial failed
SCIPharoMonticelloLoadSpecTest(TestAsserter)>>assert:description:resumable:
SCIPharoMonticelloLoadSpecTest(TestAsserter)>>deny:description:resumable:
SCIPharoMonticelloLoadSpecTest(TestAsserter)>>deny:description:
SCIPharoMonticelloLoadSpecTest(TestAsserter)>>deny:
SCIPharoMonticelloLoadSpecTest>>testAddLoadedClassesFrom ...deny: (self spec loadedClasses isEmpty)
SCIPharoMonticelloLoadSpecTest(TestCase)>>performTest

SmalltalkCISpecTest
 ✗ #testClassesToTest (9ms)
TestFailure: Assertion failed
SmalltalkCISpecTest(TestAsserter)>>assert:description:resumable:
SmalltalkCISpecTest(TestAsserter)>>assert:description:
SmalltalkCISpecTest(Object)>>assert:
SmalltalkCISpecTest>>testClassesToTest ...assert: (classes includes: SmalltalkCITest)
SmalltalkCISpecTest(TestCase)>>performTest

SmalltalkCITest
 ✗ #testClassesForCategories (6ms)
TestFailure: Assertion failed
SmalltalkCITest(TestAsserter)>>assert:description:resumable:
SmalltalkCITest(TestAsserter)>>assert:description:
SmalltalkCITest(Object)>>assert:
SmalltalkCITest>>testClassesForCategories ...assert: (classes includes: SmalltalkCI)
SmalltalkCITest(TestCase)>>performTest
 ✗ #testClassesFrom (6ms)
TestFailure: Assertion failed
SmalltalkCITest(TestAsserter)>>assert:description:resumable:
SmalltalkCITest(TestAsserter)>>assert:description:
SmalltalkCITest(Object)>>assert:
SmalltalkCITest>>testClassesFrom ...assert: (classes includes: SmalltalkCI)
SmalltalkCITest(TestCase)>>performTest
 ✗ #testClassesInCategory (0ms)
TestFailure: Assertion failed
SmalltalkCITest(TestAsserter)>>assert:description:resumable:
SmalltalkCITest(TestAsserter)>>assert:description:
SmalltalkCITest(Object)>>assert:
SmalltalkCITest>>testClassesInCategory ...assert: (classes includes: SmalltalkCI)
SmalltalkCITest(TestCase)>>performTest
 ✗ #testClassesWithCategoryNames (2ms)
TestFailure: Assertion failed
SmalltalkCITest(TestAsserter)>>assert:description:resumable:
SmalltalkCITest(TestAsserter)>>assert:description:
SmalltalkCITest(Object)>>assert:
SmalltalkCITest>>testClassesWithCategoryNames ...assert: (classes includes: SmalltalkCI)
SmalltalkCITest(TestCase)>>performTest

SCITestReporterXMLTest
 ✗ #testReportSuccess (22ms)
TestFailure: Assertion failed
SCITestReporterXMLTest(TestAsserter)>>assert:description:resumable:
SCITestReporterXMLTest(TestAsserter)>>assert:description:
SCITestReporterXMLTest(Object)>>assert:
SCITestReporterXMLTest>>testReportSuccess ...assert: index > 0
SCITestReporterXMLTest(TestCase)>>performTest

SCITestRunnerTest
 ✗ #testRunClasses (11ms)
TestFailure: Assertion failed
SCITestRunnerTest(TestAsserter)>>assert:description:resumable:
SCITestRunnerTest(TestAsserter)>>assert:description:
SCITestRunnerTest(Object)>>assert:
SCITestRunnerTest>>testRunClasses ...assert: runner isSuccessful
SCITestRunnerTest(TestCase)>>performTest
 ✗ #testRunSpecNamed (12ms)
TestFailure: Assertion failed
SCITestRunnerTest(TestAsserter)>>assert:description:resumable:
SCITestRunnerTest(TestAsserter)>>assert:description:
SCITestRunnerTest(Object)>>assert:
SCITestRunnerTest>>testRunSpecNamed ...assert: runner isSuccessful
SCITestRunnerTest(TestCase)>>performTest

https://github.com/hpi-swa/smalltalkCI/actions/runs/7159267912/job/19492329016?pr=619#step:4:996

theseion commented 8 months ago

That's because those images are not the alpha images. The URL for the Pharo 12 images ends in 120. We should probably remove Pharo64-12 from the list of images, or at least from the test matrix. That image hasn't been updated in a while.

fniephaus commented 8 months ago

We should probably remove Pharo64-12 from the list of images, or at least from the test matrix.

I'm not sure I understand. Is Pharo 12 considered stable or not? Nonetheless, the tests are failing in a weird way in that image. Are you sure the code you recently added to find classes in categories is doing what it should be?

theseion commented 8 months ago

No, Pharo 12 isn't stable yet. Alpha and 12 are the same thing, but there is already a 120 image, which isn't updated by the nightly build. That means that Pharo64-12 is not the same thing as Pharo64-alpha.

Yes, the code should be fine. It works for the alpha build.