Closed KiruthikaJanakiraman closed 11 months ago
This PR aims to fix the flaky test: org.apereo.portal.io.xml.IdentityImportExportTest.testStylesheetDescriptor40ImportExport similar to previously raised PRs: https://github.com/uPortal-Project/uPortal/pull/2718 and https://github.com/uPortal-Project/uPortal/pull/2719
I found and confirmed the flaky behaviour of the tests using an open-source research tool NonDex, which shuffles implementations of nondeterminism operations.
Problem The test case org.apereo.portal.io.xml.IdentityImportExportTest.testStylesheetDescriptor40ImportExport fails because the order of targetElements: channel and portlet in the exported xml varies upon nondeterministic shuffling.
targetElements
channel
portlet
https://github.com/uPortal-Project/uPortal/blob/7fe3d5d9078bd0be9b0442be37067bb399eab7fd/uPortal-webapp/src/test/resources/org/apereo/portal/io/xml/ssd/test_4-0.stylesheet-descriptor.xml#L49-L50
Solution This PR fixes the flakiness by sorting the targetElements in exportData before comparing importData and exportData XMLs.
Steps to reproduce The following command can be used to reproduce the assertion errors and verify the fix.
Integrate NonDex:
Add the following snippet to the top of the build.gradle in $PROJ_DIR:
plugins { id 'edu.illinois.nondex' version '2.1.1-1' }
Append to build.gradle in $PROJ_DIR:
apply plugin: 'edu.illinois.nondex'
Execute Test with Gradle:
./gradlew --info uPortal-webapp:test --tests org.apereo.portal.io.xml.IdentityImportExportTest.testStylesheetDescriptor40ImportExport
Run NonDex:
./gradlew --info uPortal-webapp:nondexTest --tests=org.apereo.portal.io.xml.IdentityImportExportTest.testStylesheetDescriptor40ImportExport --nondexRuns=50 -x autoLintGradle
Test Environment:
Java version "1.8.0_381" macOS Venture Version 13.4.1 (22F82)
Please let me know if you have any concerns or questions.
Got it. In that case, "under determined" makes more sense. Thanks!
Checklist
Description of change
This PR aims to fix the flaky test: org.apereo.portal.io.xml.IdentityImportExportTest.testStylesheetDescriptor40ImportExport similar to previously raised PRs: https://github.com/uPortal-Project/uPortal/pull/2718 and https://github.com/uPortal-Project/uPortal/pull/2719
I found and confirmed the flaky behaviour of the tests using an open-source research tool NonDex, which shuffles implementations of nondeterminism operations.
Problem The test case org.apereo.portal.io.xml.IdentityImportExportTest.testStylesheetDescriptor40ImportExport fails because the order of
targetElements
:channel
andportlet
in the exported xml varies upon nondeterministic shuffling.https://github.com/uPortal-Project/uPortal/blob/7fe3d5d9078bd0be9b0442be37067bb399eab7fd/uPortal-webapp/src/test/resources/org/apereo/portal/io/xml/ssd/test_4-0.stylesheet-descriptor.xml#L49-L50
Solution This PR fixes the flakiness by sorting the
targetElements
in exportData before comparing importData and exportData XMLs.Steps to reproduce The following command can be used to reproduce the assertion errors and verify the fix.
Integrate NonDex:
Add the following snippet to the top of the build.gradle in $PROJ_DIR:
Append to build.gradle in $PROJ_DIR:
Execute Test with Gradle:
Run NonDex:
Test Environment:
Please let me know if you have any concerns or questions.