[BUG] weird test failure in CopyCollectionTest

line-o commented 2 years ago

When running the tests as part of the maven package goal these failures were reported on my machine. update The same issue was now also observed happening in CI by @joewiz

[ERROR] Failures:
[ERROR]   CopyCollectionTest.copyToNonExistentAsDBA:107->checkAttributes:252 Created value is not correct
Expected: not <1647864985369L>
     but: was <1647864985369L>
[ERROR]   CopyCollectionTest.copyToNonExistentAsOther:127->checkAttributes:252 Created value is not correct
Expected: not <1647864985363L>
     but: was <1647864985363L>
[ERROR]   CopyCollectionTest.copyToNonExistentAsSelf:87->checkAttributes:252 Created value is not correct
Expected: not <1647864985373L>
     but: was <1647864985373L>
[INFO]
[ERROR] Tests run: 4815, Failures: 3, Errors: 0, Skipped: 106

The weird thing is that it seems that the expected values are actually returned based on the output.

Context (please always complete the following information):

OS: macOS 12.3
eXist-db version: 6.1.0-SNAPSHOT
Java Version Zulu 8 arm u322

Additional context

How is eXist-db installed? built from source
Any custom changes in e.g. conf.xml? none

duncdrum commented 2 years ago

I see a different error in exist/extensions/modules/expathrepo, but that one consistently. mvn clean test -fae:

<testcase name="" classname="org.exist.repo.PackageTriggerTest" time="2.032">
    <error type="java.io.IOException">
        <![CDATA[java.io.IOException: storeDocument received a null InputStream when trying to store a Binary Document at org.exist.repo.PackageTriggerTest.setup(PackageTriggerTest.java:81)]]>
    </error>
</testcase>

mvn -v 
Apache Maven 3.8.5 (3599d3414f046de2324203b78ddcf9b5e4388aa0)
Maven home: /opt/homebrew/Cellar/maven/3.8.5/libexec
Java version: 1.8.0_322, vendor: Azul Systems, Inc., runtime: /Library/Java/JavaVirtualMachines/zulu-8.jdk/Contents/Home/jre
Default locale: en_GB, platform encoding: UTF-8
OS name: "mac os x", version: "12.3", arch: "aarch64", family: "mac"

funnily : mvn clean verify -fae

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  10:05 min
[INFO] Finished at: 2022-03-22T11:42:00+01:00
[INFO] ------------------------------------------------------------------------

duncdrum commented 2 years ago

@line-o looks like joe had the same error on ci https://github.com/eXist-db/exist/pull/4269#issuecomment-1068375035

joewiz commented 2 years ago

(I didn't experience it myself - I just saw it in the GitHub Actions CI error logs.)

duncdrum commented 2 years ago

so repeated the same with mvn clean package -Dmac.signing=false -Ddocker=false -DskipTests -Ddependency-check.skip=true -P '!installer':

…
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:06 min
[INFO] Finished at: 2022-03-23T10:38:49+01:00
[INFO] ------------------------------------------------------------------------

the difference in the expathrepo test failing with mvn test while passing with mvn verify remains

to sum up,

building on M1 works in principle see #4229 at least I can't reproduce your errors
the flaky test appears on non ARM or M1 systems so something else is going on
a test shouldn't reliably fail when executed as part of test but pass when executed as verify either our maven config needs some tweaks, or the test needs to be executed with some dependency resolved in the verify phase only

line-o commented 2 years ago

Today, with support from @duncdrum I was able to have a full successful test run on M1. Since we saw the above issue also in CI, I guess this is not processor architecture related. I will change the title of this issue.

line-o commented 2 years ago

What I did in order to get to a running setup was:

git clean -dfX
mvn package

eXist-db / exist

[BUG] weird test failure in CopyCollectionTest #4278