search

prestodb / presto

The official home of the Presto distributed SQL query engine for big data
http://prestodb.io
Apache License 2.0
15.92k stars 5.33k forks source link

Flaky Tests in TestQuickStatsProvider #22407

Open imjalpreet opened 5 months ago

imjalpreet commented 5 months ago 
Error:  Tests run: 2955, Failures: 2, Errors: 0, Skipped: 90, Time elapsed: 2,796.671 s <<< FAILURE! - in TestSuite
Error:  com.facebook.presto.hive.statistics.TestQuickStatsProvider.quickStatsBuildTimeIsBounded  Time elapsed: 2.644 s  <<< FAILURE!
java.lang.RuntimeException: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
    at com.facebook.presto.hive.statistics.TestQuickStatsProvider.lambda$quickStatsBuildTimeIsBounded$13(TestQuickStatsProvider.java:410)
    at com.facebook.presto.hive.RetryDriver.run(RetryDriver.java:139)
    at com.facebook.presto.hive.statistics.TestQuickStatsProvider.quickStatsBuildTimeIsBounded(TestQuickStatsProvider.java:399)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:135)
    at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:673)
    at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:220)
    at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50)
    at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:945)
    at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:193)
    at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
    at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)
    Suppressed: java.lang.RuntimeException: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
        ... 18 more
    Caused by: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
        at org.testng.Assert.fail(Assert.java:110)
        at org.testng.Assert.failNotEquals(Assert.java:1413)
        at org.testng.Assert.assertEqualsImpl(Assert.java:149)
        at org.testng.Assert.assertEquals(Assert.java:131)
        at org.testng.Assert.assertEquals(Assert.java:643)
        at com.facebook.presto.hive.statistics.TestQuickStatsProvider.lambda$quickStatsBuildTimeIsBounded$13(TestQuickStatsProvider.java:405)
        ... 17 more
    Suppressed: java.lang.RuntimeException: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
        ... 18 more
    Caused by: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
        at org.testng.Assert.fail(Assert.java:110)
        at org.testng.Assert.failNotEquals(Assert.java:1413)
        at org.testng.Assert.assertEqualsImpl(Assert.java:149)
        at org.testng.Assert.assertEquals(Assert.java:131)
        at org.testng.Assert.assertEquals(Assert.java:643)
        at com.facebook.presto.hive.statistics.TestQuickStatsProvider.lambda$quickStatsBuildTimeIsBounded$13(TestQuickStatsProvider.java:405)
        ... 17 more
    Suppressed: java.lang.RuntimeException: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
        ... 18 more
    Caused by: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
        at org.testng.Assert.fail(Assert.java:110)
        at org.testng.Assert.failNotEquals(Assert.java:1413)
        at org.testng.Assert.assertEqualsImpl(Assert.java:149)
        at org.testng.Assert.assertEquals(Assert.java:131)
        at org.testng.Assert.assertEquals(Assert.java:643)
        at com.facebook.presto.hive.statistics.TestQuickStatsProvider.lambda$quickStatsBuildTimeIsBounded$13(TestQuickStatsProvider.java:405)
        ... 17 more
    Suppressed: java.lang.RuntimeException: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
        ... 18 more
    Caused by: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
        at org.testng.Assert.fail(Assert.java:110)
        at org.testng.Assert.failNotEquals(Assert.java:1413)
        at org.testng.Assert.assertEqualsImpl(Assert.java:149)
        at org.testng.Assert.assertEquals(Assert.java:131)
        at org.testng.Assert.assertEquals(Assert.java:643)
        at com.facebook.presto.hive.statistics.TestQuickStatsProvider.lambda$quickStatsBuildTimeIsBounded$13(TestQuickStatsProvider.java:405)
        ... 17 more
    Suppressed: java.lang.RuntimeException: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
        ... 18 more
    Caused by: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
        at org.testng.Assert.fail(Assert.java:110)
        at org.testng.Assert.failNotEquals(Assert.java:1413)
        at org.testng.Assert.assertEqualsImpl(Assert.java:149)
        at org.testng.Assert.assertEquals(Assert.java:131)
        at org.testng.Assert.assertEquals(Assert.java:643)
        at com.facebook.presto.hive.statistics.TestQuickStatsProvider.lambda$quickStatsBuildTimeIsBounded$13(TestQuickStatsProvider.java:405)
        ... 17 more
    Suppressed: java.lang.RuntimeException: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
        ... 18 more
    Caused by: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
        at org.testng.Assert.fail(Assert.java:110)
        at org.testng.Assert.failNotEquals(Assert.java:1413)
        at org.testng.Assert.assertEqualsImpl(Assert.java:149)
        at org.testng.Assert.assertEquals(Assert.java:131)
        at org.testng.Assert.assertEquals(Assert.java:643)
        at com.facebook.presto.hive.statistics.TestQuickStatsProvider.lambda$quickStatsBuildTimeIsBounded$13(TestQuickStatsProvider.java:405)
        ... 17 more
    Suppressed: java.lang.RuntimeException: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
        ... 18 more
    Caused by: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
        at org.testng.Assert.fail(Assert.java:110)
        at org.testng.Assert.failNotEquals(Assert.java:1413)
        at org.testng.Assert.assertEqualsImpl(Assert.java:149)
        at org.testng.Assert.assertEquals(Assert.java:131)
        at org.testng.Assert.assertEquals(Assert.java:643)
        at com.facebook.presto.hive.statistics.TestQuickStatsProvider.lambda$quickStatsBuildTimeIsBounded$13(TestQuickStatsProvider.java:406)
        ... 17 more
    Suppressed: java.lang.RuntimeException: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
        ... 18 more
    Caused by: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
        at org.testng.Assert.fail(Assert.java:110)
        at org.testng.Assert.failNotEquals(Assert.java:1413)
        at org.testng.Assert.assertEqualsImpl(Assert.java:149)
        at org.testng.Assert.assertEquals(Assert.java:131)
        at org.testng.Assert.assertEquals(Assert.java:643)
        at com.facebook.presto.hive.statistics.TestQuickStatsProvider.lambda$quickStatsBuildTimeIsBounded$13(TestQuickStatsProvider.java:406)
        ... 17 more
Caused by: java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
    at org.testng.Assert.fail(Assert.java:110)
    at org.testng.Assert.failNotEquals(Assert.java:1413)
    at org.testng.Assert.assertEqualsImpl(Assert.java:149)
    at org.testng.Assert.assertEquals(Assert.java:131)
    at org.testng.Assert.assertEquals(Assert.java:643)
    at com.facebook.presto.hive.statistics.TestQuickStatsProvider.lambda$quickStatsBuildTimeIsBounded$13(TestQuickStatsProvider.java:405)
    ... 17 more

Error:  com.facebook.presto.hive.statistics.TestQuickStatsProvider.testConcurrentFetchForSamePartition  Time elapsed: 0.152 s  <<< FAILURE!
java.lang.AssertionError: For [partition1] partitionExpected one of the partitions stats to be empty. Actual partitionStatistics1 [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}], partitionStatistics2 [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}]
    at org.testng.Assert.fail(Assert.java:110)
    at com.facebook.presto.hive.statistics.TestQuickStatsProvider.testConcurrentFetchForSamePartition(TestQuickStatsProvider.java:296)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:135)
    at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:673)
    at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:220)
    at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50)
    at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:945)
    at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:193)
    at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
    at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)

[INFO] 
[INFO] Results:
[INFO] 
Error:  Failures: 
Error:    TestQuickStatsProvider.quickStatsBuildTimeIsBounded:399->lambda$quickStatsBuildTimeIsBounded$13:410 Runtime java.lang.AssertionError: expected [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}] but found [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong.empty, rowCount=OptionalLong.empty, inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={}}]
Error:  com.facebook.presto.hive.statistics.TestQuickStatsProvider.testConcurrentFetchForSamePartition
Error:    Run 1: TestQuickStatsProvider.testConcurrentFetchForSamePartition:296 For [partition1] partitionExpected one of the partitions stats to be empty. Actual partitionStatistics1 [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}], partitionStatistics2 [PartitionStatistics{basicStatistics=HiveBasicStatistics{fileCount=OptionalLong[42], rowCount=OptionalLong[4242], inMemoryDataSizeInBytes=OptionalLong.empty, onDiskDataSizeInBytes=OptionalLong.empty}, columnStatistics={column=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[-2147483648], max=OptionalLong[2147483647]}], doubleStatistics=Optional.empty, decimalStatistics=Optional.empty, dateStatistics=Optional.empty, booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty, totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0], distinctValuesCount=OptionalLong.empty}}}]

https://github.com/prestodb/presto/actions/runs/8529916842/job/23366694814?pr=22288#step:7:11881

ajaygeorge commented 5 months ago

quickStatsBuildTimeIsBounded started failing for me consistently on my mac. It fails on and off on in github ci. @aaneja can you please take a look.

aaneja commented 5 months ago

@ajaygeorge I haven't been able to repro these locally (Linux machine) so stomping out the flakiness has been tricky. I am reworking these tests to avoid any timing dependencies. In the meanwhile, please feel free to disable the tests that you're observing as flaky

aaneja commented 5 months ago

Fix : https://github.com/prestodb/presto/pull/22472