delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
https://delta.io
Apache License 2.0
7.62k stars 1.71k forks source link

[BUG][Kernel] RoaringBitmapArray create/add test methods are broken #3881

Open YotillaAntoni opened 6 days ago

YotillaAntoni commented 6 days ago

Bug

Which Delta project/connector is this regarding?

Describe the problem

io.delta.kernel.internal.deletionvectors.RoaringBitmapArray create/add methods are broken. Fail with NPE

There are two issues:

  1. The add and extendsBitMaps methods expect the bitmaps field to be not null, at least should be an array of 0 length. But the instance created from the create method doesn't have the bitmaps field initialized.
  2. The extendsBitMaps methods, copies the current bitmaps to the new bitmaps, to at the next step overwrite them with empty maps. The method should instead set empty bitmaps from the old length to the new length (instead of from 0 to the old length)

Steps to reproduce

    @Test
    void testCreate() {
        RoaringBitmapArray underTest = RoaringBitmapArray.create(1L);
        Assertions.assertTrue(underTest.contains(1L));
    }

Observed results

NPE is thrown:

java.lang.NullPointerException: Cannot read the array length because "this.bitmaps" is null

    at io.delta.kernel.internal.deletionvectors.RoaringBitmapArray.add(RoaringBitmapArray.java:262)
    at io.delta.kernel.internal.deletionvectors.RoaringBitmapArray.create(RoaringBitmapArray.java:290)

Expected results

Test pases

Further details

Environment information

Willingness to contribute

The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?