search

apache / carbondata

High performance data store solution
carbondata.apache.org
Apache License 2.0
1.43k stars 704 forks source link

[CARBONDATA-4345] update/delete operations failed when other format segments deleted from carbon table #4285

Closed maheshrajus closed 2 years ago

maheshrajus commented 2 years ago

Why is this PR needed?

Update/delete operations failed when other format segments deleted from carbon table Steps to reproduce: 1) create carbon table and load the data 2) create parquet/orc tables and load the data 3) add parquet/orc format segments in carbon table by alter add segment command 4) perform update/delete operations in carbon table and they will fail as table contains mixed format segments. This is expected behaviour only. 4) delete the other format segments which is added in step3 5) try to perform update/delete operation in carbon data. They should not fail

For update/delete operations we are checking if other format segments present in table path. If found then carbon data throwing exception by saying mixed format segments exists even though the other format segments deleted from table.

What changes were proposed in this PR?

When we are checking other format segment present in carbon table then it should check only for SUCCESS/PARTIAL_SUCCESS segments.

Does this PR introduce any user interface change?

CarbonDataQA2 commented 2 years ago

Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6379/

CarbonDataQA2 commented 2 years ago

Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4634/

CarbonDataQA2 commented 2 years ago

Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/765/

Indhumathi27 commented 2 years ago

@maheshrajus please rebase

maheshrajus commented 2 years ago

rebase is done

CarbonDataQA2 commented 2 years ago

Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4635/

CarbonDataQA2 commented 2 years ago

Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6380/

CarbonDataQA2 commented 2 years ago

Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/766/

Indhumathi27 commented 2 years ago

LGTM

Indhumathi27 commented 2 years ago

retest this please

CarbonDataQA2 commented 2 years ago

Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4639/

CarbonDataQA2 commented 2 years ago

Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6384/

CarbonDataQA2 commented 2 years ago

Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/770/

maheshrajus commented 2 years ago

restest this please

CarbonDataQA2 commented 2 years ago

Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4640/

CarbonDataQA2 commented 2 years ago

Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6385/

CarbonDataQA2 commented 2 years ago

Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/771/