search

apache / incubator-gluten

Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.
https://gluten.apache.org/
Apache License 2.0
1.18k stars 430 forks source link

[CORE] Migrate some Spark Unit Test Cases into Gluten #374

Open zzcclp opened 2 years ago

zzcclp commented 2 years ago

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

Migrate some Spark Unit Test Cases into Gluten, please refer to the detail info

Expression Test Cases:

First Phase:

Second Phase:

DataFrameXXXSuite Test Cases:

First Phase:

Second Phase:

Third Phase:

Fourth Phase:

Folders:

· ansi array.sql repeat(Not need to add support list, because it will recursively search dirs) date.sql repeat datetime-special.sql repeat decimalArithmeticOperations.sql repeat higher-order-functions.sql repeat interval.sql repeat literals.sql repeat map.sql repeat parse-schema-string.sql repeat string-functions.sql repeat timestamp.sql repeat · postgreSQL aggregates_part1.sql JiaKe aggregates_part2.sql JiaKe aggregates_part3.sql JiaKe aggregates_part4.sql JiaKe boolean.sql JiaKe case.sql JiaKe comments.sql JiaKe create_view.sql JiaKe date.sql JiaKe float4.sql JiaKe float8.sql JiaKe groupingsets.sql JiaKe insert.sql JiaKe int2.sql JiaKe int4.sql JiaKe int8.sql JiaKe interval.sql JiaKe join.sql JiaKe limit.sql JiaKe numeric.sql JiaKe select.sql JiaKe select_distinct.sql JiaKe select_having.sql JiaKe select_implicit.sql JiaKe strings.sql JiaKe text.sql JiaKe timestamp.sql JiaKe union.sql JiaKe window_part1.sql JiaKe window_part2.sql JiaKe window_part3.sql JiaKe window_part4.sql JiaKe with.sql JiaKe · subquery/exists-subquery exists-aggregate.sql hongze exists-basic.sql Yohahaha exists-cte.sql hongze exists-having.sql JiayiLiu exists-joins-and-set-ops.sql hongze exists-orderby-limit.sql hongze exists-within-and-or.sql hongze · subquery/in-subquery hongze in-basic.sql Kerwin in-group-by.sql hongze in-having.sql hongze in-joins.sql hongze in-limit.sql hongze in-multiple-columns.sql hongze in-order-by.sql hongze in-set-operations.sql hongze in-with-cte.sql hongze nested-not-in.sql hongze not-in-group-by.sql hongze not-in-joins.sql hongze not-in-unit-tests-multi-column-literal.sql hongze not-in-unit-tests-multi-column.sql hongze not-in-unit-tests-single-column-literal.sql hongze not-in-unit-tests-single-column.sql hongze simple-in.sql hongze · subquery/negative-cases invalid-correlation.sql hongze subq-input-typecheck.sql hongze · subquery/scalar-subquery scalar-subquery-predicate.sql hongze scalar-subquery-select.sql hongze · subquery/ subquery-in-from.sql hongze

· timestampNTZ datetime-special.sql Kerwin timestamp-ansi.sql Kerwin timestamp.sql Kerwin

· typeCoercion/native/ Kerwin arrayJoin.sql Kerwin binaryComparison.sql Kerwin booleanEquality.sql Kerwin caseWhenCoercion.sql Kerwin concat.sql Kerwin dateTimeOperations.sql Kerwin decimalPrecision.sql Kerwin division.sql Kerwin elt.sql ifCoercion.sql Kerwin implicitTypeCasts.sql Kerwin inConversion.sql Kerwin mapZipWith.sql Kerwin mapconcat.sql Kerwin promoteStrings.sql Kerwin stringCastAndExpressions.sql Kerwin widenSetOperationTypes.sql Kerwin windowFrameCoercion.sql Kerwin

Fifth Phase

GlutenDatasetPrimitiveSuite

GlutenDatasetSuite

GlutenJsonFunctionsSuite

GlutenFileBasedDataSourceSuite

GlutenEnsureRequirementsSuite

GlutenCoalesceShufflePartitionsSuite

GlutenDSV2CharVarcharTestSuite

GlutenBinaryFileFormatSuite

GlutenJsonV1Suite

GlutenOrcFilterSuite

GlutenOrcV1FilterSuite

GlutenParquetInteroperabilitySuite

GlutenParquetV2PartitionDiscoverySuite

GlutenParquetV1QuerySuite

GlutenParquetV2QuerySuite

GlutenParquetRebaseDatetimeV1Suite

GlutenParquetRebaseDatetimeV2Suite

GlutenParquetV1SchemaPruningSuite

GlutenParquetV2SchemaPruningSuite

GlutenParquetSchemaInferenceSuite

GlutenParquetSchemaSuite

GlutenFileFormatWriterSuite

GlutenParquetCodecSuite

GlutenOrcCodecSuite

GlutenFileSourceStrategySuite

GlutenOrcReadSchemaSuite

GlutenVectorizedOrcReadSchemaSuite

GlutenMergedOrcReadSchemaSuite

GlutenDataSourceV2SQLSuite

GlutenWriteDistributionAndOrderingSuite

GlutenBucketedReadWithoutHiveSupportSuite

GlutenInsertSuite

GlutenDataSourceV2Suite

GlutenCreateTableAsSelectSuite

GlutenCachedTableSuite

GlutenCsvFunctionsSuite

GlutenUnwrapCastInComparisonEndToEndSuite Jiao

Passed ones: GlutenFileSourceCharVarcharTestSuite

GlutenFileScanSuite

GlutenNestedDataSourceV1Suite

GlutenNestedDataSourceV2Suite

GlutenDatasetAggregatorSuite

GlutenDatasetOptimizationSuite

GlutenFileSourceSQLInsertTestSuite

GlutenDSV2SQLInsertTestSuite

GlutenXPathFunctionsSuite

GlutenProductAggSuite

GlutenReplaceNullWithFalseInPredicateEndToEndSuite

GlutenCSVv1Suite

GlutenCSVv2Suite

GlutenCSVLegacyTimeParserSuite

GlutenJsonV2Suite

GlutenJsonLegacyTimeParserSuite

GlutenTextV1Suite

GlutenTextV2Suite

GlutenOrcColumnarBatchReaderSuite

GlutenOrcPartitionDiscoverySuite

GlutenOrcV1PartitionDiscoverySuite

GlutenOrcV1QuerySuite

GlutenOrcSourceSuite

GlutenOrcV1SchemaPruningSuite

GlutenOrcV2SchemaPruningSuite

GlutenParquetColumnIndexSuite

GlutenParquetCompressionCodecPrecedenceSuite

GlutenParquetEncodingSuite

GlutenParquetFileFormatV1Suite

GlutenParquetFileFormatV2Suite

GlutenParquetV1FilterSuite

GlutenParquetV2FilterSuite

GlutenParquetIOSuite

GlutenParquetV1PartitionDiscoverySuite

GlutenParquetProtobufCompatibilitySuite

GlutenParquetThriftCompatibilitySuite

GlutenFileTableSuite

GlutenBucketingUtilsSuite

GlutenDataSourceStrategySuite

GlutenDataSourceSuite

GlutenFileIndexSuite

GlutenHadoopFileLinesReaderSuite

GlutenPathFilterStrategySuite

GlutenPathFilterSuite

GlutenPruneFileSourcePartitionsSuite

GlutenCSVReadSchemaSuite

GlutenHeaderCSVReadSchemaSuite

GlutenJsonReadSchemaSuite

GlutenParquetReadSchemaSuite

GlutenVectorizedParquetReadSchemaSuite

GlutenMergedParquetReadSchemaSuite

GlutenDataSourceV2DataFrameSessionCatalogSuite

GlutenDataSourceV2DataFrameSuite

GlutenDataSourceV2FunctionSuite

GlutenDataSourceV2SQLSessionCatalogSuite

GlutenFileDataSourceV2FallBackSuite

GlutenLocalScanSuite

GlutenSupportsCatalogOptionsSuite

GlutenTableCapabilityCheckSuite

GlutenBucketedWriteWithoutHiveSupportSuite

GlutenDDLSourceLoadSuite

GlutenDisableUnnecessaryBucketedScanWithoutHiveSupportSuite

GlutenDisableUnnecessaryBucketedScanWithoutHiveSupportSuiteAE

GlutenExternalCommandRunnerSuite

GlutenFilteredScanSuite

GlutenFiltersSuite

GlutenPartitionedWriteSuite

GlutenPathOptionSuite

GlutenPrunedScanSuite

GlutenResolvedDataSourceSuite

GlutenSaveLoadSuite

GlutenTableScanSuite

GlutenApproxCountDistinctForIntervalsQuerySuite

GlutenConfigBehaviorSuite

GlutenCountMinSketchAggQuerySuite

GlutenCTEHintSuite

GlutenCTEInlineSuiteAEOff

GlutenCTEInlineSuiteAEOn

GlutenDataFrameHintSuite

GlutenDataFrameWriterV2Suite

GlutenDatasetCacheSuite

GlutenExpressionsSchemaSuite

GlutenExtraStrategiesSuite

GlutenMiscFunctionsSuite

GlutenProcessingTimeSuite

GlutenScalaReflectionRelationSuite

GlutenSerializationSuite

GlutenTypedImperativeAggregateSuite

GlutenDatasetSerializerRegistratorSuite

GlutenDeprecatedAPISuite

GlutenMetadataCacheSuite

GlutenSimpleShowCreateTableSuite

GlutenStatisticsCollectionSuite

Describe the solution you'd like A clear and concise description of what you want to happen.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

Yohahaha commented 1 year ago

exists-basic.sql

I would like to pick this tests, please mark this, thanks.