datavane / tis

Support agile DataOps Based on Flink, DataX and Flink-CDC, Chunjun with Web-UI
https://tis.pub
Apache License 2.0
1.04k stars 221 forks source link

Hive Haddop 增加Cloudera的版本支持 #323

Closed baisui1981 closed 3 months ago

baisui1981 commented 6 months ago

使用者生产环境中常使用cloudera作为Haddop适配版本,所以TIS中需要增量针对cloudera的版本支持

<properties>
<hadoop3-version>3.0.0-cdh6.3.2</hadoop3-version>
<hive.version>2.1.1-cdh6.3.2</hive.version>
</properties>
<repositories>
        <repository>
            <id>cloudera</id>
            <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
        </repository>
    </repositories>
baisui1981 commented 3 months ago

添加了id 为cdh的profile

<profile>

            <id>cdh</id>
            <properties>
                <spark2.version>3.2.1</spark2.version>
                <hadoop-version>3.0.0-cdh6.3.2</hadoop-version>
                <spark.dist.dir.name>spark-${spark2.version}-bin-hadoop3.2</spark.dist.dir.name>
                <!--                <hudi.version>0.12.2</hudi.version>-->
                <!--                <hive.version>3.1.3</hive.version>-->

                <hudi.version>0.14.1</hudi.version>
                <hive.version>2.1.1-cdh6.3.2</hive.version>

                <scala.binary.version>2.12</scala.binary.version>
                <scala.version>2.12.9</scala.version>
                <appname>all</appname>
            </properties>
            <repositories>
                <repository>
                    <id>cloudera</id>
                    <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
                </repository>
            </repositories>
            <modules>

            </modules>
        </profile>