apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
7.94k stars 1.8k forks source link

从hive抽取数据到oracle报错 #977

Closed xll-gits closed 2 years ago

xll-gits commented 2 years ago

Search before asking

What happened

启动程序时总是报不能创建hdfs目录,程序就直接中断

SeaTunnel Version

seatunnel-1.5.7

SeaTunnel Config

spark {
  spark.app.name = "T1"
  spark.yarn.queue = "root"
  spark.executor.instances = 2
  spark.executor.cores = 2
  spark.executor.memory = "2g"
  spark.sql.catalogImplementation = "hive"
}
input {
    hive {
        pre_sql = "select id,name from temp.t1"
        result_table_name = "t1"
    }
}
filter {}
output {
    jdbc {
        driver = "oracle.jdbc.driver.OracleDriver"
        url = "jdbc:oracle:thin:@${ip}:1521/orcl"
        table = "t1"
        user = "dw"
        password = "${pass}"
        save_mode = "append"
    }
}

Running Command

./start-seatunnel.sh -c ../config/batch.conf -m yarn -e client

Error Exception

2022-01-08 16:49:58 ERROR FileFactory:571 -  Failed to create directory path /user/hive/warehouse
2022-01-08 16:49:58 ERROR FileFactory:571 -  Failed to create directory path /user/hive/warehouse/default
                                                                                             **                                                 
                                                                                           ***#                                                 
                                        *                                                  ***#                                                 
                                       **                                                    *#                                                 
                                       **                                                    *#                                                 
                                      ***                                                    *#                                                 
       *******   *******    *******  **#******#*  **#*  **********  **********    *******    *#                                                 
       *#*  **  *** **#**  *#*  *#*    #*    *#*   *#*  **#*****#*  **#*****#*   *** **#**   *#                                                 
       *#*   *  **   **#*  **   *#*    #*     #*    #*   *#**  ***   *#**  ***   **   **#*   *#                                                 
       *#**  * **#######*  **   *#*    #*     #*    #*    #*    **    #*    **  **#######*   *#                                                 
       **#***  *#*            ***#*    #*     #*    #*    #*    *#    #*    *#  *#*          *#                                                 
        **##** *#*          **** #*    #*     #*    #*    #*    *#    #*    *#  *#*          *#                                                 
         ***#***#*      ** ***   #*    #*     #*    #*    #*    *#    #*    *#  *#*      **  *#                                                 
       *   **#**##*    *** ***  *#*    #*     #*   *#*    #*    *#    #*    *#  *##*    ***  *#                                                 
       **   *#* *#**  ***  ***  *#**   *** *  ***  *#**  *#*    *#*  *#*    *#*  *#**  ***   *#*                                                
       ***  *#* **#******  *#*****#**  *#***  *#****##** *#**  **#*  *#**  **#*  **#******  **#*                                                
       *#*****   *******   **********  *****  ************##****##****##****##**  *******  **##**                                               
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'log4j2.debug' to show Log4j2 internal initialization logging.
2022-01-08 16:50:10 INFO  SparkUI:54 - Stopped Spark web UI at http://*.*.0.9:22770

Flink or Spark Version

Spark version 2.4.5-hw-ei-310012

Java or Scala Version

Java 1.8.0_272 ojdbc8-12.2.0.1.jar

Screenshots

image

Are you willing to submit PR?

Code of Conduct

davidzollo commented 2 years ago

good job, as a global project, please describe the title and content in English, thx

xll-gits commented 2 years ago

Oracle JDBC is in the wrong place