[Bug] [ftp] source has type list of STRING rather than list of OBJECT

Alxe1 commented 1 year ago

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened

env {
  spark.app.name = "SeaTunnel"
  spark.executor.instances = 2
  spark.executor.cores = 1
  spark.executor.memory = "1g"
}

source {
  FtpFile {
    path="/root/agent.json"
    host="myhost"
    port=22
    user=my_user_name
    password="my password"
    type="text"
  }

  delimiter=":"
  schema {
    fields {
      name=string
      value=string
    }
  }
  result_table_name="local_table"
}

transform {
  Sql {
    sql="select name, value from local_table"
  }
}

sink {

  Console {}

}

I run ./bin/start-seatunnel-spark-connector-v2.sh -c test.conf -m local[2] -e client but went wrong:

log4j:WARN No appenders could be found for logger (org.apache.seatunnel.core.starter.config.ConfigBuilder).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" org.apache.seatunnel.shade.com.typesafe.config.ConfigException$WrongType: test.conf: 19: source has type list of STRING rather than list of OBJECT
    at org.apache.seatunnel.shade.com.typesafe.config.impl.SimpleConfig.getHomogeneousWrappedList(SimpleConfig.java:452)
    at org.apache.seatunnel.shade.com.typesafe.config.impl.SimpleConfig.getObjectList(SimpleConfig.java:460)
    at org.apache.seatunnel.shade.com.typesafe.config.impl.SimpleConfig.getConfigList(SimpleConfig.java:465)
    at org.apache.seatunnel.core.starter.spark.SparkStarter.lambda$getPluginIdentifiers$7(SparkStarter.java:269)
    at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:269)
    at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
    at org.apache.seatunnel.core.starter.spark.SparkStarter.getPluginIdentifiers(SparkStarter.java:273)
    at org.apache.seatunnel.core.starter.spark.SparkStarter.getConnectorJarDependencies(SparkStarter.java:184)
    at org.apache.seatunnel.core.starter.spark.SparkStarter.buildCommands(SparkStarter.java:125)
    at org.apache.seatunnel.core.starter.spark.SparkStarter.main(SparkStarter.java:97)

SeaTunnel Version

2.3.0-beta

SeaTunnel Config

env {
  spark.app.name = "SeaTunnel"
  spark.executor.instances = 2
  spark.executor.cores = 1
  spark.executor.memory = "1g"
}

source {
  FtpFile {
    path="/root/agent.json"
    host="myhost"
    port=22
    user=my_user_name
    password="my password"
    type="text"
  }

  delimiter=":"
  schema {
    fields {
      name=string
      value=string
    }
  }
  result_table_name="local_table"
}

transform {
  Sql {
    sql="select name, value from local_table"
  }
}

sink {

  Console {}

}

Running Command

./bin/start-seatunnel-spark-connector-v2.sh -c test.conf -m local[2] -e client

Error Exception

log4j:WARN No appenders could be found for logger (org.apache.seatunnel.core.starter.config.ConfigBuilder).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" org.apache.seatunnel.shade.com.typesafe.config.ConfigException$WrongType: test.conf: 19: source has type list of STRING rather than list of OBJECT
    at org.apache.seatunnel.shade.com.typesafe.config.impl.SimpleConfig.getHomogeneousWrappedList(SimpleConfig.java:452)
    at org.apache.seatunnel.shade.com.typesafe.config.impl.SimpleConfig.getObjectList(SimpleConfig.java:460)
    at org.apache.seatunnel.shade.com.typesafe.config.impl.SimpleConfig.getConfigList(SimpleConfig.java:465)
    at org.apache.seatunnel.core.starter.spark.SparkStarter.lambda$getPluginIdentifiers$7(SparkStarter.java:269)
    at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:269)
    at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
    at org.apache.seatunnel.core.starter.spark.SparkStarter.getPluginIdentifiers(SparkStarter.java:273)
    at org.apache.seatunnel.core.starter.spark.SparkStarter.getConnectorJarDependencies(SparkStarter.java:184)
    at org.apache.seatunnel.core.starter.spark.SparkStarter.buildCommands(SparkStarter.java:125)
    at org.apache.seatunnel.core.starter.spark.SparkStarter.main(SparkStarter.java:97)

Flink or Spark Version

spark 2.4.8

Java or Scala Version

java 1.8

Screenshots

No response

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 1 year ago

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.

apache / seatunnel