apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
8.02k stars 1.82k forks source link

[Bug] [Release] v2.3.4 binary distribution broken in Linux due to CRLF #6362

Closed tongyifan closed 8 months ago

tongyifan commented 8 months ago

Search before asking

What happened

Deployment from binary distribution not working in Linux due to CRLF

image

SeaTunnel Version

2.3.4

SeaTunnel Config

-

Running Command

export version="2.3.4"
wget "https://archive.apache.org/dist/seatunnel/${version}/apache-seatunnel-${version}-bin.tar.gz"
tar -xzvf "apache-seatunnel-${version}-bin.tar.gz"

cd apache-seatunnel-2.3.4
sh bin/install-plugin.sh 2.3.4

Error Exception

Install SeaTunnel connectors plugins, usage version is 2.3.4
install connector :  connector-amazondynamodb
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-assert
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-cassandra
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-cdc-mysql
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-cdc-mongodb
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-cdc-sqlserver
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-clickhouse
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-datahub
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-dingtalk
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-doris
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-elasticsearch
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-email
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-file-ftp
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-file-hadoop
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-file-local
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-file-oss
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-file-jindo-oss
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-file-s3
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-file-sftp
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-google-sheets
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-google-firestore
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-hive
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-base
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-feishu
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-gitlab
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-github
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-jira
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-klaviyo
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-lemlist
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-myhours
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-notion
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-onesignal
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-wechat
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-hudi
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-iceberg
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-influxdb
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-iotdb
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-jdbc
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-kafka
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-kudu
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-maxcompute
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-mongodb
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-neo4j
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-openmldb
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-pulsar
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-rabbitmq
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-redis
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-s3-redshift
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-sentry
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-slack
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-socket
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-starrocks
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-tablestore
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-selectdb-cloud
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-hbase
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-amazonsqs
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

Code of Conduct

dzygoon commented 8 months ago

+1,mvnw 脚本格式为windows,非unix格式,即便使用dos2unix转换格式也无法使用。

sunhailin-Leo commented 8 months ago

@tongyifan @dzygoon

dzygoon commented 8 months ago

@tongyifan @dzygoon

  • Linux 我试了一下用这个命令:find <seatunnel_home 路径> -type f -print0 | xargs -0 dos2unix -- ,然后再执行插件安装就好了
  • 肯定不知道谁提交配置或者脚本的时候给了 windows 的换行符上去。。。

确实可以了,我完全没想到转换整个目录的文件格式,估计不止是mvnw 这个脚本的问题。

sunhailin-Leo commented 8 months ago

@tongyifan @dzygoon

  • Linux 我试了一下用这个命令:find <seatunnel_home 路径> -type f -print0 | xargs -0 dos2unix -- ,然后再执行插件安装就好了
  • 肯定不知道谁提交配置或者脚本的时候给了 windows 的换行符上去。。。

确实可以了,我完全没想到转换整个目录的文件格式,估计不止是mvnw 这个脚本的问题。

如果用仓库分支那里构建是没问题的,就是 apache 上的 bin 包有问题~

BranGao19 commented 8 months ago

@tongyifan @dzygoon

  • Linux 我试了一下用这个命令:find <seatunnel_home 路径> -type f -print0 | xargs -0 dos2unix -- ,然后再执行插件安装就好了
  • 肯定不知道谁提交配置或者脚本的时候给了 windows 的换行符上去。。。

确实可以了,我完全没想到转换整个目录的文件格式,估计不止是mvnw 这个脚本的问题。

如果用仓库分支那里构建是没问题的,就是 apache 上的 bin 包有问题~

我用最新分支windows maven构建也是有这个问题的

dzygoon commented 8 months ago

@tongyifan @dzygoon

  • Linux 我试了一下用这个命令:find <seatunnel_home 路径> -type f -print0 | xargs -0 dos2unix -- ,然后再执行插件安装就好了
  • 肯定不知道谁提交配置或者脚本的时候给了 windows 的换行符上去。。。

确实可以了,我完全没想到转换整个目录的文件格式,估计不止是mvnw 这个脚本的问题。

如果用仓库分支那里构建是没问题的,就是 apache 上的 bin 包有问题~

我用最新分支windows maven构建也是有这个问题的

修改下./.mvn/wrapper/maven-wrapper.properties的换行符,有人提交代码的时候把win换行符换上了。

qxhy123 commented 8 months ago

Mac上遇到了同样的问题, 换成v2.3.3就正常了

Carl-Zhou-CN commented 8 months ago

May be used temporarily, https://dist.apache.org/repos/dist/dev/seatunnel/2.3.4/ But he's temporary, waiting for a new release

Carl-Zhou-CN commented 8 months ago

@tongyifan @dzygoon @sunhailin-Leo @qxhy123 hi ,the issue has been resolved, please re-download the distribution

dzygoon commented 8 months ago

@tongyifan @dzygoon @sunhailin-Leo @qxhy123 hi ,the issue has been resolved, please re-download the distribution

good job :D