apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
7.97k stars 1.8k forks source link

[Bug] quick-start-flink not working !!! ? #6832

Open zhouhoo opened 5 months ago

zhouhoo commented 5 months ago

Search before asking

What happened

When I try seatunne on flink as offical doc says in the link, I got no error , flink job running but console had NO sink data output as the doc says!!! I tried to ask slack channel for help ,but slack link is outdated!!! BTW, I tried to sink to mysql database, still no data sinked!

SeaTunnel Version

2.3.3

SeaTunnel Config

only modified seatunnel-env.sh file:
FLINK_HOME=/root/shen/flink-1.13.5

Running Command

1. add mysql-connector-j-8.3.0.jar to lib, 
2. install some plugins including fake-source and console.
3. ./bin/start-seatunnel-flink-13-connector-v2.sh --config ./config/v2.batch.config.template

Error Exception

Job has been submitted with JobID 607d2d65753670b5f381fc502cb65c57
Program execution finished
Job with JobID 607d2d65753670b5f381fc502cb65c57 has finished.
Job Runtime: 2207 ms

Zeta or Flink or Spark Version

flink 1.13.5

Java or Scala Version

No response

Screenshots

无标题

Are you willing to submit PR?

Code of Conduct

liunaijie commented 5 months ago

you submitted config is fake source to console, the console out is print in your worker node. not your submit job node.

liunaijie commented 5 months ago

you can update the config, write data to a local file or other connector, then verify the result

zhouhoo commented 5 months ago

you submitted config is fake source to console, the console out is print in your worker node. not your submit job node.

flink worker node? I start flink 、seatunnel all on same host, only one host. just a quick start.

zhouhoo commented 5 months ago

you can update the config, write data to a local file or other connector, then verify the result

I also sink to mysql database, no data sinked in.

liunaijie commented 5 months ago

try this way to verify:

  1. submit v2.streaming.conf.template config ( this is streaming job config)
  2. login flink ui check running flink jobs, check whether this job has submitted to your flink cluster.
  3. check log
zhouhoo commented 5 months ago

try this way to verify:

  1. submit v2.streaming.conf.template config ( this is streaming job config)
  2. login flink ui check running flink jobs, check whether this job has submitted to your flink cluster.
  3. check log

I tried, job was successfully submitted to flink, no error ,running status. but no data consuming. flink logs print some checkpoings was done successfully. seatunnel has no log output.

liunaijie commented 5 months ago

try this way to verify:

  1. submit v2.streaming.conf.template config ( this is streaming job config)
  2. login flink ui check running flink jobs, check whether this job has submitted to your flink cluster.
  3. check log

I tried, job was successfully submitted to flink, no error ,running status. but no data consuming. flink logs print some checkpoings was done successfully. seatunnel has no log output.

Hi, i package from dev, using flink 1.13.5. some update to v2.streaming.conf.template :

  1. update env.parallelism and FakeSource.parallelism to 1
  2. update FakeSource.row.num to 100

Then it can run well. i can see the result in taskmanager's log. image

github-actions[bot] commented 4 months ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.