Closed QiuZiXian closed 1 year ago
field_mapper{
age = age
name = new_name
}
may be modified to
field_mapper = {
age = age
name = new_name
}
thanks. but it not works.I rechecked my configuration. The correct configuration is as follows But there are new problems:
2023-10-23 15:05:45,804 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-405331] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2023-10-23 15:05:45,809 INFO com.hazelcast.internal.server.tcp.TcpServerConnection - [localhost]:5801 [seatunnel-405331] [5.1] Connection[id=1, /127.0.0.1:5801->/127.0.0.1:31027, qualifier=null, endpoint=[127.0.0.1]:31027, remoteUuid=91603d25-04c5-44ab-a0aa-dd99cefbe535, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Connection closed by the other side
2023-10-23 15:05:45,810 INFO com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel-405331] [5.1] Removed connection to endpoint: [localhost]:5801:44a6f32d-2736-47eb-88d3-322b2f4caff9, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/127.0.0.1:31027->localhost/127.0.0.1:5801}, remoteAddress=[localhost]:5801, lastReadTime=2023-10-23 15:05:45.797, lastWriteTime=2023-10-23 15:05:45.759, closedTime=2023-10-23 15:05:45.805, connected server version=5.1}
2023-10-23 15:05:45,810 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-405331] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
2023-10-23 15:05:45,812 INFO com.hazelcast.client.impl.ClientEndpointManager - [localhost]:5801 [seatunnel-405331] [5.1] Destroying ClientEndpoint{connection=Connection[id=1, /127.0.0.1:5801->/127.0.0.1:31027, qualifier=null, endpoint=[127.0.0.1]:31027, remoteUuid=91603d25-04c5-44ab-a0aa-dd99cefbe535, alive=false, connectionType=JVM, planeIndex=-1], clientUuid=91603d25-04c5-44ab-a0aa-dd99cefbe535, clientName=hz.client_1, authenticated=true, clientVersion=5.1, creationTime=1698044742786, latest clientAttributes=lastStatisticsCollectionTime=1698044742839,enterprise=false,clientType=JVM,clientVersion=5.1,clusterConnectionTimestamp=1698044742715,clientAddress=127.0.0.1,clientName=hz.client_1,credentials.principal=null,os.committedVirtualMemorySize=17767665664,os.freePhysicalMemorySize=437083398144,os.freeSwapSpaceSize=0,os.maxFileDescriptorCount=409600,os.openFileDescriptorCount=57,os.processCpuTime=12840000000,os.systemLoadAverage=2.89,os.totalPhysicalMemorySize=540564987904,os.totalSwapSpaceSize=0,runtime.availableProcessors=48,runtime.freeMemory=976949352,runtime.maxMemory=1029177344,runtime.totalMemory=1029177344,runtime.uptime=3466,runtime.usedMemory=52227992, labels=[]}
2023-10-23 15:05:45,813 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-405331] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2023-10-23 15:05:45,814 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed SeaTunnel client......
2023-10-23 15:05:45,814 INFO com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-405331] [5.1] [localhost]:5801 is SHUTTING_DOWN
2023-10-23 15:05:45,818 INFO com.hazelcast.internal.partition.impl.MigrationManager - [localhost]:5801 [seatunnel-405331] [5.1] Shutdown request of Member [localhost]:5801 - 44a6f32d-2736-47eb-88d3-322b2f4caff9 this is handled
2023-10-23 15:05:45,827 INFO com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-405331] [5.1] Shutting down connection manager...
2023-10-23 15:05:45,830 INFO com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-405331] [5.1] Shutting down node engine...
2023-10-23 15:05:48,347 INFO com.hazelcast.instance.impl.NodeExtension - [localhost]:5801 [seatunnel-405331] [5.1] Destroying node NodeExtension.
2023-10-23 15:05:48,348 INFO com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-405331] [5.1] Hazelcast Shutdown is completed in 2531 ms.
2023-10-23 15:05:48,348 INFO com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-405331] [5.1] [localhost]:5801 is SHUTDOWN
2023-10-23 15:05:48,348 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed HazelcastInstance ......
2023-10-23 15:05:48,348 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed metrics executor service ......
2023-10-23 15:05:48,350 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - run shutdown hook because get close signal
run shutdown hook because get close signal Is this an unsuccessful run?
Will there be a visual console address if the operation is successful?
thanks. but it not works.I rechecked my configuration. The correct configuration is as follows But there are new problems:
2023-10-23 15:05:45,804 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-405331] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN 2023-10-23 15:05:45,809 INFO com.hazelcast.internal.server.tcp.TcpServerConnection - [localhost]:5801 [seatunnel-405331] [5.1] Connection[id=1, /127.0.0.1:5801->/127.0.0.1:31027, qualifier=null, endpoint=[127.0.0.1]:31027, remoteUuid=91603d25-04c5-44ab-a0aa-dd99cefbe535, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Connection closed by the other side 2023-10-23 15:05:45,810 INFO com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel-405331] [5.1] Removed connection to endpoint: [localhost]:5801:44a6f32d-2736-47eb-88d3-322b2f4caff9, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/127.0.0.1:31027->localhost/127.0.0.1:5801}, remoteAddress=[localhost]:5801, lastReadTime=2023-10-23 15:05:45.797, lastWriteTime=2023-10-23 15:05:45.759, closedTime=2023-10-23 15:05:45.805, connected server version=5.1} 2023-10-23 15:05:45,810 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-405331] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED 2023-10-23 15:05:45,812 INFO com.hazelcast.client.impl.ClientEndpointManager - [localhost]:5801 [seatunnel-405331] [5.1] Destroying ClientEndpoint{connection=Connection[id=1, /127.0.0.1:5801->/127.0.0.1:31027, qualifier=null, endpoint=[127.0.0.1]:31027, remoteUuid=91603d25-04c5-44ab-a0aa-dd99cefbe535, alive=false, connectionType=JVM, planeIndex=-1], clientUuid=91603d25-04c5-44ab-a0aa-dd99cefbe535, clientName=hz.client_1, authenticated=true, clientVersion=5.1, creationTime=1698044742786, latest clientAttributes=lastStatisticsCollectionTime=1698044742839,enterprise=false,clientType=JVM,clientVersion=5.1,clusterConnectionTimestamp=1698044742715,clientAddress=127.0.0.1,clientName=hz.client_1,credentials.principal=null,os.committedVirtualMemorySize=17767665664,os.freePhysicalMemorySize=437083398144,os.freeSwapSpaceSize=0,os.maxFileDescriptorCount=409600,os.openFileDescriptorCount=57,os.processCpuTime=12840000000,os.systemLoadAverage=2.89,os.totalPhysicalMemorySize=540564987904,os.totalSwapSpaceSize=0,runtime.availableProcessors=48,runtime.freeMemory=976949352,runtime.maxMemory=1029177344,runtime.totalMemory=1029177344,runtime.uptime=3466,runtime.usedMemory=52227992, labels=[]} 2023-10-23 15:05:45,813 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-405331] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN 2023-10-23 15:05:45,814 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed SeaTunnel client...... 2023-10-23 15:05:45,814 INFO com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-405331] [5.1] [localhost]:5801 is SHUTTING_DOWN 2023-10-23 15:05:45,818 INFO com.hazelcast.internal.partition.impl.MigrationManager - [localhost]:5801 [seatunnel-405331] [5.1] Shutdown request of Member [localhost]:5801 - 44a6f32d-2736-47eb-88d3-322b2f4caff9 this is handled 2023-10-23 15:05:45,827 INFO com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-405331] [5.1] Shutting down connection manager... 2023-10-23 15:05:45,830 INFO com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-405331] [5.1] Shutting down node engine... 2023-10-23 15:05:48,347 INFO com.hazelcast.instance.impl.NodeExtension - [localhost]:5801 [seatunnel-405331] [5.1] Destroying node NodeExtension. 2023-10-23 15:05:48,348 INFO com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-405331] [5.1] Hazelcast Shutdown is completed in 2531 ms. 2023-10-23 15:05:48,348 INFO com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-405331] [5.1] [localhost]:5801 is SHUTDOWN 2023-10-23 15:05:48,348 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed HazelcastInstance ...... 2023-10-23 15:05:48,348 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed metrics executor service ...... 2023-10-23 15:05:48,350 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - run shutdown hook because get close signal
run shutdown hook because get close signal Is this an unsuccessful run?
Did you find any logs with 'ConsoleSinkWriter'?If you can, you will look some output records.
Will there be a visual console address if the operation is successful?
thanks,but I mean this:https://seatunnel.apache.org/seatunnel_web/1.0.0/deploy where can I get help of or commit the issue of seatunnel_web ?
I deploy seatunnel by the method of quick-start-seatunnel-engin. and The following code runs successfully ` cd "apache-seatunnel-${version}" ./bin/seatunnel.sh --config ./config/v2.batch.config.template -e local
but with [rest-api](https://seatunnel.apache.org/docs/2.3.3/seatunnel-engine/rest-api) post task always fail.(Whether it is fake data or use mysql) ![image](https://github.com/apache/seatunnel/assets/28780269/71ee437f-55d8-4488-bab0-55f9320cecd5) ![image](https://github.com/apache/seatunnel/assets/28780269/9b275716-9d29-4786-9c9b-276fa7f97ac3)
json
{ "env": { "job.mode": "batch" }, "source": [ { "plugin_name": "FakeSource", "result_table_name": "fake", "row.num": 100, "schema": { "fields": { "name": "string", "age": "int", "card": "int" } } } ], "transform": [ ], "sink": [ { "plugin_name": "Console", "source_table_name": ["fake"] } ] } `
seatunnel web repo: https://github.com/apache/seatunnel-web.
Search before asking
What happened
do as https://seatunnel.apache.org/docs/2.3.3/start-v2/locally/quick-start-seatunnel-engine. ./bin/seatunnel.sh --config ./config/v2.batch.config.template -e local throw error: ` 2023-10-23 11:25:10,845 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:191) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34) Caused by: java.lang.ClassCastException: java.util.ArrayList cannot be cast to java.util.Map at java.util.ArrayList.forEach(ArrayList.java:1257) at org.apache.seatunnel.core.starter.utils.ConfigShadeUtils.processConfig(ConfigShadeUtils.java:150) at org.apache.seatunnel.core.starter.utils.ConfigShadeUtils.decryptConfig(ConfigShadeUtils.java:119) at org.apache.seatunnel.core.starter.utils.ConfigShadeUtils.decryptConfig(ConfigShadeUtils.java:104) at org.apache.seatunnel.core.starter.utils.ConfigBuilder.ofInner(ConfigBuilder.java:53) at org.apache.seatunnel.core.starter.utils.ConfigBuilder.lambda$of$1(ConfigBuilder.java:67) at java.util.Optional.orElseGet(Optional.java:267) at org.apache.seatunnel.core.starter.utils.ConfigBuilder.of(ConfigBuilder.java:67) at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.(MultipleTableJobConfigParser.java:127)
at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.getJobConfigParser(JobExecutionEnvironment.java:63)
at org.apache.seatunnel.engine.core.job.AbstractJobEnvironment.getLogicalDag(AbstractJobEnvironment.java:109)
at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.execute(JobExecutionEnvironment.java:73)
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:143)
... 2 more
2023-10-23 11:25:10,845 ERROR org.apache.seatunnel.core.starter.SeaTunnel -
Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:191) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34) Caused by: java.lang.ClassCastException: java.util.ArrayList cannot be cast to java.util.Map at java.util.ArrayList.forEach(ArrayList.java:1257) at org.apache.seatunnel.core.starter.utils.ConfigShadeUtils.processConfig(ConfigShadeUtils.java:150) at org.apache.seatunnel.core.starter.utils.ConfigShadeUtils.decryptConfig(ConfigShadeUtils.java:119) at org.apache.seatunnel.core.starter.utils.ConfigShadeUtils.decryptConfig(ConfigShadeUtils.java:104) at org.apache.seatunnel.core.starter.utils.ConfigBuilder.ofInner(ConfigBuilder.java:53) at org.apache.seatunnel.core.starter.utils.ConfigBuilder.lambda$of$1(ConfigBuilder.java:67) at java.util.Optional.orElseGet(Optional.java:267) at org.apache.seatunnel.core.starter.utils.ConfigBuilder.of(ConfigBuilder.java:67) at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.(MultipleTableJobConfigParser.java:127)
at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.getJobConfigParser(JobExecutionEnvironment.java:63)
at org.apache.seatunnel.engine.core.job.AbstractJobEnvironment.getLogicalDag(AbstractJobEnvironment.java:109)
at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.execute(JobExecutionEnvironment.java:73)
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:143)
... 2 more`
SeaTunnel Version
2.3.3
SeaTunnel Config
Error Exception
Zeta or Flink or Spark Version
zeta
Java or Scala Version
java1.8
ava version "1.8.0_181" Java(TM) SE Runtime Environment (build 1.8.0_181-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
Screenshots
Are you willing to submit PR?
Code of Conduct