[BUG] jdbc Connector PostgreSQL java.lang.NoClassDefFoundError: net/postgis/jdbc/geometry/Geometry

[MZSY]

Search before asking

• [X] I had searched in the issues and found no similar issues.

What happened

我需要的使用场景是：将多源数据导入到postgresql数据库中 我在apache-seatunnel-2.3.3/plugins/jdbc/lib下放了postgresql-42.6.0.jar和postgis-jdbc-2021.1.0.jar两个jar包，都是从maven仓库里下载的，但当我执行测试脚本的时候，我遇到了报错，希望能帮我看看这个问题

SeaTunnel Version

2.3.3

SeaTunnel Config

env {
  execution.parallelism = 1
  job.mode = "BATCH"
}

source {
  jdbc {
    url = "jdbc:postgresql://172.16.111.3:5433/test"
    driver = "org.postgresql.Driver"
    user = "postgres"
    password = "testdb"
    query = "select * from testdb_0808"
  }
}

transform {
}

sink {
  Console {}
}

Running Command

/data/seaTunnel/apache-seatunnel-2.3.3/bin/seatunnel.sh --config /data/seaTunnel/apache-seatunnel-2.3.3/config/seatunnel.postgresql.conf.template -e local

Error Exception

2023-12-18 18:37:39,655 ERROR org.apache.seatunnel.core.starter.SeaTunnel - 

===============================================================================

2023-12-18 18:37:39,655 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Fatal Error, 

2023-12-18 18:37:39,655 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Please submit bug report in https://github.com/apache/seatunnel/issues

2023-12-18 18:37:39,655 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Reason:SeaTunnel job executed failed 

2023-12-18 18:37:39,656 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
    at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:191)
    at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
    at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.lang.NoClassDefFoundError: net/postgis/jdbc/geometry/Geometry
    at java.lang.Class.getDeclaredConstructors0(Native Method)
    at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
    at java.lang.Class.getConstructor0(Class.java:3075)
    at java.lang.Class.getDeclaredConstructor(Class.java:2178)
    at org.postgresql.jdbc.PgConnection.getObject(PgConnection.java:739)
    at org.postgresql.jdbc.PgResultSet.getObject(PgResultSet.java:3020)
    at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.dialect.psql.PostgresJdbcRowConverter.toInternal(PostgresJdbcRowConverter.java:59)
    at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcInputFormat.nextRecord(JdbcInputFormat.java:220)
    at org.apache.seatunnel.connectors.seatunnel.jdbc.source.JdbcSourceReader.pollNext(JdbcSourceReader.java:63)
    at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.collect(SourceFlowLifeCycle.java:150)
    at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.collect(SourceSeaTunnelTask.java:95)
    at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168)
    at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:100)
    at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:613)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.ClassNotFoundException: net.postgis.jdbc.geometry.Geometry
    at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
    at org.apache.seatunnel.engine.common.loader.SeaTunnelBaseClassLoader.loadClassWithoutExceptionHandling(SeaTunnelBaseClassLoader.java:56)
    at org.apache.seatunnel.engine.common.loader.SeaTunnelChildFirstClassLoader.loadClassWithoutExceptionHandling(SeaTunnelChildFirstClassLoader.java:95)
    at org.apache.seatunnel.engine.common.loader.SeaTunnelBaseClassLoader.loadClass(SeaTunnelBaseClassLoader.java:47)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
    ... 19 more

    at org.apache.seatunnel.engine.client.job.ClientJobProxy.waitForJobComplete(ClientJobProxy.java:122)
    at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:184)
    ... 2 more

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[MZSY]

这里报错说找不到这个class文件java.lang.NoClassDefFoundError: net/postgis/jdbc/geometry/Geometry，我去看了postgis-jdbc.jar的包结构，只能找到net/postgis/jdbc这一级目录，往后的找不到了。我不确定是代码找的目录写错了，或者是我用的包版本不对，又或者是我其他地方配置有问题，请帮我看看这个问题，十分感谢

[Anubisxcw]

看不了一点，我的报错是Caused by: java.lang.ClassNotFoundException: org.apache.seatunnel.api.source.SeaTunnelSource at java.net.URLClassLoader.findClass(URLClassLoader.java:407)

[Carl-Zhou-CN]

这里报错说找不到这个class文件java.lang.NoClassDefFoundError: net/postgis/jdbc/geometry/Geometry，我去看了postgis-jdbc.jar的包结构，只能找到net/postgis/jdbc这一级目录，往后的找不到了。我不确定是代码找的目录写错了，或者是我用的包版本不对，又或者是我其他地方配置有问题，请帮我看看这个问题，十分感谢

Can you provide ddl and pgsql versions of the source table?

[MZSY]

DDL：CREATE TABLE "public"."geom_test1206" ( "AREA_CODE" varchar(20) COLLATE "pg_catalog"."default", "NAME" varchar(160) COLLATE "pg_catalog"."default", "geom" geometry(GEOMETRY), "gid" varchar(50) COLLATE "pg_catalog"."default", "cxflm" varchar(255) COLLATE "pg_catalog"."default", "center_geom" geometry(GEOMETRY), "geom_3857" geometry(GEOMETRY), "xian_name" varchar(255) COLLATE "pg_catalog"."default", "old_code" varchar(255) COLLATE "pg_catalog"."default", "test_geom" geometry(GEOMETRY), "is_frozen" int4 DEFAULT 0 ) ;

ALTER TABLE "public"."geom_test1206" OWNER TO "postgres";

pgsql version is 11.6

I'm really sorry that I switched my mailbox before and I didn't receive your reply. Could this issue be related to the version of the pg database? I know my version is probably old.

在 2023-12-24 16:22:50，"Carl-Zhou-CN" @.***> 写道：

这里报错说找不到这个class文件java.lang.NoClassDefFoundError: net/postgis/jdbc/geometry/Geometry，我去看了postgis-jdbc.jar的包结构，只能找到net/postgis/jdbc这一级目录，往后的找不到了。我不确定是代码找的目录写错了，或者是我用的包版本不对，又或者是我其他地方配置有问题，请帮我看看这个问题，十分感谢

Can you provide ddl and pgsql versions of the source table?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[Carl-Zhou-CN]

@MZSY hi,It seems that you also need to add and change the jar package, you can select the corresponding jar package version to try https://repo1.maven.org/maven2/net/postgis/postgis-geometry/2.5.1/postgis-geometry-2.5.1.jar,similar problems occurred when I removed the package, but I did not find pgsql :11.6 environment to try, I hope you can help to try it

[frozer798]

@Carl-Zhou-CN In fact, I later tried to remove the postgis jar package and keep only the postgresql jar package, and there was no such problem. If I just want to copy spatial data from one pg database to another pg database, is it necessary to use postgis jar packages? PS.In fact, I also encountered another problem when importing data, I found that if there is a field like 3857_geom (number beginning) or AREA_NAME (all caps) in the source table, the transmission process will also report an error problem

[frozer798]

Shall I raise this issue again or consult it here?

[Carl-Zhou-CN]

Shall I raise this issue again or consult it here?

You can create a new issue and associate it

[Carl-Zhou-CN]

@Carl-Zhou-CN In fact, I later tried to remove the postgis jar package and keep only the postgresql jar package, and there was no such problem. If I just want to copy spatial data from one pg database to another pg database, is it necessary to use postgis jar packages? PS.In fact, I also encountered another problem when importing data, I found that if there is a field like 3857_geom (number beginning) or AREA_NAME (all caps) in the source table, the transmission process will also report an error problem

Can you help refine the documentation for this？

[frozer798]

@Carl-Zhou-CN In fact, I later tried to remove the postgis jar package and keep only the postgresql jar package, and there was no such problem. If I just want to copy spatial data from one pg database to another pg database, is it necessary to use postgis jar packages? PS.In fact, I also encountered another problem when importing data, I found that if there is a field like 3857_geom (number beginning) or AREA_NAME (all caps) in the source table, the transmission process will also report an error problem

Can you help refine the documentation for this？

I'm sorry, I don't understand what you mean about refining documentation, do you mean that I should take this commission issue or something else?

[Carl-Zhou-CN]

I think you found a way to add jar packages differently due to PostgreSQL versions. I was hoping you could help me add a reminder to the documentation, re you interested？ https://github.com/apache/seatunnel/blob/dev/docs/en/connector-v2/source/PostgreSQL.md

[frozer798]

I'd love to help refine the markdown documentation, but I don't think I've tested a strong correlation between the PostgreSQL version and the jdbc jar package version. The PG version I was using was 11.6, but the jdbc jar package I was using was the latest 42.6.x version and I had no other problems executing it

[Carl-Zhou-CN]

I'd love to help refine the markdown documentation, but I don't think I've tested a strong correlation between the PostgreSQL version and the jdbc jar package version. The PG version I was using was 11.6, but the jdbc jar package I was using was the latest 42.6.x version and I had no other problems executing it

Thank you very much, PostgreSQL to PostgreSQL I will find time to build 11.6 to try, it is essentially jdbc, if jdbc is compatible, it is compatible

[frozer798]

I'd love to help refine the markdown documentation, but I don't think I've tested a strong correlation between the PostgreSQL version and the jdbc jar package version. The PG version I was using was 11.6, but the jdbc jar package I was using was the latest 42.6.x version and I had no other problems executing it

Thank you very much, PostgreSQL to PostgreSQL I will find time to build 11.6 to try, it is essentially jdbc, if jdbc is compatible, it is compatible

You're welcome. I wasn't very helpful

[Carl-Zhou-CN]

@frozer798 hi，I tried to use postgis/postgis:11-3.3 docker image to build the environment. Do as you say only need to add the https://repo1.maven.org/maven2/org/postgresql/postgresql/42.3.3/postgresql-42.3.3.jar PostgreSQL to PostgreSQL is also running properly and data is synchronized properly

[Carl-Zhou-CN]

Sorry, there seems to be other problems, I will update after checking

@frozer798 hi，I tried to use postgis/postgis:11-3.3 docker image to build the environment. Do as you say only need to add the https://repo1.maven.org/maven2/org/postgresql/postgresql/42.3.3/postgresql-42.3.3.jar PostgreSQL to PostgreSQL is also running properly and data is synchronized properly

[Carl-Zhou-CN]

@frozer798 Sorry,current version seatunnel cannot transfer PostgreSQL to PostgreSQL geometry at this time because geometry is currently recognized as a String，It will be added in the next version

[Carl-Zhou-CN]

link #5872

[frozer798]

@frozer798 Sorry,current version seatunnel cannot transfer PostgreSQL to PostgreSQL geometry at this time because geometry is currently recognized as a String，It will be added in the next version

Did you mean that transferring geometry data from PostgreSQL database to PostgreSQL database is not currently supported?

[Carl-Zhou-CN]

@frozer798 Sorry,current version seatunnel cannot transfer PostgreSQL to PostgreSQL geometry at this time because geometry is currently recognized as a String，It will be added in the next version

Did you mean that transferring geometry data from PostgreSQL database to PostgreSQL database is not currently supported?

Yes, but you can refer to it https://github.com/apache/seatunnel/blob/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/connector-jdbc-e2e-part-3/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/JdbcPostgresIT.java t provides a roundabout way of thinking

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.