HashDataInc / bireme

Bireme is an incremental synchronization tool for the Greenplum / HashData data warehouse
https://hashdatainc.github.io/bireme/
Apache License 2.0
137 stars 53 forks source link

导入数据时异常 #21

Closed DongJianzhi0223 closed 7 years ago

DongJianzhi0223 commented 7 years ago
cn.hashdata.bireme.BiremeException: Fail to create tmporary table.
        at cn.hashdata.bireme.ChangeLoader.createTemporaryTable(ChangeLoader.java:503) ~[bireme-1.0.0.jar:?]
        at cn.hashdata.bireme.ChangeLoader.getConnection(ChangeLoader.java:204) ~[bireme-1.0.0.jar:?]
        at cn.hashdata.bireme.ChangeLoader.call(ChangeLoader.java:116) [bireme-1.0.0.jar:?]
        at cn.hashdata.bireme.ChangeLoader.call(ChangeLoader.java:44) [bireme-1.0.0.jar:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_121]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_121]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_121]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
Caused by: org.postgresql.util.PSQLException: ERROR: Interconnect error reading register message from 180.163.8.254:41787: format not recognized  (seg1 edaddw05:40000 pid=2699)
  Detail: msgBytes=65 expected=32 sockfd=26 local=10.141.39.211:1981
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2102) ~[postgresql-9.1-901.jdbc4.jar:?]
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1835) ~[postgresql-9.1-901.jdbc4.jar:?]
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257) ~[postgresql-9.1-901.jdbc4.jar:?]
        at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:500) ~[postgresql-9.1-901.jdbc4.jar:?]
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:374) ~[postgresql-9.1-901.jdbc4.jar:?]
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:302) ~[postgresql-9.1-901.jdbc4.jar:?]
        at cn.hashdata.bireme.ChangeLoader.createTemporaryTable(ChangeLoader.java:500) ~[bireme-1.0.0.jar:?]
        ... 9 more

您好,在BIREME执行过程中发现报出该错误,定位到是 while (iterator.hasNext() && !cxt.stop) { data = iterator.next().getBytes("UTF-8"); pipeOut.write(data); } 在getBytes时出现的问题。 请您协助定位一下原因,谢谢!

wangzw commented 7 years ago

Please post Greenplum log of seg1 on edaddw05.

wangzw commented 7 years ago

I just check the Greenplum code, it seems a bug of gpdb. Please report this issue to Greenplum.

https://github.com/greenplum-db/gpdb/issues

DongJianzhi0223 commented 7 years ago

您好:

现在出现这种情况后,程序就假死掉了,进程还在,但是不继续工作了,如果是GPDB的BUG,是否先可以改成将异常抛出,程序退出,我做一个守护进程重启程序即可。这样做会有什么问题吗?

wangzw commented 7 years ago

We will refactor failover process in next release. Thanks.

DongJianzhi0223 commented 7 years ago

you are welcome.