search

IBMStreams / streamsx.jdbc

Supports Streams application to work with databases via JDBC
http://ibmstreams.github.io/streamsx.jdbc
Other
3 stars 15 forks source link

terminate from jdbcRun #108

Closed lpkamesam closed 4 years ago

lpkamesam commented 4 years ago

I see some issues when I use jdbcRun . When I specify 'sqlFailureAction' to 'log , I do get log of the records but after some records created in the log file I see the pe getting into stopped and unhealthy state and I don't believe after that state it is processing any records. Is there any limit on how many errors one can have or some parameter that jdbcRun to terminate after an error ?

lpkamesam commented 4 years ago

I found something in the logs and I was wondering where this is set (isJobCancellation) . Is this the reason why jdbcRun stops?
30 Mar 2020 17:02:48.439 [89592] DEBUG #splapptrc,J[5],P[62],updatedStreamSR M[?:com.ibm.streamsx.jdbc.JDBCRun.getStatementParameterArrays:-1] - Parameter statementParameters Value: 3 30 Mar 2020 17:02:48.445 [89592] DEBUG #splapptrc,J[5],P[62],updatedStreamSR M[?:com.ibm.streamsx.jdbc.JDBCRun.handleException:-1] - SQL Exception SQL Code: -302 30 Mar 2020 17:02:48.445 [89592] DEBUG #splapptrc,J[5],P[62],updatedStreamSR M[?:com.ibm.streamsx.jdbc.JDBCRun.handleException:-1] - SQL Exception SQL State: 22001 30 Mar 2020 17:02:48.446 [89592] DEBUG #splapptrc,J[5],P[62],updatedStreamSR M[?:com.ibm.streamsx.jdbc.JDBCRun.handleException:-1] - SQL Exception SQL Message: DB2 SQL Error: SQLCODE=-302, SQLSTATE=22001, SQLERRMC=null, DRIVER=3.67.26 30 Mar 2020 17:02:48.446 [89592] DEBUG #splapptrc,J[5],P[62],updatedStreamSR M[?:com.ibm.streamsx.jdbc.JDBCRun.submitErrorTuple:-1] - Submit error tuple... 30 Mar 2020 17:02:48.448 [89592] ERROR #splapptrc,J[5],P[62],updatedStreamSR,spl_javaop M[?:com.ibm.streams.operator.internal.jni.JNIBridge.process:-1] - java/sql/Connection.isValid(I)Z 30 Mar 2020 17:02:48.448 [89592] ERROR #splapptrc,J[5],P[62],updatedStreamSR,spl_javaop M[?:?:0] - java.lang.AbstractMethodError: java/sql/Connection.isValid(I)Z 30 Mar 2020 17:02:48.448 [89592] ERROR #splapptrc,J[5],P[62],updatedStreamSR,spl_javaop M[?:?:0] - com.ibm.streamsx.jdbc.JDBCClientHelper.isValidConnection(JDBCClientHelper.java:221) 30 Mar 2020 17:02:48.449 [89592] ERROR #splapptrc,J[5],P[62],updatedStreamSR,spl_javaop M[?:?:0] - com.ibm.streamsx.jdbc.JDBCRun.handleException(JDBCRun.java:791) 30 Mar 2020 17:02:48.449 [89592] ERROR #splapptrc,J[5],P[62],updatedStreamSR,spl_javaop M[?:?:0] - com.ibm.streamsx.jdbc.JDBCRun.processTuple(JDBCRun.java:744) 30 Mar 2020 17:02:48.449 [89592] ERROR #splapptrc,J[5],P[62],updatedStreamSR,spl_javaop M[?:?:0] - com.ibm.streamsx.jdbc.AbstractJDBCOperator.process(AbstractJDBCOperator.java:579) 30 Mar 2020 17:02:48.449 [89592] ERROR #splapptrc,J[5],P[62],updatedStreamSR,spl_javaop M[?:?:0] - com.ibm.streams.operator.internal.runtime.api.OperatorAdapter.processTuple(OperatorAdapter.java:1592) 30 Mar 2020 17:02:48.450 [89592] ERROR #splapptrc,J[5],P[62],updatedStreamSR,spl_javaop M[?:?:0] - com.ibm.streams.operator.internal.runtime.api.OperatorAdapter$1.tuple(OperatorAdapter.java:826) 30 Mar 2020 17:02:48.450 [89592] ERROR #splapptrc,J[5],P[62],updatedStreamSR,spl_javaop M[?:?:0] - com.ibm.streams.operator.internal.runtime.api.OperatorAdapter$1.tuple(OperatorAdapter.java:819) 30 Mar 2020 17:02:48.450 [89592] ERROR #splapptrc,J[5],P[62],updatedStreamSR,spl_javaop M[?:?:0] - com.ibm.streams.operator.internal.ports.SwitchingHandler$Alternate.tuple(SwitchingHandler.java:162) 30 Mar 2020 17:02:48.450 [89592] ERROR #splapptrc,J[5],P[62],updatedStreamSR,spl_javaop M[?:?:0] - com.ibm.streams.operator.internal.network.DeserializingStream.tuple(DeserializingStream.java:76) 30 Mar 2020 17:02:48.451 [89592] ERROR #splapptrc,J[5],P[62],updatedStreamSR,spl_javaop M[?:?:0] - com.ibm.streams.operator.internal.network.DeserializingStream.processRawTuple(DeserializingStream.java:65) 30 Mar 2020 17:02:48.451 [89592] ERROR #splapptrc,J[5],P[62],updatedStreamSR,spl_javaop M[?:?:0] - com.ibm.streams.operator.internal.runtime.api.InputPortsConduit.processRawTuple(InputPortsConduit.java:100) 30 Mar 2020 17:02:48.451 [89592] ERROR #splapptrc,J[5],P[62],updatedStreamSR,spl_javaop M[?:?:0] - com.ibm.streams.operator.internal.jni.JNIBridge.process(JNIBridge.java:316) 30 Mar 2020 17:02:48.454 [89592] ERROR #splapptrc,J[5],P[62],updatedStreamSR,spl_function M[JavaOp.cpp:processTuple:1592] - CDISR5126E: A Java exception occurred at Operator.process() during the JavaOp invocation. 30 Mar 2020 17:02:48.476 [89592] DEBUG spl_pe M[PEImpl.cpp:handleOperatorFailure:618] - Exception received during onMessage(): A Java exception occurred at Operator.process() during the JavaOp invocation. 30 Mar 2020 17:02:48.476 [89592] ERROR #splapptrc,J[5],P[62],updatedStreamSR,#splapptrc,J[5],P[62],updatedStreamSR,spl_operator M[PEImpl.cpp:handleOperatorFailure:619] - CDISR5030E: An exception occurred during the execution of the updatedStreamSR operator. The exception is: A Java exception occurred at Operator.process() during the JavaOp invocation. 30 Mar 2020 17:02:48.477 [89592] INFO spl_pe M[PEImpl.cpp:shutdown:1605] - Shutdown request received by PE... 30 Mar 2020 17:02:48.477 [89592] INFO spl_pe M[PEImpl.cpp:shutdown:1613] - isJobCancellation is set to 0 30 Mar 2020 17:02:48.477 [89592] INFO spl_pe M[PEImpl.cpp:shutdown:1620] - shutdownRequested set to true... 30 Mar 2020 17:02:48.478 [89592] DEBUG spl_pe M[PEImpl.cpp:preparePortsForTermination:1441] - Shutting down PE ports... 30 Mar 2020 17:02:48.963 [89592] DEBUG spl_pe M[PEImpl.cpp:preparePortsForTermination:1444] - Completed shutting down PE ports.

anouri commented 4 years ago

I have checked your logs. The problem is one SQL error. SQL Error: SQLCODE=-302, SQLSTATE=22001, SQLERRMC=null, DRIVER=3.67.26 The issue may be due to a spelling error or an incorrect datatype or size. https://www.ibm.com/mysupport/s/question/0D50z000062kxIbCAI/resolve-db2-sqlcode302-sqlstate22001-in-jdbc-gateway-log-file?language=en_US

This error stops the running PE.

Please use the latest version of streamsx.jdbc https://github.com/IBMStreams/streamsx.jdbc/releases/tag/v1.8.0

The JDBCRun operator has two output ports, The first port is for SQL results and the secund optional port is to get details SQL error code and SQL message. Please define the secund optional port and check details about SQL error

(stream<rsSchema> runSql ; 
stream<tuple<insertSchema> inTuple, JdbcSqlStatus_T error> errors) = JDBCRun(pulse)
        {
.....

} More details in: https://github.com/IBMStreams/streamsx.jdbc/wiki/How-to-connect-to-a-DB2-database-via-JDBC-toolkit

anouri commented 4 years ago

corrected in release version 1.8.0 https://github.com/IBMStreams/streamsx.jdbc/releases/tag/v1.8.0