search

verdict-project / verdict

Interactive-Speed Analytics: 200x Faster, 200x Fewer Cluster Resources, Approximate Query Processing
http://verdictdb.org
Apache License 2.0
248 stars 66 forks source link

Communication failure error for large table #226

Closed pyongjoo closed 6 years ago

pyongjoo commented 6 years ago

The error occurred for MySQL. It looks like the JDBC communications automatically close after a while. I have tried adding this option &autoReconnect=true&failOverReadOnly=false&maxReconnects=10 in JDBC connection string, but the error still occured.

org.verdictdb.exception.VerdictDBDbmsException: Issued the following query: create schema if not exists verdictdbmeta
Communications link failure

The last packet successfully received from the server was 3,104,474 milliseconds ago.  The last packet sent successfully to the server was 13 milliseconds ago.
        at org.verdictdb.connection.JdbcConnection.executeSingle(JdbcConnection.java:229)
        at org.verdictdb.connection.JdbcConnection.execute(JdbcConnection.java:135)
        at org.verdictdb.connection.ConcurrentJdbcConnection.execute(ConcurrentJdbcConnection.java:128)
        at org.verdictdb.connection.CachedDbmsConnection.execute(CachedDbmsConnection.java:45)
        at org.verdictdb.metastore.ScrambleMetaStore.addToStore(ScrambleMetaStore.java:145)
        at org.verdictdb.metastore.ScrambleMetaStore.addToStore(ScrambleMetaStore.java:117)
        at org.verdictdb.coordinator.ExecutionContext.streamsql(ExecutionContext.java:192)
        at org.verdictdb.coordinator.ExecutionContext.sql(ExecutionContext.java:119)
        at org.verdictdb.VerdictContext.sql(VerdictContext.java:290)
        at org.verdictdb.PrepareVerdictDB.main(PrepareVerdictDB.java:22)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:282)
        at java.lang.Thread.run(Thread.java:748)
pyongjoo commented 6 years ago

I will use a single JDBC connection (instead of a pool of them) for creating scrambles.