timeseries / qstudio

qStudio - Free SQL Analysis Tool
https://www.timestored.com/qstudio/
Apache License 2.0
416 stars 17 forks source link

DolphinDB update driver #22

Closed ryanhamilton closed 1 month ago

ryanhamilton commented 5 months ago

qStudio relies on databases to supply JDBC compliant standard results. Most of the issues below are areas where either the specification is not met or a slightly incorrect result is returned.

Issues:

  1. Server tree doesn't work (stack trace below).
  2. Timestamp column reported as type=12 VARCHAR. SO time series graph doesn't work (screenshot below).
  3. Selecting anything but a table displays nothing.
  4. Some types don't work.
java.lang.RuntimeException: java.io.IOException: localhost:8848 Server response: 'select * from schema(mod)."colDefs" where name == '%' => The input for schema function must be a table or a database.' script: 'select * from schema(%).colDefs where name = '%';'
    at com.dolphindb.jdbc.JDBCDataBaseMetaData.getColumnsOriginMetaData(JDBCDataBaseMetaData.java:160)
    at com.dolphindb.jdbc.JDBCDataBaseMetaData.getColumns(JDBCDataBaseMetaData.java:116)
    at org.apache.commons.dbcp2.DelegatingDatabaseMetaData.getColumns(DelegatingDatabaseMetaData.java:226)
    at com.timestored.connections.MetaInfo.lambda$1(MetaInfo.java:121)

image

Recommended Actions for DolphinDB

Testing new Versions:

  1. Run qStudio against dolphindb as shown here https://www.timestored.com/qstudio/database/dolphindb This will download jdbc-1.XXX to C:\Users{{username}}\qStudio\libs on windows.
  2. Close qStudio
  3. To test a new jar, remove the 1.XX and replace with jdbc-2.00.11.1-jar-with-dependencies

Items to Test

  1. Get server tree working. This relies on conn.getMetaData().getColumns(catalog, schemaPattern, tableName, columnName) . The result MUST contain result of format: TABLE_CAT:`,TABLE_SCHEM:`,TABLE_NAME:n,COLUMN_NAME:c,DATA_TYPE:0i,TYPE_NAME:`int$t
  2. Check possible all date/time types report as DATE/TIME/TIMESTAMP , jdbc types = 91/92/93. (https://docs.oracle.com/javase/8/docs/api/constant-values.html#java.sql.Types.DATE).

Other Recommendation - Non-tabular data.

To render non-tabular data either qStudio needs to understand the result of each database OR the JDBC driver must convert non-tabular results to a ResultSet. Currently dolphindb demand is not enough to justify adding the 1.4MB dependency for DolphinDB therefore ideally DolphinDB would convert non-tabular results to result sets. In particular I would recommend this for lists and dictionaries.

Example of converting kdb dictionaries / lists to resultsets:


 // @Ryan added to improve display of kdb data structures in Pulse
 private static ResultSet convertToTable(Object k) {
         if(k instanceof c.Dict) {
             c.Dict d = ((c.Dict)k);
             int rows = c.n(k);
             String[] keys = new String[rows];
             String[] vals = new String[rows];
             for(int i=0; i<rows; i++) {
                 keys[i] = KdbHelper.asLine(c.at(d.x, i), true);
                 vals[i] = KdbHelper.asLine(c.at(d.y, i), true);
             }
             return new SimpleResultSet(new String[] {"keys","vals"}, new Object[] {keys, vals});
         } else if(k!=null && k.getClass().isArray()) {
             int rows = c.n(k);
             String[] vals = new String[rows];
             for(int i=0; i<rows; i++) {
                 vals[i] = KdbHelper.asLine(c.at(k, i), true);
             }
             return new SimpleResultSet(new String[] {"vals"}, new Object[] {vals});
         } else {
             Object[] vals = new Object[] { new String[] { KdbHelper.asLine(k) }};
             return new SimpleResultSet(new String[] {"val"}, vals);
         }
 }
sunnyinchina commented 4 months ago

how to update?

ryanhamilton commented 4 months ago

I have contacted DolphinDB for assistance. Tested and reported issues with jdbc-2.00.11.1-jar-with-dependencies.jar

ryanhamilton commented 3 months ago

md.getColumns(catalog, schemaPattern, tableName, columnName); catalog = null schemaPattern = tableName = columnName = "%"

ryanhamilton commented 1 month ago

Fixed in 3.03