search

schemacrawler / SchemaCrawler

Free database schema discovery and comprehension tool
http://www.schemacrawler.com/
Other
1.6k stars 199 forks source link

Null Pointer Exception when crawling schema for Sybase #67

Closed kvallish closed 8 years ago

kvallish commented 8 years ago

Following #66 when schema crawler tries to get the table details from my sybase database, am hitting the below exception:

May 05, 2016 10:21:15 AM us.fatehi.commandlineparser.CommandLineUtility logFullStackTrace SEVERE: Exception retrieving table information: null schemacrawler.schemacrawler.SchemaCrawlerException: Exception retrieving table information: null at schemacrawler.crawl.SchemaCrawler.crawlTables(SchemaCrawler.java:708) at schemacrawler.crawl.SchemaCrawler.crawl(SchemaCrawler.java:766) at schemacrawler.tools.executable.BaseStagedExecutable.execute(BaseStagedExecutable.java:92) at schemacrawler.tools.commandline.SchemaCrawlerCommandLine.execute(SchemaCrawlerCommandLine.java:133) at schemacrawler.Main.main(Main.java:88) Caused by: java.lang.NullPointerException at schemacrawler.filter.TableTypesFilter.test(TableTypesFilter.java:107) at schemacrawler.filter.TableTypesFilter.test(TableTypesFilter.java:39) at java.util.function.Predicate.lambda$and$0(Unknown Source) at java.util.function.Predicate.lambda$and$0(Unknown Source) at schemacrawler.crawl.TablesReducer.doReduce(TablesReducer.java:78) at schemacrawler.crawl.TablesReducer.reduce(TablesReducer.java:67) at schemacrawler.crawl.MutableCatalog.reduce(MutableCatalog.java:389) at schemacrawler.crawl.SchemaCrawler.lambda$crawlTables$29(SchemaCrawler.java:594) at sf.util.StopWatch.time(StopWatch.java:156) at schemacrawler.crawl.SchemaCrawler.crawlTables(SchemaCrawler.java:590) ... 4 more

schemacrawler commented 8 years ago

I'll take a look.

schemacrawler commented 8 years ago

I am unable to set up a Sybase server easily. Still trying.

fadein commented 8 years ago

@sualeh, good news! I am running into the exact same error as @kvallish, but with a MySQL database. My stacktrace looks the same as @kvallish's except the line numbers are a bit different, so I may be using a different version of SchemaCrawler than they are.

I wonder if the problem may come down to a JVM or Java language issue?

Here is my info: The command I'm trying to run

C:\Users\erikf\Desktop\schemacrawler-14.08.06-main\_schemacrawler>schemacrawler.cmd -user=root -password= -server=mysql  -infolevel=minimum -loglevel=CONFIG -command=list -url=jdbc:mysql://192.168.56.102:3306/mysql

The resulting exception

Jun 23, 2016 9:10:09 AM us.fatehi.commandlineparser.CommandLineUtility logFullStackTrace
SEVERE: Exception retrieving table information: null
schemacrawler.schemacrawler.SchemaCrawlerException: Exception retrieving table information: null
        at schemacrawler.crawl.SchemaCrawler.crawlTables(SchemaCrawler.java:739)
        at schemacrawler.crawl.SchemaCrawler.crawl(SchemaCrawler.java:797)
        at schemacrawler.tools.executable.BaseStagedExecutable.execute(BaseStagedExecutable.java:92)
        at schemacrawler.tools.commandline.SchemaCrawlerCommandLine.execute(SchemaCrawlerCommandLine.java:133)
        at schemacrawler.Main.main(Main.java:91)
Caused by: java.lang.NullPointerException
        at schemacrawler.filter.TableTypesFilter.test(TableTypesFilter.java:107)
        at schemacrawler.filter.TableTypesFilter.test(TableTypesFilter.java:39)
        at java.util.function.Predicate.lambda$and$0(Unknown Source)
        at java.util.function.Predicate.lambda$and$0(Unknown Source)
        at schemacrawler.crawl.TablesReducer.doReduce(TablesReducer.java:78)
        at schemacrawler.crawl.TablesReducer.reduce(TablesReducer.java:67)
        at schemacrawler.crawl.MutableCatalog.reduce(MutableCatalog.java:389)
        at schemacrawler.crawl.SchemaCrawler.lambda$crawlTables$29(SchemaCrawler.java:610)
        at sf.util.StopWatch.time(StopWatch.java:156)
        at schemacrawler.crawl.SchemaCrawler.crawlTables(SchemaCrawler.java:606)
        ... 4 more

SchemaCrawler version 14.08.06

Java & JVM

java version "1.8.0_91"
Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
Java HotSpot(TM) Client VM (build 25.91-b14, mixed mode, sharing)

Client-side operating system Windows 7 Professional, SP1 Microsoft Windows [Version 6.1.7601]

Database Server MySQL Server version: 5.5.44-0ubuntu0.14.04.1 (Ubuntu)

schemacrawler commented 8 years ago

Thanks, @fadein - this will be easier for me to reproduce.

SpyroTEQ commented 8 years ago

Meh, I got the exact same issue when trying to draw the diagram of a MySQL database. Same lines

schemacrawler.cmd -server mysql -host=localhost -port=3306 -user=### -password=### -database=### -tables=.*\.ctc_.* -infolevel=minimum -command=graph -outputformat=png -outputfile=graph.png -schemas=### -loglevel=CONFIG

MySQL 5.7.10 (so probably not a DB-related issue) Windows 7 SP1, java 1.8.0_92 No issue with the diagram sample (so maybe a feature of MySQL not supported in SchemaCrawler?)

schemacrawler commented 8 years ago

This week, I am going to try to reproduce and investigate the source of the problem. Meanwhile, if someone can give me a test schema, that will help me get there faster. Thanks.

fadein commented 8 years ago

I have just reproduced the issue with this Northwind database for MySQL in the exact same environment as in my 1st report above. I hope that this is helpful to you.

This is the output which resulted:

C:\Users\erikf\Desktop\schemacrawler-14.08.06-main_schemacrawler>schemacrawler.cmd -user=root -password= -server=mysql -infolevel=minimum -loglevel=CONFIG -command=list -url=jdbc:mysql://192.168.56.102/northwind INFO: SchemaCrawler, v14.08.06 Jun 27, 2016 9:51:55 AM us.fatehi.commandlineparser.CommandLineUtility logSafeAr guments INFO: Command line: -user=root -password=***** -server=mysql -infolevel=minimum -loglevel=CONFIG -command=list -url=jdbc:mysql://192.168.56.102/northwind Jun 27, 2016 9:51:55 AM us.fatehi.commandlineparser.CommandLineUtility logSystem Properties CONFIG: System properties: java.awt.graphicsenv=sun.awt.Win32GraphicsEnvironment java.awt.printerjob=sun.awt.windows.WPrinterJob java.class.version=52.0 java.endorsed.dirs=C:\Program Files (x86)\Java\jre8\lib\endorsed java.ext.dirs=C:\Program Files (x86)\Java\jre8\lib\ext;C:\Windows\Sun\Java\lib\e xt java.home=C:\Program Files (x86)\Java\jre8 java.io.tmpdir=C:\Users\erikf\AppData\Local\Temp\ java.runtime.name=Java(TM) SE Runtime Environment java.runtime.version=1.8.0_91-b14 java.specification.name=Java Platform API Specification java.specification.vendor=Oracle Corporation java.specification.version=1.8 java.vendor=Oracle Corporation java.vendor.url=http://java.oracle.com/ java.vendor.url.bug=http://bugreport.sun.com/bugreport/ java.version=1.8.0_91 java.vm.info=mixed mode, sharing java.vm.name=Java HotSpot(TM) Client VM java.vm.specification.name=Java Virtual Machine Specification java.vm.specification.vendor=Oracle Corporation java.vm.specification.version=1.8 java.vm.vendor=Oracle Corporation java.vm.version=25.91-b14 os.arch=x86 os.name=Windows 7 os.version=6.1 Jun 27, 2016 9:51:55 AM us.fatehi.commandlineparser.CommandLineUtility logSystem Properties CONFIG: Classpath: lib/h2-1.4.192.jar lib/hsqldb-2.3.4.jar lib/jtds-1.3.1.jar lib/mariadb-java-client-1.4.5.jar lib/mysql-connector-java-6.0.2.jar lib/postgresql-9.4.1208.jar lib/schemacrawler-14.08.06.jar lib/schemacrawler-api-14.08.06-tests.jar lib/schemacrawler-db2-14.08.06.jar lib/schemacrawler-h2-14.08.06.jar lib/schemacrawler-hsqldb-14.08.06.jar lib/schemacrawler-lint-14.08.06.jar lib/schemacrawler-mariadb-14.08.06.jar lib/schemacrawler-mysql-14.08.06.jar lib/schemacrawler-offline-14.08.06.jar lib/schemacrawler-oracle-14.08.06.jar lib/schemacrawler-postgresql-14.08.06.jar lib/schemacrawler-sqlite-14.08.06.jar lib/schemacrawler-sqlserver-14.08.06.jar lib/schemacrawler-sybaseiq-14.08.06.jar lib/sqlite-jdbc-3.7.8.jar lib/xmlpull-1.1.3.1.jar lib/xpp3_min-1.1.4c.jar lib/xstream-1.4.9.jar . Jun 27, 2016 9:51:55 AM schemacrawler.tools.databaseconnector.DatabaseConnectorR egistry loadDatabaseConnectorRegistry CONFIG: Loading database connector, db2=schemacrawler.server.db2.DB2DatabaseConn ector Jun 27, 2016 9:51:55 AM schemacrawler.tools.databaseconnector.DatabaseConnectorR egistry loadDatabaseConnectorRegistry CONFIG: Loading database connector, h2=schemacrawler.server.h2.H2DatabaseConnect or Jun 27, 2016 9:51:55 AM schemacrawler.tools.databaseconnector.DatabaseConnectorR egistry loadDatabaseConnectorRegistry CONFIG: Loading database connector, hsqldb=schemacrawler.server.hsqldb.HyperSQLD atabaseConnector Jun 27, 2016 9:51:55 AM schemacrawler.tools.databaseconnector.DatabaseConnectorR egistry loadDatabaseConnectorRegistry CONFIG: Loading database connector, mariadb=schemacrawler.server.mariadb.MariaDB DatabaseConnector Jun 27, 2016 9:51:55 AM schemacrawler.tools.databaseconnector.DatabaseConnectorR egistry loadDatabaseConnectorRegistry CONFIG: Loading database connector, mysql=schemacrawler.server.mysql.MySQLDataba seConnector Jun 27, 2016 9:51:55 AM schemacrawler.tools.databaseconnector.DatabaseConnectorR egistry loadDatabaseConnectorRegistry CONFIG: Loading database connector, offline=schemacrawler.tools.offline.OfflineD atabaseConnector Jun 27, 2016 9:51:55 AM schemacrawler.tools.databaseconnector.DatabaseConnectorR egistry loadDatabaseConnectorRegistry CONFIG: Loading database connector, oracle=schemacrawler.server.oracle.OracleDat abaseConnector Jun 27, 2016 9:51:55 AM schemacrawler.tools.databaseconnector.DatabaseConnectorR egistry loadDatabaseConnectorRegistry CONFIG: Loading database connector, postgresql=schemacrawler.server.postgresql.P ostgreSQLDatabaseConnector Jun 27, 2016 9:51:55 AM schemacrawler.tools.databaseconnector.DatabaseConnectorR egistry loadDatabaseConnectorRegistry CONFIG: Loading database connector, sqlite=schemacrawler.tools.sqlite.SQLiteData baseConnector Jun 27, 2016 9:51:55 AM schemacrawler.tools.databaseconnector.DatabaseConnectorR egistry loadDatabaseConnectorRegistry CONFIG: Loading database connector, sqlserver=schemacrawler.server.sqlserver.Sql ServerDatabaseConnector Jun 27, 2016 9:51:55 AM schemacrawler.tools.databaseconnector.DatabaseConnectorR egistry loadDatabaseConnectorRegistry CONFIG: Loading database connector, sybaseiq=schemacrawler.server.sybaseiq.Sybas eIQDatabaseConnector Jun 27, 2016 9:51:55 AM schemacrawler.tools.databaseconnector.DatabaseConnectorR egistry logRegisteredJdbcDrivers CONFIG: Registered JDBC drivers, [com.mysql.cj.jdbc.Driver 6.0, net.sourceforge. jtds.jdbc.Driver 1.3, org.h2.Driver 1.4, org.hsqldb.jdbc.JDBCDriver 2.3, org.mar iadb.jdbc.Driver 1.4, org.postgresql.Driver 9.4, org.sqlite.JDBC 3.7, schemacraw ler.tools.offline.jdbc.OfflineDriver 0.0] Jun 27, 2016 9:51:55 AM schemacrawler.tools.commandline.SchemaCrawlerCommandLine

INFO: Using database plugin, mysql - MySQL Jun 27, 2016 9:51:55 AM schemacrawler.schemacrawler.Config loadProperties CONFIG: Cannot load properties from file, C:\Users\erikf\Desktop\schemacrawler-1 4.08.06-main_schemacrawler\schemacrawler.config.properties Jun 27, 2016 9:51:55 AM schemacrawler.schemacrawler.Config loadProperties CONFIG: Cannot load properties from file, C:\Users\erikf\Desktop\schemacrawler-1 4.08.06-main_schemacrawler\schemacrawler.additional.config.properties Jun 27, 2016 9:51:55 AM schemacrawler.tools.commandline.SchemaCrawlerOptionsPars er getOptions WARNING: Please provide a -schemas option for efficient retrieval of database me tadata Jun 27, 2016 9:51:55 AM schemacrawler.schemacrawler.BaseDatabaseConnectionOption s getConnection WARNING: Database password is not provided Jun 27, 2016 9:51:55 AM schemacrawler.schemacrawler.BaseDatabaseConnectionOption s getConnection INFO: Making connection to jdbc:mysql://192.168.56.102/northwind for user 'root', with properties {} Jun 27, 2016 9:51:55 AM schemacrawler.schemacrawler.BaseDatabaseConnectionOption s getConnection INFO: Opened database connection, org.mariadb.jdbc.MariaDbConnection@33ce2c Jun 27, 2016 9:51:55 AM schemacrawler.schemacrawler.BaseDatabaseConnectionOption s logConnection INFO: Connected to MySQL 5.5.44-0ubuntu0.14.04.1 using JDBC driver MariaDB connector/J 1.4.5 Jun 27, 2016 9:51:55 AM schemacrawler.tools.executable.BaseStagedExecutable exec ute INFO: Executing SchemaCrawler command, "list" Jun 27, 2016 9:51:55 AM schemacrawler.tools.executable.BaseStagedExecutable exec ute CONFIG: schemacrawler.schemacrawler.SchemaCrawlerOptions@1851384[ childTableFilterDepth: 0 columnInclusionRule: IncludeAll grepColumnInclusionRule: null grepDefinitionInclusionRule: null grepInvertMatch: false grepOnlyMatching: false grepRoutineColumnInclusionRule: null hideEmptyTables: false parentTableFilterDepth: 0 routineColumnInclusionRule: IncludeAll routineInclusionRule: IncludeAll routineTypes: [procedure, function] schemaInclusionRule: IncludeAll schemaInfoLevel: minimum sequenceInclusionRule: RegularExpressionRule@5a99da-include//-exclude/._/] synonymInclusionRule: RegularExpressionRule@1f37ef1-include//-exclude/._/] tableInclusionRule: IncludeAll tableNamePattern: null tableTypes: [BASE TABLE, TABLE, VIEW] title: ] Jun 27, 2016 9:51:55 AM schemacrawler.tools.executable.BaseStagedExecutable exec ute CONFIG: schemacrawler.tools.options.OutputOptions@1ccaa[ inputEncodingCharset: UTF-8 inputResource: null outputEncodingCharset: UTF-8 outputFormatValue: text outputResource: null ] Jun 27, 2016 9:51:55 AM schemacrawler.crawl.RetrieverConnection CONFIG: Database supports catalogs Jun 27, 2016 9:51:55 AM schemacrawler.crawl.RetrieverConnection CONFIG: Database does not support schemas Jun 27, 2016 9:51:55 AM schemacrawler.crawl.RetrieverConnection CONFIG: Database identifier quote string is "`" Jun 27, 2016 9:51:55 AM schemacrawler.crawl.RetrieverConnection CONFIG: Supported table types are [base table, system view, view] Jun 27, 2016 9:51:55 AM schemacrawler.crawl.SchemaCrawler crawlSchemas INFO: Crawling schemas Jun 27, 2016 9:51:55 AM schemacrawler.crawl.SchemaRetriever retrieveAllSchemas INFO: Retrieving all schemas Jun 27, 2016 9:51:55 AM schemacrawler.crawl.SchemaRetriever retrieveAllCatalogs INFO: Retrieving all catalogs Jun 27, 2016 9:51:55 AM schemacrawler.crawl.SchemaCrawler crawlSchemas INFO: Total time taken for "crawlSchemas" - 00:00:00.008 hours - 0.0% - 00:00:00.007 - "retrieveSchemas" - 0.0% - 00:00:00.001 - "sortAndFilterSchemas" Jun 27, 2016 9:51:55 AM schemacrawler.crawl.SchemaCrawler crawlDatabaseInfo INFO: Crawling SchemaCrawler information Jun 27, 2016 9:51:55 AM schemacrawler.crawl.SchemaCrawler crawlDatabaseInfo INFO: Retrieving database information Jun 27, 2016 9:51:55 AM schemacrawler.crawl.SchemaCrawler lambda$crawlDatabaseIn fo$12 INFO: Not retrieving additional database information, since this was not request ed Jun 27, 2016 9:51:55 AM schemacrawler.crawl.SchemaCrawler crawlDatabaseInfo INFO: Retrieving JDBC driver information Jun 27, 2016 9:51:55 AM schemacrawler.crawl.SchemaCrawler lambda$crawlDatabaseIn fo$14 INFO: Not retrieving additional JDBC driver information, since this was not requ ested Jun 27, 2016 9:51:55 AM schemacrawler.crawl.SchemaCrawler crawlDatabaseInfo INFO: Retrieving SchemaCrawler crawl information Jun 27, 2016 9:51:55 AM schemacrawler.crawl.SchemaCrawler crawlDatabaseInfo INFO: Total time taken for "crawlDatabaseInfo" - 00:00:00.037 hours - 0.0% - 00:00:00.000 - "retrieveDatabaseInfo" - 0.0% - 00:00:00.000 - "retrieveAdditionalDatabaseInfo" - 0.0% - 00:00:00.000 - "retrieveJdbcDriverInfo" - 0.0% - 00:00:00.001 - "retrieveAdditionalJdbcDriverInfo" - 0.0% - 00:00:00.036 - "retrieveCrawlHeaderInfo" Jun 27, 2016 9:51:55 AM schemacrawler.crawl.SchemaCrawler crawlColumnDataTypes INFO: Crawling column data types Jun 27, 2016 9:51:55 AM schemacrawler.crawl.SchemaCrawler lambda$crawlColumnData Types$9 INFO: Not retrieving system column data types, since this was not requested Jun 27, 2016 9:51:55 AM schemacrawler.crawl.SchemaCrawler lambda$crawlColumnData Types$10 INFO: Not retrieving user column data types, since this was not requested Jun 27, 2016 9:51:55 AM schemacrawler.crawl.SchemaCrawler crawlColumnDataTypes INFO: Total time taken for "crawlColumnDataTypes" - 00:00:00.003 hours - 0.0% - 00:00:00.002 - "retrieveSystemColumnDataTypes" - 0.0% - 00:00:00.001 - "retrieveUserDefinedColumnDataTypes" Jun 27, 2016 9:51:55 AM schemacrawler.crawl.SchemaCrawler crawlTables INFO: Crawling tables Jun 27, 2016 9:51:55 AM schemacrawler.crawl.TableRetriever retrieveTables INFO: Retrieving tables for schema: information_schema Jun 27, 2016 9:51:55 AM schemacrawler.crawl.MetadataResultSet close INFO: "retrieveTables" results had 0 rows Jun 27, 2016 9:51:55 AM schemacrawler.crawl.TableRetriever retrieveTables INFO: Retrieving tables for schema: loe Jun 27, 2016 9:51:55 AM schemacrawler.crawl.MetadataResultSet close INFO: "retrieveTables" results had 58 rows Jun 27, 2016 9:51:55 AM schemacrawler.crawl.TableRetriever retrieveTables INFO: Retrieving tables for schema: mysql Jun 27, 2016 9:51:55 AM schemacrawler.crawl.MetadataResultSet close INFO: "retrieveTables" results had 24 rows Jun 27, 2016 9:51:55 AM schemacrawler.crawl.TableRetriever retrieveTables INFO: Retrieving tables for schema: northwind Jun 27, 2016 9:51:55 AM schemacrawler.crawl.MetadataResultSet close INFO: "retrieveTables" results had 29 rows Jun 27, 2016 9:51:55 AM schemacrawler.crawl.TableRetriever retrieveTables INFO: Retrieving tables for schema: performance_schema Jun 27, 2016 9:51:55 AM schemacrawler.crawl.MetadataResultSet close INFO: "retrieveTables" results had 17 rows Jun 27, 2016 9:51:55 AM schemacrawler.crawl.SchemaCrawler lambda$crawlTables$28 WARNING: Foreign-keys are not being retrieved, so tables cannot be sorted using the natural sort order SchemaCrawler 14.08.06 Error: Exception retrieving table information: null Re-run SchemaCrawler with just the -? option for help Or, re-run SchemaCrawler with an additional -loglevel=CONFIG option for details on the error Jun 27, 2016 9:51:56 AM us.fatehi.commandlineparser.CommandLineUtility logSafeAr guments INFO: SchemaCrawler, v14.08.06 Jun 27, 2016 9:51:56 AM us.fatehi.commandlineparser.CommandLineUtility logSafeAr guments INFO: Command line: -user=root -password=***** -server=mysql -infolevel=minimum -loglevel=CONFIG -command=list -url=jdbc:mysql://192.168.56.102/northwind Jun 27, 2016 9:51:56 AM us.fatehi.commandlineparser.CommandLineUtility logFullSt ackTrace SEVERE: Exception retrieving table information: null schemacrawler.schemacrawler.SchemaCrawlerException: Exception retrieving table i nformation: null at schemacrawler.crawl.SchemaCrawler.crawlTables(SchemaCrawler.java:739) ``` at schemacrawler.crawl.SchemaCrawler.crawl(SchemaCrawler.java:797) at schemacrawler.tools.executable.BaseStagedExecutable.execute(BaseStage ``` dExecutable.java:92) at schemacrawler.tools.commandline.SchemaCrawlerCommandLine.execute(Sche maCrawlerCommandLine.java:133) at schemacrawler.Main.main(Main.java:91) Caused by: java.lang.NullPointerException at schemacrawler.filter.TableTypesFilter.test(TableTypesFilter.java:107) ``` at schemacrawler.filter.TableTypesFilter.test(TableTypesFilter.java:39) at java.util.function.Predicate.lambda$and$0(Unknown Source) at java.util.function.Predicate.lambda$and$0(Unknown Source) at schemacrawler.crawl.TablesReducer.doReduce(TablesReducer.java:78) at schemacrawler.crawl.TablesReducer.reduce(TablesReducer.java:67) at schemacrawler.crawl.MutableCatalog.reduce(MutableCatalog.java:389) at schemacrawler.crawl.SchemaCrawler.lambda$crawlTables$29(SchemaCrawler ``` .java:610) at sf.util.StopWatch.time(StopWatch.java:156) at schemacrawler.crawl.SchemaCrawler.crawlTables(SchemaCrawler.java:606) ``` ... 4 more ```
schemacrawler commented 8 years ago

Erik, @SpyroTEQ - to resolve the MySql issue, please delete the mariadb-java-client-1.4.5.jar from the lib folder. I am still investigating the Sybase issue.

schemacrawler commented 8 years ago

Release 14.09.01 of SchemaCrawler will address this issue. SchemaCrawler will no longer be distributed with a MariaDB JDBC driver for MySQL, since that interferes with MySQL processing. Also, the MySQL JDBC URL has been fixed for backward compatibility. The null pointer issue with Sybase has been addressed.