search

OHDSI / MedlineXmlToDatabase

A command line Java application for parsing MEDLINE XML files and inserting the data into a relational database
Apache License 2.0
19 stars 11 forks source link

error when 'analyse': PSQLException #12

Closed lemonysoft closed 5 years ago

lemonysoft commented 5 years ago

Hello. I have problem when using 'analyse'. How can I overcome?

Processing pubmed19n0972.xml.gz Creating tables create table medcit_keywordlist_keyword ( value text, majortopicyn varchar(255), medcit_keywordlist_keyword_order int, medcit_keywordlist_order int, pmid int, pmid_version int, mml:math_id varchar(255), mml:math_mml:msub_xmlns:mml varchar(255), mml:math_overflow varchar(255), mml:math_xmlns:mml varchar(255), primary key (pmid,pmid_version,medcit_keywordlist_keyword_order,medcit_keywordlist_order) );

org.postgresql.util.PSQLException: ERROR: syntax error at or near ":" Position: 195 at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:559) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:403) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:395) at org.ohdsi.databases.ConnectionWrapper.execute(ConnectionWrapper.java:111) at org.ohdsi.databases.ConnectionWrapper.createTable(ConnectionWrapper.java:222) at org.ohdsi.databases.ConnectionWrapper.createTableUsingVariableTypes(ConnectionWrapper.java:259) at org.ohdsi.medlineXmlToDatabase.MedlineCitationAnalyser.createTables(MedlineCitationAnalyser.java:154) at org.ohdsi.medlineXmlToDatabase.MedlineAnalyserMain.createDatabase(MedlineAnalyserMain.java:76) at org.ohdsi.medlineXmlToDatabase.MedlineAnalyserMain.main(MedlineAnalyserMain.java:45) at org.ohdsi.medlineXmlToDatabase.MainClass.main(MainClass.java:41) Exception in thread "main" java.lang.RuntimeException: Error inserting data at org.ohdsi.databases.ConnectionWrapper.execute(ConnectionWrapper.java:122) at org.ohdsi.databases.ConnectionWrapper.createTable(ConnectionWrapper.java:222) at org.ohdsi.databases.ConnectionWrapper.createTableUsingVariableTypes(ConnectionWrapper.java:259) at org.ohdsi.medlineXmlToDatabase.MedlineCitationAnalyser.createTables(MedlineCitationAnalyser.java:154) at org.ohdsi.medlineXmlToDatabase.MedlineAnalyserMain.createDatabase(MedlineAnalyserMain.java:76) at org.ohdsi.medlineXmlToDatabase.MedlineAnalyserMain.main(MedlineAnalyserMain.java:45) at org.ohdsi.medlineXmlToDatabase.MainClass.main(MainClass.java:41)

schuemie commented 5 years ago

This sounds like the problem that was fixed here. Can you confirm that you're running the latest version?

lemonysoft commented 5 years ago

Sorry for disturbing you. It caused by our cluster framework. Now it works very well.

lemonysoft commented 5 years ago

Is it related to server-side?

Processing pubmed19n0316.xml.gz Processing pubmed19n0317.xml.gz Processing pubmed19n0318.xml.gz Processing pubmed19n0319.xml.gz Processing pubmed19n0320.xml.gz Processing pubmed19n0321.xml.gz Processing pubmed19n0322.xml.gz java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at java.net.Socket.connect(Socket.java:538) at sun.net.NetworkClient.doConnect(NetworkClient.java:180) at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:840) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1587) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492) at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:647) at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.startEntity(XMLEntityManager.java:1304) at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.startEntity(XMLEntityManager.java:1240) at com.sun.org.apache.xerces.internal.impl.XMLDTDScannerImpl.startPE(XMLDTDScannerImpl.java:741) at com.sun.org.apache.xerces.internal.impl.XMLDTDScannerImpl.skipSeparator(XMLDTDScannerImpl.java:2110) at com.sun.org.apache.xerces.internal.impl.XMLDTDScannerImpl.scanDecls(XMLDTDScannerImpl.java:2073) at com.sun.org.apache.xerces.internal.impl.XMLDTDScannerImpl.scanDTDExternalSubset(XMLDTDScannerImpl.java:306) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$DTDDriver.dispatch(XMLDocumentScannerImpl.java:1174) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$DTDDriver.next(XMLDocumentScannerImpl.java:1045) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:959) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:842) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771) at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:243) at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:339) at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:121) at org.ohdsi.medlineXmlToDatabase.XMLFileIterator$DecompressAndParseThread.process(XMLFileIterator.java:130) at org.ohdsi.utilities.concurrency.BatchProcessingThread.run(BatchProcessingThread.java:42)

lemonysoft commented 5 years ago

Processing pubmed19n0327.xml.gz java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:210) at java.net.SocketInputStream.read(SocketInputStream.java:141) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678) at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:848) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1587) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492) at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:647) at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.startEntity(XMLEntityManager.java:1304) at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.startEntity(XMLEntityManager.java:1240) at com.sun.org.apache.xerces.internal.impl.XMLDTDScannerImpl.startPE(XMLDTDScannerImpl.java:741) at com.sun.org.apache.xerces.internal.impl.XMLDTDScannerImpl.skipSeparator(XMLDTDScannerImpl.java:2110) at com.sun.org.apache.xerces.internal.impl.XMLDTDScannerImpl.scanDecls(XMLDTDScannerImpl.java:2073) at com.sun.org.apache.xerces.internal.impl.XMLDTDScannerImpl.scanDTDExternalSubset(XMLDTDScannerImpl.java:306) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$DTDDriver.dispatch(XMLDocumentScannerImpl.java:1174) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$DTDDriver.next(XMLDocumentScannerImpl.java:1045) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:959) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:842) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771) at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:243) at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:339) at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:121) at org.ohdsi.medlineXmlToDatabase.XMLFileIterator$DecompressAndParseThread.process(XMLFileIterator.java:130) at org.ohdsi.utilities.concurrency.BatchProcessingThread.run(BatchProcessingThread.java:42)

schuemie commented 5 years ago

This looks like an NLM server issue. Whenever the XML parser parses an XML file it connects to the NLM server to retrieve the DTD. I don't know how to disable that behavior, and I don't think it is necessary. Perhaps their server is just being overloaded?