Closed HappyCustomers closed 6 years ago
essiembre
commented
6 years ago Which version are you using? I recommend using the 2.0.0 snapshot version.
It is a major upgrade that fixes several issues.
If you are already using that version, can you share your config? Maybe it tries to commit a long value into a numeric DB field that is not long enough?
HappyCustomers
commented
6 years ago I am using the 2.0.0 snapshot version. please find attached the config file along with sample URL for your reference thank you
I am using the following MYSQL driver -> mysql-connector-java-5.1.45-bin.jar
Tried today with different URLs getting the "Data Source is closed"
The SQL committer commits around 12 -20 rows in MYSQL database table and get the following error`
When I run the same URLs for SOLR committer I get 214 pages committed in SOLR
com.norconex.committer.core.CommitterException: Could not commit batch to database. at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:479) at com.norconex.committer.core.AbstractBatchCommitter.commitAndCleanBatch(AbstractBatchCommitter.java:179) at com.norconex.committer.core.AbstractBatchCommitter.cacheOperationAndCommitIfReady(AbstractBatchCommitter.java:208) at com.norconex.committer.core.AbstractBatchCommitter.commitAddition(AbstractBatchCommitter.java:143) at com.norconex.committer.core.AbstractFileQueueCommitter.commit(AbstractFileQueueCommitter.java:222) at com.norconex.committer.sql.SQLCommitter.commit(SQLCommitter.java:424) at com.norconex.collector.core.crawler.AbstractCrawler.execute(AbstractCrawler.java:274) at com.norconex.collector.core.crawler.AbstractCrawler.doExecute(AbstractCrawler.java:228) at com.norconex.collector.core.crawler.AbstractCrawler.startExecution(AbstractCrawler.java:184) at com.norconex.jef4.job.AbstractResumableJob.execute(AbstractResumableJob.java:49) at com.norconex.jef4.suite.JobSuite.runJob(JobSuite.java:355) at com.norconex.jef4.suite.JobSuite.doExecute(JobSuite.java:296) at com.norconex.jef4.suite.JobSuite.execute(JobSuite.java:168) at com.norconex.collector.core.AbstractCollector.start(AbstractCollector.java:132) at com.norconex.collector.core.AbstractCollectorLauncher.launch(AbstractCollectorLauncher.java:95) at com.norconex.collector.http.HttpCollector.main(HttpCollector.java:74) Caused by: java.sql.SQLException: Data source is closed at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2016) at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1533) at org.apache.commons.dbutils.AbstractQueryRunner.prepareConnection(AbstractQueryRunner.java:319) at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:327) at com.norconex.committer.sql.SQLCommitter.runExists(SQLCommitter.java:584) at com.norconex.committer.sql.SQLCommitter.recordExists(SQLCommitter.java:575) at com.norconex.committer.sql.SQLCommitter.sqlInsertDoc(SQLCommitter.java:527) at com.norconex.committer.sql.SQLCommitter.addOperation(SQLCommitter.java:508) at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:465)
essiembre
commented
6 years ago With your config, I was able to reproduce. It turns out to be a bug, which was fixed. Please try the latest snapshot release and confirm.
It everything is working fine for you with this fix, we'll make it a formal release.
HappyCustomers
commented
6 years ago Sorry did not see your message. will check and let you know
HappyCustomers
commented
6 years ago I tested and it looks like it is working. I will test few more websites and confirm in a day. Thank you one more issue i observed is I wanted only the pages within http://www.xxx.com/bangalore, however the colelctor is fetching all the pages under http://www.xxxs.com/.
I have added the following settings though "---filter class="$urlFilter" onMatch="include">http://www.xxx.com/bangalore/.* / filter--"
Can You please let me know if any configuration settings required? I am using the same config file which i had attached earlier
essiembre
commented
6 years ago Have a look at the documentation here. You'll see the group tag is rather called <referenceFilters> and the class RegexReferenceFilter. You have something different in your config.
HappyCustomers
commented
6 years ago Thank You, The Regex Filter is working now, For some website jpg URLS are getting downloaded, i will further evaluate and let you know the pattern if any .
Also I confirm that the MYSQL committer is working now. Tested with 25 websites and they are getting committed in MYSQL with no errors.
Thank you once again
essiembre
commented
6 years ago Thanks for confirming.
As for your jpg, if you are referring to your reference filter in the crawler defaults, know that the whole block gets replaced when you specify entries in individual crawler settings. Unless you have multiple crawlers, I would not recommend using defaults to facilitate maintenance.
If you do have multiple crawlers in the same config and would like to not overwrite entire blocks, you can use configuration fragments. If you suspect an issue with the filter itself, please create a new ticket under the corresponding github project.
HappyCustomers
commented
6 years ago Hi , I am getting the error in SQL Committer again. I have sent the log file by email
Test Example: 2018-09-06 20:01:01 INFO - Max queue size reached (10). Committing Test Example: 2018-09-06 20:01:01 INFO - Committing 10 files Test Example: 2018-09-06 20:01:01 INFO - Sending 5 commit operations to SQL database. Test Example: 2018-09-06 20:01:01 INFO - Checking if table "FH_1200" exists... Test Example: 2018-09-06 20:01:05 INFO - Table "FH_1200" does not exist. Attempting to create it... Test Example: 2018-09-06 20:01:05 INFO - Table created. Test Example: 2018-09-06 20:01:05 ERROR - Could not commit batched operations. com.norconex.committer.core.CommitterException: Could not commit batch to database. at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:480) at com.norconex.committer.core.AbstractBatchCommitter.commitAndCleanBatch(AbstractBatchCommitter.java:179) at com.norconex.committer.core.AbstractBatchCommitter.cacheOperationAndCommitIfReady(AbstractBatchCommitter.java:208) at com.norconex.committer.core.AbstractBatchCommitter.commitAddition(AbstractBatchCommitter.java:143) at com.norconex.committer.core.AbstractFileQueueCommitter.commit(AbstractFileQueueCommitter.java:222) at com.norconex.committer.sql.SQLCommitter.commit(SQLCommitter.java:425) at com.norconex.committer.core.AbstractCommitter.commitIfReady(AbstractCommitter.java:146) at com.norconex.committer.core.AbstractCommitter.add(AbstractCommitter.java:97) at com.norconex.collector.core.pipeline.committer.CommitModuleStage.execute(CommitModuleStage.java:34) at com.norconex.collector.core.pipeline.committer.CommitModuleStage.execute(CommitModuleStage.java:27) at com.norconex.commons.lang.pipeline.Pipeline.execute(Pipeline.java:91) at com.norconex.collector.http.crawler.HttpCrawler.executeCommitterPipeline(HttpCrawler.java:379) at com.norconex.collector.core.crawler.AbstractCrawler.processImportResponse(AbstractCrawler.java:595) at com.norconex.collector.core.crawler.AbstractCrawler.processNextQueuedCrawlData(AbstractCrawler.java:541) at com.norconex.collector.core.crawler.AbstractCrawler.processNextReference(AbstractCrawler.java:419) at com.norconex.collector.core.crawler.AbstractCrawler$ProcessReferencesRunnable.run(AbstractCrawler.java:812) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.sql.SQLException: Incorrect string value: '\xF0\x9F\x90\x92\xE2\x9C...' for column 'content' at row 1 Query: INSERT INTO FH_1200(document_reference,Server,keywords,imagepath,description,id,title,Date,content) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
essiembre
commented
6 years ago The error is your database telling you that you are trying to insert characters it does not support. \xF0\x9F... are hexa decimal representations of those characters, not the actual characters. More than likely those are special characters not supported by your configured database character set.
You can try to strip all these characters using the Importer ReplaceTagger or equivalent, but if they are important to you, I suggest you check with your database vendor/community what you can do on the database side to insert these characters safely (e.g. changing the default character set).
Unless you are experiencing the exact same issue again, next time please open a new ticket for new issues as opposed to add to a closed ticket (which can easilly be missed).
I am getting the following errors while using SQL Committer V2. Can you please help?
Hello Hotel Example: 2018-08-09 21:59:33 INFO - Hello Hotel Example: 100% completed (323 processed/323 total) Hello Hotel Example: 2018-08-09 21:59:33 INFO - Hello Hotel Example: Deleting orphan references (if any)... Hello Hotel Example: 2018-08-09 21:59:33 INFO - Hello Hotel Example: Deleted 0 orphan references... Hello Hotel Example: 2018-08-09 21:59:33 INFO - Hello Hotel Example: Crawler finishing: committing documents. Hello Hotel Example: 2018-08-09 21:59:33 INFO - Committing 89 files Hello Hotel Example: 2018-08-09 21:59:33 INFO - Sending 10 commit operations to SQL database. Hello Hotel Example: 2018-08-09 21:59:33 INFO - Checking if table "crawler7" exists... Hello Hotel Example: 2018-08-09 21:59:33 INFO - Table "crawler7" does not exist. Attempting to create it... Hello Hotel Example: 2018-08-09 21:59:33 INFO - Table created. Hello Hotel Example: 2018-08-09 21:59:34 INFO - Done sending commit operations to database. Hello Hotel Example: 2018-08-09 21:59:34 INFO - Sending 10 commit operations to SQL database. Hello Hotel Example: 2018-08-09 21:59:34 INFO - Done sending commit operations to database. Hello Hotel Example: 2018-08-09 21:59:34 INFO - Sending 10 commit operations to SQL database. Hello Hotel Example: 2018-08-09 21:59:34 INFO - Done sending commit operations to database. Hello Hotel Example: 2018-08-09 21:59:34 INFO - Sending 10 commit operations to SQL database. Hello Hotel Example: 2018-08-09 21:59:34 ERROR - Could not commit batched operations. com.norconex.committer.core.CommitterException: Could not commit batch to database. at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:479) at com.norconex.committer.core.AbstractBatchCommitter.commitAndCleanBatch(AbstractBatchCommitter.java:179) at com.norconex.committer.core.AbstractBatchCommitter.cacheOperationAndCommitIfReady(AbstractBatchCommitter.java:208) at com.norconex.committer.core.AbstractBatchCommitter.commitAddition(AbstractBatchCommitter.java:143) at com.norconex.committer.core.AbstractFileQueueCommitter.commit(AbstractFileQueueCommitter.java:222) at com.norconex.committer.sql.SQLCommitter.commit(SQLCommitter.java:424) at com.norconex.collector.core.crawler.AbstractCrawler.execute(AbstractCrawler.java:274) at com.norconex.collector.core.crawler.AbstractCrawler.doExecute(AbstractCrawler.java:228) at com.norconex.collector.core.crawler.AbstractCrawler.startExecution(AbstractCrawler.java:184) at com.norconex.jef4.job.AbstractResumableJob.execute(AbstractResumableJob.java:49) at com.norconex.jef4.suite.JobSuite.runJob(JobSuite.java:355) at com.norconex.jef4.suite.JobSuite.doExecute(JobSuite.java:296) at com.norconex.jef4.suite.JobSuite.execute(JobSuite.java:168) at com.norconex.collector.core.AbstractCollector.start(AbstractCollector.java:132) at com.norconex.collector.core.AbstractCollectorLauncher.launch(AbstractCollectorLauncher.java:95) at com.norconex.collector.http.HttpCollector.main(HttpCollector.java:74) Caused by: java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.Integer at com.norconex.committer.sql.SQLCommitter.runExists(SQLCommitter.java:584) at com.norconex.committer.sql.SQLCommitter.recordExists(SQLCommitter.java:575) at com.norconex.committer.sql.SQLCommitter.sqlInsertDoc(SQLCommitter.java:527) at com.norconex.committer.sql.SQLCommitter.addOperation(SQLCommitter.java:508) at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:465) ... 15 more Hello Hotel Example: 2018-08-09 21:59:39 INFO - Sending 10 commit operations to SQL database. Hello Hotel Example: 2018-08-09 21:59:39 ERROR - Could not commit batched operations. com.norconex.committer.core.CommitterException: Could not commit batch to database. at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:479) at com.norconex.committer.core.AbstractBatchCommitter.commitAndCleanBatch(AbstractBatchCommitter.java:179) at com.norconex.committer.core.AbstractBatchCommitter.cacheOperationAndCommitIfReady(AbstractBatchCommitter.java:208) at com.norconex.committer.core.AbstractBatchCommitter.commitAddition(AbstractBatchCommitter.java:143) at com.norconex.committer.core.AbstractFileQueueCommitter.commit(AbstractFileQueueCommitter.java:222) at com.norconex.committer.sql.SQLCommitter.commit(SQLCommitter.java:424) at com.norconex.collector.core.crawler.AbstractCrawler.execute(AbstractCrawler.java:274) at com.norconex.collector.core.crawler.AbstractCrawler.doExecute(AbstractCrawler.java:228) at com.norconex.collector.core.crawler.AbstractCrawler.startExecution(AbstractCrawler.java:184) at com.norconex.jef4.job.AbstractResumableJob.execute(AbstractResumableJob.java:49) at com.norconex.jef4.suite.JobSuite.runJob(JobSuite.java:355) at com.norconex.jef4.suite.JobSuite.doExecute(JobSuite.java:296) at com.norconex.jef4.suite.JobSuite.execute(JobSuite.java:168) at com.norconex.collector.core.AbstractCollector.start(AbstractCollector.java:132) at com.norconex.collector.core.AbstractCollectorLauncher.launch(AbstractCollectorLauncher.java:95) at com.norconex.collector.http.HttpCollector.main(HttpCollector.java:74) Caused by: java.sql.SQLException: Data source is closed at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2016) at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1533) at org.apache.commons.dbutils.AbstractQueryRunner.prepareConnection(AbstractQueryRunner.java:319) at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:327) at com.norconex.committer.sql.SQLCommitter.runExists(SQLCommitter.java:584) at com.norconex.committer.sql.SQLCommitter.recordExists(SQLCommitter.java:575) at com.norconex.committer.sql.SQLCommitter.sqlInsertDoc(SQLCommitter.java:527) at com.norconex.committer.sql.SQLCommitter.addOperation(SQLCommitter.java:508) at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:465) ... 15 more Hello Hotel Example: 2018-08-09 21:59:44 INFO - Sending 10 commit operations to SQL database. Hello Hotel Example: 2018-08-09 21:59:44 ERROR - Could not commit batched operations. com.norconex.committer.core.CommitterException: Could not commit batch to database. at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:479) at com.norconex.committer.core.AbstractBatchCommitter.commitAndCleanBatch(AbstractBatchCommitter.java:179) at com.norconex.committer.core.AbstractBatchCommitter.cacheOperationAndCommitIfReady(AbstractBatchCommitter.java:208) at com.norconex.committer.core.AbstractBatchCommitter.commitAddition(AbstractBatchCommitter.java:143) at com.norconex.committer.core.AbstractFileQueueCommitter.commit(AbstractFileQueueCommitter.java:222) at com.norconex.committer.sql.SQLCommitter.commit(SQLCommitter.java:424) at com.norconex.collector.core.crawler.AbstractCrawler.execute(AbstractCrawler.java:274) at com.norconex.collector.core.crawler.AbstractCrawler.doExecute(AbstractCrawler.java:228) at com.norconex.collector.core.crawler.AbstractCrawler.startExecution(AbstractCrawler.java:184) at com.norconex.jef4.job.AbstractResumableJob.execute(AbstractResumableJob.java:49) at com.norconex.jef4.suite.JobSuite.runJob(JobSuite.java:355) at com.norconex.jef4.suite.JobSuite.doExecute(JobSuite.java:296) at com.norconex.jef4.suite.JobSuite.execute(JobSuite.java:168) at com.norconex.collector.core.AbstractCollector.start(AbstractCollector.java:132) at com.norconex.collector.core.AbstractCollectorLauncher.launch(AbstractCollectorLauncher.java:95) at com.norconex.collector.http.HttpCollector.main(HttpCollector.java:74) Caused by: java.sql.SQLException: Data source is closed at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2016) at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1533) at org.apache.commons.dbutils.AbstractQueryRunner.prepareConnection(AbstractQueryRunner.java:319) at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:327) at com.norconex.committer.sql.SQLCommitter.runExists(SQLCommitter.java:584) at com.norconex.committer.sql.SQLCommitter.recordExists(SQLCommitter.java:575) at com.norconex.committer.sql.SQLCommitter.sqlInsertDoc(SQLCommitter.java:527) at com.norconex.committer.sql.SQLCommitter.addOperation(SQLCommitter.java:508) at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:465) ... 15 more Hello Hotel Example: 2018-08-09 21:59:49 INFO - Sending 10 commit operations to SQL database. Hello Hotel Example: 2018-08-09 21:59:49 ERROR - Could not commit batched operations. com.norconex.committer.core.CommitterException: Could not commit batch to database. at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:479) at com.norconex.committer.core.AbstractBatchCommitter.commitAndCleanBatch(AbstractBatchCommitter.java:179) at com.norconex.committer.core.AbstractBatchCommitter.cacheOperationAndCommitIfReady(AbstractBatchCommitter.java:208) at com.norconex.committer.core.AbstractBatchCommitter.commitAddition(AbstractBatchCommitter.java:143) at com.norconex.committer.core.AbstractFileQueueCommitter.commit(AbstractFileQueueCommitter.java:222) at com.norconex.committer.sql.SQLCommitter.commit(SQLCommitter.java:424) at com.norconex.collector.core.crawler.AbstractCrawler.execute(AbstractCrawler.java:274) at com.norconex.collector.core.crawler.AbstractCrawler.doExecute(AbstractCrawler.java:228) at com.norconex.collector.core.crawler.AbstractCrawler.startExecution(AbstractCrawler.java:184) at com.norconex.jef4.job.AbstractResumableJob.execute(AbstractResumableJob.java:49) at com.norconex.jef4.suite.JobSuite.runJob(JobSuite.java:355) at com.norconex.jef4.suite.JobSuite.doExecute(JobSuite.java:296) at com.norconex.jef4.suite.JobSuite.execute(JobSuite.java:168) at com.norconex.collector.core.AbstractCollector.start(AbstractCollector.java:132) at com.norconex.collector.core.AbstractCollectorLauncher.launch(AbstractCollectorLauncher.java:95) at com.norconex.collector.http.HttpCollector.main(HttpCollector.java:74) Caused by: java.sql.SQLException: Data source is closed at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2016) at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1533) at org.apache.commons.dbutils.AbstractQueryRunner.prepareConnection(AbstractQueryRunner.java:319) at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:327) at com.norconex.committer.sql.SQLCommitter.runExists(SQLCommitter.java:584) at com.norconex.committer.sql.SQLCommitter.recordExists(SQLCommitter.java:575) at com.norconex.committer.sql.SQLCommitter.sqlInsertDoc(SQLCommitter.java:527) at com.norconex.committer.sql.SQLCommitter.addOperation(SQLCommitter.java:508) at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:465) ... 15 more Hello Hotel Example: 2018-08-09 21:59:54 INFO - Sending 10 commit operations to SQL database. Hello Hotel Example: 2018-08-09 21:59:54 ERROR - Could not commit batched operations. com.norconex.committer.core.CommitterException: Could not commit batch to database. at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:479) at com.norconex.committer.core.AbstractBatchCommitter.commitAndCleanBatch(AbstractBatchCommitter.java:179) at com.norconex.committer.core.AbstractBatchCommitter.cacheOperationAndCommitIfReady(AbstractBatchCommitter.java:208) at com.norconex.committer.core.AbstractBatchCommitter.commitAddition(AbstractBatchCommitter.java:143) at com.norconex.committer.core.AbstractFileQueueCommitter.commit(AbstractFileQueueCommitter.java:222) at com.norconex.committer.sql.SQLCommitter.commit(SQLCommitter.java:424) at com.norconex.collector.core.crawler.AbstractCrawler.execute(AbstractCrawler.java:274) at com.norconex.collector.core.crawler.AbstractCrawler.doExecute(AbstractCrawler.java:228) at com.norconex.collector.core.crawler.AbstractCrawler.startExecution(AbstractCrawler.java:184) at com.norconex.jef4.job.AbstractResumableJob.execute(AbstractResumableJob.java:49) at com.norconex.jef4.suite.JobSuite.runJob(JobSuite.java:355) at com.norconex.jef4.suite.JobSuite.doExecute(JobSuite.java:296) at com.norconex.jef4.suite.JobSuite.execute(JobSuite.java:168) at com.norconex.collector.core.AbstractCollector.start(AbstractCollector.java:132) at com.norconex.collector.core.AbstractCollectorLauncher.launch(AbstractCollectorLauncher.java:95) at com.norconex.collector.http.HttpCollector.main(HttpCollector.java:74) Caused by: java.sql.SQLException: Data source is closed at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2016) at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1533) at org.apache.commons.dbutils.AbstractQueryRunner.prepareConnection(AbstractQueryRunner.java:319) at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:327) at com.norconex.committer.sql.SQLCommitter.runExists(SQLCommitter.java:584) at com.norconex.committer.sql.SQLCommitter.recordExists(SQLCommitter.java:575) at com.norconex.committer.sql.SQLCommitter.sqlInsertDoc(SQLCommitter.java:527) at com.norconex.committer.sql.SQLCommitter.addOperation(SQLCommitter.java:508) at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:465) ... 15 more Hello Hotel Example: 2018-08-09 21:59:59 INFO - Sending 10 commit operations to SQL database. Hello Hotel Example: 2018-08-09 21:59:59 INFO - Hello Hotel Example: Crawler executed in 8 minutes 32 seconds. Hello Hotel Example: 2018-08-09 21:59:59 INFO - Hello Hotel Example: Closing sitemap store... Hello Hotel Example: 2018-08-09 21:59:59 ERROR - Execution failed for job: Hello Hotel Example com.norconex.committer.core.CommitterException: Could not commit batch to database. at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:479) at com.norconex.committer.core.AbstractBatchCommitter.commitAndCleanBatch(AbstractBatchCommitter.java:179) at com.norconex.committer.core.AbstractBatchCommitter.cacheOperationAndCommitIfReady(AbstractBatchCommitter.java:208) at com.norconex.committer.core.AbstractBatchCommitter.commitAddition(AbstractBatchCommitter.java:143) at com.norconex.committer.core.AbstractFileQueueCommitter.commit(AbstractFileQueueCommitter.java:222) at com.norconex.committer.sql.SQLCommitter.commit(SQLCommitter.java:424) at com.norconex.collector.core.crawler.AbstractCrawler.execute(AbstractCrawler.java:274) at com.norconex.collector.core.crawler.AbstractCrawler.doExecute(AbstractCrawler.java:228) at com.norconex.collector.core.crawler.AbstractCrawler.startExecution(AbstractCrawler.java:184) at com.norconex.jef4.job.AbstractResumableJob.execute(AbstractResumableJob.java:49) at com.norconex.jef4.suite.JobSuite.runJob(JobSuite.java:355) at com.norconex.jef4.suite.JobSuite.doExecute(JobSuite.java:296) at com.norconex.jef4.suite.JobSuite.execute(JobSuite.java:168) at com.norconex.collector.core.AbstractCollector.start(AbstractCollector.java:132) at com.norconex.collector.core.AbstractCollectorLauncher.launch(AbstractCollectorLauncher.java:95) at com.norconex.collector.http.HttpCollector.main(HttpCollector.java:74) Caused by: java.sql.SQLException: Data source is closed at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2016) at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1533) at org.apache.commons.dbutils.AbstractQueryRunner.prepareConnection(AbstractQueryRunner.java:319) at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:327) at com.norconex.committer.sql.SQLCommitter.runExists(SQLCommitter.java:584) at com.norconex.committer.sql.SQLCommitter.recordExists(SQLCommitter.java:575) at com.norconex.committer.sql.SQLCommitter.sqlInsertDoc(SQLCommitter.java:527) at com.norconex.committer.sql.SQLCommitter.addOperation(SQLCommitter.java:508) at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:465) ... 15 more Hello Hotel Example: 2018-08-09 21:59:59 INFO - Running Hello Hotel Example: END (Thu Aug 09 21:51:26 IST 2018)