essiembre
commented
6 years ago From #2:
The error is your database telling you that you are trying to insert characters it does not support. \xF0\x9F... are hexa decimal representations of those characters, not the actual characters. More than likely those are special characters not supported by your configured database character set.
You can try to strip all these characters using the Importer ReplaceTagger or equivalent, but if they are important to you, I suggest you check with your database vendor/community what you can do on the database side to insert these characters safely (e.g. changing the default character set).
Unless you are experiencing the exact same issue again, next time please open a new ticket for new issues as opposed to add to a closed ticket (which can easilly be missed).
HappyCustomers
I am getting the following error while committing to mysql.
Test Example: 2018-09-06 20:01:01 INFO - Max queue size reached (10). Committing Test Example: 2018-09-06 20:01:01 INFO - Committing 10 files Test Example: 2018-09-06 20:01:01 INFO - Sending 5 commit operations to SQL database. Test Example: 2018-09-06 20:01:01 INFO - Checking if table "FH_1200" exists... Test Example: 2018-09-06 20:01:05 INFO - Table "FH_1200" does not exist. Attempting to create it... Test Example: 2018-09-06 20:01:05 INFO - Table created. Test Example: 2018-09-06 20:01:05 ERROR - Could not commit batched operations. com.norconex.committer.core.CommitterException: Could not commit batch to database. at com.norconex.committer.sql.SQLCommitter.commitBatch(SQLCommitter.java:480) at com.norconex.committer.core.AbstractBatchCommitter.commitAndCleanBatch(AbstractBatchCommitter.java:179) at com.norconex.committer.core.AbstractBatchCommitter.cacheOperationAndCommitIfReady(AbstractBatchCommitter.java:208) at com.norconex.committer.core.AbstractBatchCommitter.commitAddition(AbstractBatchCommitter.java:143) at com.norconex.committer.core.AbstractFileQueueCommitter.commit(AbstractFileQueueCommitter.java:222) at com.norconex.committer.sql.SQLCommitter.commit(SQLCommitter.java:425) at com.norconex.committer.core.AbstractCommitter.commitIfReady(AbstractCommitter.java:146) at com.norconex.committer.core.AbstractCommitter.add(AbstractCommitter.java:97) at com.norconex.collector.core.pipeline.committer.CommitModuleStage.execute(CommitModuleStage.java:34) at com.norconex.collector.core.pipeline.committer.CommitModuleStage.execute(CommitModuleStage.java:27) at com.norconex.commons.lang.pipeline.Pipeline.execute(Pipeline.java:91) at com.norconex.collector.http.crawler.HttpCrawler.executeCommitterPipeline(HttpCrawler.java:379) at com.norconex.collector.core.crawler.AbstractCrawler.processImportResponse(AbstractCrawler.java:595) at com.norconex.collector.core.crawler.AbstractCrawler.processNextQueuedCrawlData(AbstractCrawler.java:541) at com.norconex.collector.core.crawler.AbstractCrawler.processNextReference(AbstractCrawler.java:419) at com.norconex.collector.core.crawler.AbstractCrawler$ProcessReferencesRunnable.run(AbstractCrawler.java:812) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.sql.SQLException: Incorrect string value: '\xF0\x9F\x90\x92\xE2\x9C...' for column 'content' at row 1 Query: INSERT INTO FH_1200(document_reference,Server,keywords,imagepath,description,id,title,Date,content) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
I am sending the configuration by email with the same subject
Thank You