jprante
commented
9 years ago Do you have the SQL statement?
It would also be nice to have insight into an example of the data from DB which causes the trouble.
Somewhere, a "d" field is present in JSON, which you do not allow in the mapping, maybe a subfield inside of "CORE_LOG", which is not on object but of type string.
Hello,
I seem to have come across an issue where the ES mapping is throwing an error from the jdbc plugin, I think it is due to an issue with the plugin, and it is causing bulk transfer to fail, and me to get node closed errors, it seems to be something with the mapping, although the mapping is labeled as string, I have tried strict and false for dynamic values in the ES mapping, and both seem to throw this error, as it is mapped as a string I don't understand why it is trying to parse it as nested, please let me know
thank you
here is the mapping and stack trace
14-11-12 10:12:31,996][ERROR][river.jdbc.BulkNodeClient] bulk [29] failed with 3 failed items, failure message = failure in bulk execution:[58]: index [core42], type [jdbc], id [103756], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ][62]: index [core42], type [jdbc], id [103760], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ][147]: index [core42], type [jdbc], id [103767], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ] [2014-11-12 10:12:32,061][ERROR][river.jdbc.BulkNodeClient] bulk [29] failed with 3 failed items, failure message = failure in bulk execution:[38]: index [core42], type [jdbc], id [103756], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ][42]: index [core42], type [jdbc], id [103760], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ][127]: index [core42], type [jdbc], id [103767], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ] [2014-11-12 10:13:12,003][ERROR][river.jdbc.BulkNodeClient] bulk [37] failed with 1 failed items, failure message = failure in bulk execution:[99]: index [core42], type [jdbc], id [103747], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ] [2014-11-12 10:13:12,077][ERROR][river.jdbc.BulkNodeClient] bulk [37] failed with 1 failed items, failure message = failure in bulk execution:[71]: index [core42], type [jdbc], id [103747], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ] [2014-11-12 10:13:31,618][ERROR][river.jdbc.BulkNodeClient] bulk [29] failed with 3 failed items, failure message = failure in bulk execution:[29]: index [core42], type [jdbc], id [103756], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ][33]: index [core42], type [jdbc], id [103760], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ][118]: index [core42], type [jdbc], id [103767], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ] [2014-11-12 10:13:31,660][ERROR][river.jdbc.BulkNodeClient] bulk [29] failed with 2 failed items, failure message = failure in bulk execution:[60]: index [core42], type [jdbc], id [103756], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ][64]: index [core42], type [jdbc], id [103760], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ] [2014-11-12 10:13:36,665][ERROR][river.jdbc.BulkNodeClient] bulk [30] failed with 1 failed items, failure message = failure in bulk execution:[4]: index [core42], type [jdbc], id [103767], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ] [2014-11-12 10:14:11,633][ERROR][river.jdbc.BulkNodeClient] bulk [37] failed with 1 failed items, failure message = failure in bulk execution:[59]: index [core42], type [jdbc], id [103747], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ] [2014-11-12 10:14:11,710][ERROR][river.jdbc.BulkNodeClient] bulk [37] failed with 1 failed items, failure message = failure in bulk execution:[91]: index [core42], type [jdbc], id [103747], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ] [2014-11-12 10:14:30,195][ERROR][river.jdbc.BulkNodeClient] bulk [29] failed with 3 failed items, failure message = failure in bulk execution:[45]: index [core42], type [jdbc], id [103756], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ][49]: index [core42], type [jdbc], id [103760], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ][134]: index [core42], type [jdbc], id [103767], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ] [2014-11-12 10:14:30,256][ERROR][river.jdbc.BulkNodeClient] bulk [29] failed with 3 failed items, failure message = failure in bulk execution:[55]: index [core42], type [jdbc], id [103756], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ][59]: index [core42], type [jdbc], id [103760], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ][144]: index [core42], type [jdbc], id [103767], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ] [2014-11-12 10:15:10,204][ERROR][river.jdbc.BulkNodeClient] bulk [37] failed with 1 failed items, failure message = failure in bulk execution:[82]: index [core42], type [jdbc], id [103747], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ] [2014-11-12 10:15:10,280][ERROR][river.jdbc.BulkNodeClient] bulk [37] failed with 1 failed items, failure message = failure in bulk execution:[94]: index [core42], type [jdbc], id [103747], message [MapperParsingException[failed to parse [CORE_LOG]]; nested: ElasticsearchIllegalArgumentException[unknown property [d]]; ] [2014-11-12 10:15:58,523][ERROR][river.jdbc.RiverThread ] interrupted while shutdownjava.io.IOException: interrupted while shutdown at org.xbib.elasticsearch.plugin.jdbc.pipeline.executor.SimplePipelineExecutor.shutdown(SimplePipelineExecutor.java:145) at org.xbib.elasticsearch.plugin.jdbc.RiverThread.afterPipelineExecutions(RiverThread.java:140) at org.xbib.elasticsearch.plugin.jdbc.RiverThread.run(RiverThread.java:130) at java.lang.Thread.run(Unknown Source) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) [2014-11-12 10:15:58,525][ERROR][river.jdbc.RiverThread ] interrupted while shutdownjava.io.IOException: interrupted while shutdown at org.xbib.elasticsearch.plugin.jdbc.pipeline.executor.SimplePipelineExecutor.shutdown(SimplePipelineExecutor.java:145) at org.xbib.elasticsearch.plugin.jdbc.RiverThread.afterPipelineExecutions(RiverThread.java:140) at org.xbib.elasticsearch.plugin.jdbc.RiverThread.run(RiverThread.java:130) at java.lang.Thread.run(Unknown Source) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) [2014-11-12 10:15:58,524][ERROR][river.jdbc.RiverThread ] interrupted while shutdownjava.io.IOException: interrupted while shutdown at org.xbib.elasticsearch.plugin.jdbc.pipeline.executor.SimplePipelineExecutor.shutdown(SimplePipelineExecutor.java:145) at org.xbib.elasticsearch.plugin.jdbc.RiverThread.afterPipelineExecutions(RiverThread.java:140) at org.xbib.elasticsearch.plugin.jdbc.RiverThread.run(RiverThread.java:130) at java.lang.Thread.run(Unknown Source) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) [2014-11-12 10:15:58,523][ERROR][river.jdbc.RiverThread ] interrupted while shutdownjava.io.IOException: interrupted while shutdown at org.xbib.elasticsearch.plugin.jdbc.pipeline.executor.SimplePipelineExecutor.shutdown(SimplePipelineExecutor.java:145) at org.xbib.elasticsearch.plugin.jdbc.RiverThread.afterPipelineExecutions(RiverThread.java:140) at org.xbib.elasticsearch.plugin.jdbc.RiverThread.run(RiverThread.java:130)
and finally here is the mapping on the index, as my research is showing me this has something to do with the issue, however strict mapping to a string should have fixed it as far as I can see, I have tried strict and false dynamic mapping, both create the same results
Put core42/_mapping/jdbc { "jdbc":{ "dynamic":"false", "properties": { "CLASS": { "type": "string" }, "CORE_LEVEL": { "type": "string" }, "CORE_LOG": { "type": "string" }, "CORE_USER": { "type": "string" }, "CUSTOMER": { "type": "string" }, "DATETIME": { "type": "date", "format": "dateOptionalTime" }, "SERVER_NAME": { "type": "string" }, "WEBAPP": { "type": "string" } } } }