search

Norconex / committer-sql

Implementation of Norconex Committer for SQL (JDBC) databases.
https://opensource.norconex.com/committers/sql/
Apache License 2.0
1 stars 6 forks source link

CopyTagger fields do not end up in SQL #8

Open onixterry opened 4 years ago

onixterry commented 4 years ago

I'm trying to use the CopyTagger to copy fields from a source system into a standardized field name.

<tagger class="com.norconex.importer.handler.tagger.impl.CopyTagger">
                    <copy fromField="og_type" toField="category" overwrite="true"/>
                    </tagger>

I've experimented with placing these in the preParseHandler and the postParseHandler as well as using the original names of the fields as well as the value that is assigned when "fixFieldNames" is used. Regardless of what I do, the fields do not end up in the SQL endpoint. I've also noticed that these fields are not automatically created by the "createFieldSQL" statement.

Does the SQL Committer not support these features of Norconex? I am using the December release of the HTTP Collector and the 2.0.0 version of the SQL Committer.

essiembre commented 4 years ago

Can you share a copy of your full config? I suggest you try with the XMLFileCommitter to see if the field you are interested it makes it there. Then you'll know if the proble is with the SQL Committer or elsewhere.

onixterry commented 4 years ago

Thanks for the suggestion. I am seeing differences depending on which database I use. It seems Postgresql is better at handling the data than MariaDB. I'm using the XML to pre-validate and that is helping. Great suggestion.