marevol
commented
2 years ago Did you add nori?
Open sangkyunkim89 opened 2 years ago
marevol
commented
2 years ago Did you add nori?
sangkyunkim89
commented
2 years ago @marevol yes i have a nori
@Override
public Map<String, AnalysisModule.AnalysisProvider<TokenFilterFactory>> getTokenFilters() {
Map<String, AnalysisModule.AnalysisProvider<TokenFilterFactory>> extra = new HashMap<>();
extra.put("nori_part_of_speech", NoriPartOfSpeechStopFilterFactory::new);
extra.put("nori_readingform", NoriReadingFormFilterFactory::new);
extra.put("nori_number", NoriNumberFilterFactory::new);
return extra;
}
@Override
public Map<String, AnalysisModule.AnalysisProvider<TokenizerFactory>> getTokenizers() {
final Map<String, AnalysisModule.AnalysisProvider<TokenizerFactory>> extra = new HashMap();
extra.put("reloadable_nori_tokenizer", ReloadableNoriTokenizerFactory::new);
extra.put("nori_tokenizer", NoriTokenizerFactory::new);
return extra;
}@marevol I know you are very busy, but please ask when you can!
marevol
commented
2 years ago How did you send the mappings.json?
sangkyunkim89
commented
2 years ago @marevol For reference, there was no problem when creating an index directly on es.
thank u
{
"settings": {
"index": {
"analysis": {
"tokenizer": {
"korean_nori_tokenizer": {
"type": "reloadable_nori_tokenizer",
"decompound_mode": "discard"
}
},
"analyzer": {
"my_nori_analyzer": {
"tokenizer": "korean_nori_tokenizer",
"filter": [
"my_nori_filter"
]
}
},
"filter": {
"my_nori_filter": {
"type": "nori_part_of_speech",
"stoptags": [
"SP"
],
"updateable": true
}
}
}
}
},
"mappings": {
"properties": {
"test": {
"type": "keyword",
"fields": {
"nori": {
"search_analyzer": "my_nori_analyzer",
"analyzer": "my_nori_analyzer",
"type": "text"
}
}
}
}
}
}Could you provide information to reproduce it?
sangkyunkim89
commented
2 years ago package org.elasticsearch.plugin.analyzerTest;
import java.io.File;
import junit.framework.TestCase;
import org.codelibs.elasticsearch.runner.ElasticsearchClusterRunner;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.xcontent.XContentType;
import org.elasticsearch.node.Node;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
public class ReloadableNoriTokenizerFactoryTest extends TestCase {
private ElasticsearchClusterRunner runner;
private int numOfNode = 1;
private File[] keywordFiles;
private String clusterName;
@Before
public void setUp() throws Exception {
clusterName = "es-analysisja-" + System.currentTimeMillis();
runner = new ElasticsearchClusterRunner();
runner.onBuild(new ElasticsearchClusterRunner.Builder() {
@Override
public void build(final int number, final Settings.Builder settingsBuilder) {
settingsBuilder.put("http.cors.enabled", true);
settingsBuilder.put("http.cors.allow-origin", "*");
settingsBuilder.put("discovery.type", "single-node");
// settingsBuilder.putList("discovery.seed_hosts", "127.0.0.1:9301");
// settingsBuilder.putList("cluster.initial_master_nodes", "127.0.0.1:9301");
}
}).build(new ElasticsearchClusterRunner.Configs().clusterName(clusterName).numOfNode(numOfNode));
keywordFiles = null;
}
@After
public void cleanUp() throws Exception {
runner.close();
runner.clean();
if (keywordFiles != null) {
for (File file : keywordFiles) {
file.deleteOnExit();
}
}
}
@Test
public void test_basic() throws Exception {
runner.ensureYellow();
Node node = runner.node();
final String index = "dataset";
final String indexSettings = "{\n" +
"" +
" \"mappings\": {\n" +
" \"properties\": {\n" +
" \"hahahoho\": {\n" +
" \"type\": \"keyword\"\n" +
" }\n" +
" }\n" +
" }\n" +
"}";
runner.createIndex(index, Settings.builder().loadFromSource(indexSettings, XContentType.JSON).build());
runner.ensureYellow();
}
}
@marevol
I tried with a simple code, but this doesn't work either Am I unable to load the default plugin?
unknown setting [index.mappings.properties.hahahoho.type] please check that any required plugins are installed, or check the breaking changes documentation for removed settings java.lang.IllegalArgumentException: unknown setting [index.mappings.properties.hahahoho.type] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
marevol
commented
2 years ago It's for an index setting, not mapping.
final String indexSettings = "{\n" +
" \"properties\": {\n" +
" \"hahahoho\": {\n" +
" \"type\": \"keyword\"\n" +
" }\n" +
" }\n" +
"}";
runner.createIndex(index, builder -> builder.addMapping("_doc",
indexSettings, XContentType.JSON));@marevol From what you tested elsewhere, I think you should remove the settings property and set it up. Is it impossible to test until data is mapped and retrieved?
i have sourc code pluginType org.test.plugin.analysis.TestPlugin
org.test.plugin.analysis.TestPlugin has the actual plugin source, so why not?
mappings.json
The plugin is actually loaded, but I keep getting an error