jprante / elasticsearch-analysis-skos

SKOS analysis for Elasticsearch
Apache License 2.0
54 stars 24 forks source link

Sémantic relations between concepts didn't work. #3

Closed moees closed 10 years ago

moees commented 10 years ago

Hi. I installed analysis-skos with most of the elasticsearch versions, the actual jprante plugin can be installed without errors with ES versions up to elasticsearch-0.20.1.

While the altLabel, prefLabel worked with no problem, none of the broader or narrower relations worked with me. I have tried with the ukat_examples.n3 example and some other dictionaries, i have tried to update the jena libraries and checked the skos.rdf schema and it still not working even with elasticsearch-0.19.8

Here is the index settings/mappings: curl -XPOST 'http://localhost:9200/test' -d '{ "settings" : { "index" : { "analysis" : { "filter": { "skosfilter" : { "type": "skos", "skosFile": "/home/nemo/Téléchargements/elasticsearch/MR/elasticsearch-0.20.1/skos/ukat/ukat_examples.n3", "path": "/home/nemo/Téléchargements/elasticsearch/MR/elasticsearch-0.20.1/skos/working", "expansionType": "URI" } }, "analyzer" : { "skos" : { "type" : "custom", "tokenizer" : "keyword", "filter" : "skosfilter" } } } } }, "mappings" : { "default" : { "properties" : { "subject" : { "type" : "string", "index_analyzer" : "skos", "search_analyzer" : "standard" } } } }

}'

Please help, i need this plugin badly.

jprante commented 10 years ago

Can you check the 1.1.0 release for Elasticsearch 0.90.7?

jprante commented 10 years ago

I just read you want broader/narrower support. This is not supported in https://github.com/behas/lucene-skos and this ES plugin is just a simple port, it can not do more than the original.

Can you show examples of queries and expected results?

moees commented 10 years ago

Hi Jprante Thanks for your reply.

I installed 1.1.0 and Elasticsearch 0.90.7 with no errors. The alt and pref queries worked correctly.

I'm a liittle confused about the broader/narrower support. I didn't go deep in the source files, but it seems to be implemented in behas/lucene-skos code and the schema skos.rdf. Please tell me if you are sure it's not supported?

You can find my index settings/mappings in my first post. Here's a query that i used with the ukat_examples.n3 example :

ukat:859 rdf:type skos:Concept ; skos:prefLabel "Weapons" ; skos:altLabel "arms" ; skos:narrower ukat:18874 ; ukat:18874 rdf:type skos:Concept; skos:prefLabel "Ammunition".

curl -XGET 'localhost:9200/test/_search?pretty' -d '{ "query": { "text" : { "subject": "Ammunition" } } }'

The query gives 0 results, while it works with the keyword "arms".

jprante commented 10 years ago

Yes, query must give 0 results, because "Ammunition" is under subject URI ukat:18874, not under subject URI ukat:859.

For illustrative example, see also https://code.google.com/p/lucene-skos/wiki/UseCases

There is no example for searching for subject "Ammunition", I'm sure you will need to use a subject URI ukat:18874 for that.

moees commented 10 years ago

Please see the following citation in your link https://code.google.com/p/lucene-skos/wiki/UseCases : "For each kind of SKOS property (prefLabel, altLabel, broader, narrower, etc) it should be possible to define whether it should be considered in the expansion process. "

Let's consider that i need to use the Label-based term expansion, the filter should look like this:

{ "skosfilter" : { "type": "skos", "skosFile": "/home/nemo/Téléchargements/elasticsearch/MR/elasticsearch-0.20.1/skos/ukat/ukat_examples.n3", "path": "/home/nemo/Téléchargements/elasticsearch/MR/elasticsearch-0.20.1/skos/working", "expansionType": "LABEL" } }

And the text document looks like this:

curl -XPUT 'http://localhost:9200/test/subjects/1' -d '{ "subject" : "weapons" }'

Here, the label "weapons" has ukat:859 as resource, and this resource it self is a broader concept of ukat:18874 (as broader property is "inverseOf" narrower in skos schema ). So in term-expansion, it should expand alt/pref of the query to the concept and all the alt/pref of it's narrower concepts (entailment).

As i know, this is the purpuse of the skos.

jprante commented 10 years ago

I had to look into the code because I did not remember, it is quite a while ago I ported the code.

There is a parameter skosType where I forgot to give an example. There, you can declare the SKOS attributes for label expansion.

The SKOS attributes for skosType are: PREF ALT HIDDEN BROADER NARROWER BROADERTRANSITIVE NARROWERTRANSITIVE RELATED (delimited by a space)

Here is a full working ukat example for LABEL expansion:

https://gist.github.com/jprante/7551124

moees commented 10 years ago

Thanks for the example. it worked now. i rembember i tried these parametres but it didn't work, i must had somthing wrong.