Catmandu and ES 6.x ?

[clrh]

Hi,

I am trying to store in Elasticsearch and I got some problem. No error... I don't know how to debug.

$ cat catmandu.yml 
store:
  mod:
   package: MongoDB
   options:
    database_name: catmandu
  es:
   package: ElasticSearch
   options:
    index_name: catmandu
$ catmandu import -v MARC to mod < test.mrc 
imported 85 items
done
$ catmandu import -v MARC to es < test.mrc 
imported 85 items
done
$ catmandu count es
0
$ catmandu count mod
85

ES version: "6.5.1"

Do you have an hint ? Regards.

[nics]

Hello Claire,

you should also set the correct client version as explained here: https://metacpan.org/pod/Catmandu::Store::ElasticSearch#COMPATIBILITY

first install the client with a cpan tool:

cpanm Search::Elasticsearch::Client::6_0::Direct

then add the version to the config:

es: package: ElasticSearch options: client: '1_0::Direct' index_name: catmandu

Regards, Nicolas

On 26 Nov 2018, at 15:56, Claire Hernandez notifications@github.com wrote:

Hi,

I am trying to store in Elasticsearch and I got some problem. No error... I don't know how to debug.

$ cat catmandu.yml store: mod: package: MongoDB options: database_name: catmandu es: package: ElasticSearch options: index_name: catmandu $ catmandu import -v MARC to mod < test.mrc imported 85 items done $ catmandu import -v MARC to es < test.mrc imported 85 items done $ catmandu count es 0 $ catmandu count mod 85

ES version: "6.5.1"

Do you have an hint ? Regards.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[clrh]

First try:

$ sudo cpanm Search::Elasticsearch::Client::6_0::Direct
Search::Elasticsearch::Client::6_0::Direct is up to date. (6.00)
$ cat catmandu.yml 
store:
  es:
   package: ElasticSearch
   options:
    client: '6_0::Direct'
    index_name: catmandu

More hope but no better result. I will retry tomorrow. Thanks for quick answer.

[jorol]

See also https://librecatproject.wordpress.com/2014/12/16/day-12-index-your-data-with-elasticsearch/

[clrh]

(I'm back :)

So I verified my configuration with your welcomed feedbacks and it seems ok. The count is always to 0. The indices are created but contains no record.

I would be very glad to see an error or a log somewhere.

[nics]

Hello Claire,

there was a typo in my example config:

es: package: ElasticSearch options: client: ‘6_0::Direct' index_name: catmandu

On 28 Nov 2018, at 12:00, Claire Hernandez notifications@github.com wrote:

(I'm back :)

So I verified my configuration with your welcomed feedbacks and it seems ok. The count is always to 0. The indices are created but contains no record.

I would be very glad to see an error or a log somewhere.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/LibreCat/Catmandu-Store-Elasticsearch/issues/23#issuecomment-442407108, or mute the thread https://github.com/notifications/unsubscribe-auth/AACKXVFBLCVUn7Tf-jVd75HuGzUiZSkIks5uzmzOgaJpZM4YzMWX.

[clrh]

@nics yes I saw it and fixed in my config file (before my last test).

[nics]

You also have to set a key_prefix on your store as described here: https://metacpan.org/pod/Catmandu::Store::ElasticSearch#MIGRATING-A-STORE-FROM-ELASTICSEARCH-1.0-TO-2.0-OR-HIGHER.

This is necessary because the default catmandu id field _id is not a valid es field name anymore.

es:
  package: ElasticSearch
  options:
   client: ‘6_0::Direct'
   index_name: catmandu
   key_prefix: my_

And then make sure that your data contains that id field:

catmandu import -v MARC to es --fix 'move_field(_id, my_id)'

[nics]

The upcoming version 1.0 fully supports Elasticsearch 6.5. This is a breaking release, that maps bags to indexes, so please read the documentation and export your data first if you are upgrading. You can already test this version by installing dev release 0.9_01.