search

zhongbiaodev / py-mysql-elasticsearch-sync

MySQL to Elasticsearch sync tool, written in Python.
MIT License
178 stars 54 forks source link

how to set mapping? #7

Closed lankaola closed 8 years ago

lankaola commented 8 years ago

I want to map es_field_name: mysql_column_name like below, but when I run es-sync config.yaml, mapping: _id: mid mid: mid cont_id: cont_id title: title director: director writer: writer actor: actor

es_field_name: mysql_column_name

It throw exception: [INFO] 2016-02-23 18:53:20,869 Start to dump from stream [ERROR] 2016-02-23 18:53:20,896 Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/es_sync/init.py", line 436, in run self._sync_from_stream() File "/usr/local/lib/python2.7/site-packages/es_sync/init.py", line 402, in _sync_from_stream for item in docs: File "/usr/local/lib/python2.7/site-packages/es_sync/init.py", line 178, in _processor action_content = {'_id': item['doc'][self.id_key]}
KeyError: 'mid something I had mapping error? The key mid exists in the mysql table.....

Windfarer commented 8 years ago

This config file must be valid YAML format file. The indent in YAML format is meaningful. Please correct the indent as the example file, and try again.

Besides, if you don't want to change the name of some column, you needn't set it in mapping fields. This mapping fields just be used to set the _id or change the fields' name on the fly.

For example, the name of one of your column is "title", and you want change it to "blog_title", so you can set blog_title: title. Or if you just want keep it the same as it in mysql, you needn't to write title:title. This tool will sync all columns to ES in default situation.