Potential issue (may be an ID10T error)

[stevns]

I've logged a request on stackoverflow, where only one record gets imported when i try to designate the document id as the match key to avoid duplicates.

http://stackoverflow.com/questions/35088964/complex-rethinkdb-json-to-elasticsearch-via-logstash

Is this user error or bug?

[danielmewes]

@deontologician could you take a look at this please?

[deontologician]

@stenvns I think the issue here is that %{id} is using an id generated by logstash. In all likelihood, you'll need to create a filter in logstash to properly modify the input into the format you want. Here's an example of one that another user wrote:

https://github.com/rethinkdb/docs/issues/943#issuecomment-175250661

You may be able to modify it to make it work. Unfortunately the logstash syntax is a bit arcane, so this stuff isn't super obvious. We are planning on creating a sample filter in the documentation hopefully soon.

For now I'm going to close this since it doesn't look like a bug, but feel free to ask more questions here

[stevns]

Looking forward to the example filter. Yes, logstash is not easy :-1: Please ping a notion when/if it's there.

[deontologician]

I would just subscribe to notifications on the linked issue, that is tracking the docs for it

On Fri, Feb 5, 2016, 13:06 stevns notifications@github.com wrote:

Looking forward to the example filter. Yes, logstash is not easy [image: :-1:] Please ping a notion when/if it's there.

— Reply to this email directly or view it on GitHub https://github.com/rethinkdb/logstash-input-rethinkdb/issues/10#issuecomment-180554008 .

[stevns]

Got it :+1: