search

logstash-plugins / logstash-filter-mutate

Apache License 2.0
16 stars 75 forks source link

Uppercase / Lowercase for Unicode #17

Closed doubret closed 9 years ago

doubret commented 9 years ago

Please, add Unicode support (utf-8) for Uppercase and Lowercase.

Thanks

suyograo commented 9 years ago

There's a gem that supports unicode transformations. http://unicode-utils.rubyforge.org/. It is licensed under BSD

Cidan commented 9 years ago

+1

doubret commented 9 years ago

Is there any progress on this ?

duylong commented 9 years ago

+1

Do you have a temporary solution ?

doubret commented 9 years ago

I have a workaround that works for my use case but the best would be to use unicode gem.

Anyway, in my case my string is in unicode but it is compliant with ansi so i force string encoding before upcase/downcase like this (line 358 in mutate.rb) :

    event[field] = event[field].force_encoding("ISO-8859-1").downcase

Hope this helps.

duylong commented 9 years ago

Thank you, it works ! (in waiting a clean solution).

Strangely, no problem with logstash 1.4.x and no change in "mutate" plugin.

doubret commented 9 years ago

Strange, i had the same problem with logstash 1.4.

duylong commented 9 years ago

I change codec "line" to "plain" with migration to 1.5. Maybe it changed the behavior of the encoding.

suyograo commented 9 years ago

@guyboertje related to #33 probably

guyboertje commented 9 years ago

@suyograo - yes probably.

guyboertje commented 9 years ago

I can confirm this is a duplicate of #33 in terms of the cause and solution