shah-smit / observability-spring-demo

https://youtu.be/gJ9f32PyBnM
0 stars 0 forks source link

Logstash Ruby not handling float value correctly #3

Closed shah-smit closed 4 years ago

shah-smit commented 4 years ago

I have Kafak <> Logstash (then Ruby) <> Elastic

Kafak Input:

{"customerid":"smit","last_name":"shah","age":10,"height":10,"weight":100,"automated_email":false, "header": { "endpoint":"/pay"}, "transaction": { "amount":100.50, "currency" : "SGD"}}

Logstash:

 json {
        source => "message"
      }

 mutate {
        convert => { "amount" => "float" }
    }
    ruby {
        # Cancel 90% of events
        path => "/usr/local/etc/logstash/main.rb"
        script_params => { "percentage" => 0.9 }
      }
    }

Ruby

transaction = event.get('transaction')
        puts transaction
        puts transaction['amount']

This prints:

{"amount"=>0.1005e3, "currency"=>"SGD"}
0.1005e3

Why I am not getting 100.5?

shah-smit commented 4 years ago

ruby likes to print floats in exponential form. You can tell it not to using sprintf formatting strings such as %5.1f

For example: sprintf('%5.1f',transaction['amount'])

-- Badger

shah-smit commented 4 years ago
amount = "1234.50000"

def filtered_amount(amount)
    decimal_amt = amount.split('.', -1)[1]
    after_index = -1
    for i in 0..decimal_amt.length - 1
        current_char = decimal_amt[i]
        if current_char == '0' && after_index == -1
            after_index = i
        elsif current_char != '0'
            after_index = -1
        end
    end
    first_part = decimal_amt[0..after_index]
    return amount.split('.', -1)[0] << "." << first_part
end

puts "Before: " << amount

puts "After: " << filtered_amount(amount)

This would help with removing the padding zeros