search

kzk / webhdfs

Ruby client for Hadoop WebHDFS
Other
81 stars 46 forks source link

json file =》 hdfs #28

Closed bashhu closed 8 years ago

bashhu commented 8 years ago

i have a question： run.log each line is a json, logstash "input"

input{
  file {
    type => "run-log"
    path => "/data/log/run.log"
    codec => "json"
  }
}

that "output"

output{
     if [type] == "run-log" {
    webhdfs {
      host => "192.168.9.7"
      user => "hue"
      path => "/user/hue/run-log"
    }
    }
}

the hadoop "hadoop fs -tail /user/hue/run-log" 2016-05-06T07:06:43.332Z app-node4.nginxs.net %{message}

My configuration is wrong? Please tell me，thinks

tagomoris commented 8 years ago

How to format data is not about this module, but about logstash output plugin. Ask the logstash community or repository of that plugin for your problem.

bashhu commented 8 years ago

think you