After this i have copied a json file to the target folder. But, nothing is happening.
Below is the json and avsc files.
Json:
{"username":"miguno","tweet":"Rock: Nerf paper, scissors is fine.","timestamp": 1366150681 }
Avsc file:
{
"type" : "record",
"name" : "twitter_schema",
"namespace" : "com.miguno.avro",
"fields" : [ {
"name" : "username",
"type" : "string",
"doc" : "Name of the user account on Twitter.com"
}, {
"name" : "tweet",
"type" : "string",
"doc" : "The content of the user's Twitter message"
}, {
"name" : "timestamp",
"type" : "long",
"doc" : "Unix epoch time in seconds"
} ],
"doc:" : "A basic schema for storing Twitter messages"
}
1) Can we use codec with output format.? 2) Why it is unable to take any data.? I have movied a new file to the folder. At least it is not throwing any error.
I am trying to write the file using avro codec in logstash. Below is my simple configuration:
I have installed avro plugin in logstash and tried below configuration. Logstash just says nothing. It is getting started, and nothing after that.
input { file { path => "/Users/srini/Downloads/imp/2016-01-22/*.json" start_position => "beginning" } }
output { file { path => "/Users/srini/Downloads/imp/2016-01-32/test-%{+YYYY-MM-dd}.avro" codec => avro {schema_uri => "/Users/srini/Sri_Search/av.avsc"} } stdout { codec => rubydebug }
}
After this i have copied a json file to the target folder. But, nothing is happening.
Below is the json and avsc files.
Json:
{"username":"miguno","tweet":"Rock: Nerf paper, scissors is fine.","timestamp": 1366150681 } Avsc file:
{ "type" : "record", "name" : "twitter_schema", "namespace" : "com.miguno.avro", "fields" : [ { "name" : "username", "type" : "string", "doc" : "Name of the user account on Twitter.com" }, { "name" : "tweet", "type" : "string", "doc" : "The content of the user's Twitter message" }, { "name" : "timestamp", "type" : "long", "doc" : "Unix epoch time in seconds" } ], "doc:" : "A basic schema for storing Twitter messages" } 1) Can we use codec with output format.? 2) Why it is unable to take any data.? I have movied a new file to the folder. At least it is not throwing any error.