undefined method `on_event' for ["avro", {"schema_uri"=>"D:/foo/bar.avsc"}]:Array

[mikebridge]

In logstash-1.5.4 and logstash-codec-avro logstash-codec-avro-0.1.5, and I'm trying to output to kafka. I'm getting the error

undefined method `on_event' for ["avro", {"schema_uri"=>"D:/foo/bar.avsc"}]:Array

My schema is a list of strings:

{"namespace": "PowerMTA",
 "type": "record",
 "name": "log",
 "fields": [
     {"name": "delivery", "type": "string"},
     {"name": "type", "type": "string"},
     ...
 ]
}

I'm using jruby on Windows:

D:\> jruby -version
jruby 9.0.1.0 (2.2.2) 2015-09-02 583f336 Java HotSpot(TM) 64-Bit Server VM 24.51-b03 on 1.7.0_51-b13 +jit [Windows Server 2008 R2-amd64]

I tried to update, but it doesn't install anything:

D:\logstash-1.5.4>bin\plugin update logstash-codec-avro
You are updating logstash-codec-avro to a new version 2.0.2, which may not be co
mpatible with 0.1.5. are you sure you want to proceed (Y/N)?
Y
Updating logstash-codec-avro
No plugin updated

[BarryBaum]

did you figure out a way to resolve this? I have the same exact problem.

[mikebridge]

@BarryBaum I ultimately gave up on the avro codec and switched to a JSON-only feed. I haven't had any problems with the feed since then so I haven't revisited it.

[consulthys]

Very similar error on Logstash 2.1.1. Running a configuration with a kafka output configured with the avro codec

output {
  kafka {
    codec => {
      avro => {
        schema_uri => "/tmp/schema.avsc"
      }
    }
  }
}

...simply yields the following error:

The error reported is: 
  undefined method `on_event' for #<Array:0x2a9073ef>

[jconlon]

Seeing the same thing with Logstash 2.2.2

[alextsm]

Hi, I have had the same problem with Logstash v 2.2.2. Is there any solution?

[micahlmartin]

Similar problem on 2.3.1.

[sarwarbhuiyan]

There's a slight syntax error here:

output {
  kafka {
    codec => {
      avro => {
        schema_uri => "/tmp/schema.avsc"
      }
    }
  }
}

Should be:

output {
  kafka {
    codec =>  avro {
        schema_uri => "/tmp/schema.avsc"
      }    
  }
}

After fixing that, I get another error:

Avro::IO::AvroTypeError: The datum {"message"=>"Hello world!", "@version"=>"1", "@timestamp"=>"2016-08-03T23:09:59.415Z", "type"=>"generated", "host"=>"MacBook-Pro-3.local", "sequence"=>874} is not an example of schema {"type":"record","name":"Hello","fields":[{"name":"message","type":"string"},{"name":"@version","type":"string"},{"name":"@timestamp","type":"string"},{"name":"type","type":"string"},{"name":"host","type":"string"},{"name":"sequence","type":"int"}]}

If I try it in irb with a normal hash object called event, it works just fine. Something going on here with Event.to_hash maybe.

[talevy]

I am having trouble reproducing this problem. Here is a quick demo of how this codec is meant to be used just in case something is being done differently

Avro Codec

Here is a bare-bones example of using current Logstash Avro Codec.

In this example we will use schema.avsc as our Avro schema. and pipeline.conf as the Logstash Pipeline. Our setup will take in avro encoded binary fragments in std-in and re-encode them in the output. To help with generation of the input binary fragment, I will enlist help from avro-tools.jar.

First, let's create an input json file with our event that matches our schema.

// input.json

{"message": "hello", "version": "1.0", "timestamp": "2016-10-02", "type": "_type", "host": "localhost"}

Next, we will use avro-tools to read in this file and output the binary fragment

$ java -jar avro-tools-1.8.1.jar jsontofrag --schema-file schema.avsc input.json

Using this tool as input to Logstash, we can have an end-to-end parsing of our event between Avro and Logstash representation.

$ java -jar avro-tools-1.8.1.jar jsontofrag --schema-file schema.avsc input.json | ../logstash-2.3.4/bin/logstash -f pipeline.conf 

hello1.02016-10-02
_typelocalhost

And that is it! now you see how to use the Avro Codec in Logstash!

schema.avsc

{
    "namespace": "hello.avro",
    "type": "record",
    "name": "Hello",
    "fields": [
        {"name": "message", "type": "string"},
        {"name": "version", "type": "string"},
        {"name": "timestamp", "type": "string"},
        {"name": "type", "type": "string"},
        {"name": "host", "type": "string"}
    ]
}

pipeline.conf

input {
    stdin {
        codec => avro {
            schema_uri => "schema.avsc"
        }
    }
}

output {
    stdout {
        codec => avro {
            schema_uri => "schema.avsc"
        }
    }
}

Let me know if that helps. here is the link to this blurb for external referencing: https://usecanvas.com/talk/avro-codec/0nOFqaVzO61EB96A5DFn7o