search

confluentinc / camus

Mirror of Linkedin's Camus
Apache License 2.0
0 stars 62 forks source link

AvroMessageDecoder was expecting a byte[] whereas the Message interfa… #16

Closed daedric closed 9 years ago

daedric commented 9 years ago

…ce should be used.

This fixes the exception raised with the current master:

[CamusJob] - Error for EtlKey [topic=XXXXXX partition=0leaderId= server= service= beginOffset=27 offset=28 msgSize=612 server= checksu
n: java.lang.ClassCastException: com.linkedin.camus.etl.kafka.common.KafkaMessage cannot be cast to [B
        at com.linkedin.camus.etl.kafka.mapred.EtlRecordReader.getWrappedRecord(EtlRecordReader.java:152)
        at com.linkedin.camus.etl.kafka.mapred.EtlRecordReader.nextKeyValue(EtlRecordReader.java:292)
        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:556)
        at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
        at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassCastException: com.linkedin.camus.etl.kafka.common.KafkaMessage cannot be cast to [B
        at io.confluent.camus.etl.kafka.coders.AvroMessageDecoder.decode(AvroMessageDecoder.java:41)
        at com.linkedin.camus.etl.kafka.mapred.EtlRecordReader.getWrappedRecord(EtlRecordReader.java:142)
        ... 12 more
ConfluentJenkins commented 9 years ago

Can one of the admins verify this patch?

ewencp commented 9 years ago

ok to test

ConfluentJenkins commented 9 years ago

Refer to this link for build results (access rights to CI server needed): http://jenkins.confluent.io/job/camus-pr/2/ Test PASSed.

ghost commented 9 years ago

Hey @daedric, thank you for your Pull Request.

It looks like you haven't signed our Contributor License Agreement, yet.

The purpose of a CLA is to ensure that the guardian of a project's outputs has the necessary ownership or grants of rights over all contributions to allow them to distribute under the chosen licence. Wikipedia

You can read and sign our full Contributor License Agreement here.

Once you've signed reply with [clabot:check] to prove it.

Appreciation of efforts,

clabot

daedric commented 9 years ago

I get an error when I try to submit the cla... [clabot:check]

ghost commented 9 years ago

@confluentinc It looks like @daedric just signed our Contributor License Agreement. :+1:

Always at your service,

clabot

daedric commented 9 years ago

Any news ?

ewencp commented 9 years ago

LGTM. Seems we missed this when merging in the upstream changes to switch from byte[] to Message. Thanks for the patch @daedric!