Recommendation requests are not dispatched using HTTP

[morellodev]

Hi,

I'm facing with this exception while computing environment is waiting for recommendation requests during the TEST phase (recommendation-manager.log file):

13 Jun 2016 09:32:48,494 ERROR [PollableSourceRunner-KafkaSource-r1] (org.apache.flume.source.kafka.KafkaSource.process:153)  - KafkaSource EXCEPTION, {}
java.lang.NullPointerException
    at org.apache.flume.instrumentation.MonitoredCounterGroup.increment(MonitoredCounterGroup.java:261)
    at org.apache.flume.instrumentation.kafka.KafkaSourceCounter.incrementKafkaEmptyCount(KafkaSourceCounter.java:49)
    at org.apache.flume.source.kafka.KafkaSource.process(KafkaSource.java:146)
    at org.apache.flume.source.PollableSourceRunner$PollingRunner.run(PollableSourceRunner.java:139)
    at java.lang.Thread.run(Thread.java:745)

Has somebody any idea to solve this issue? I'm using HTTP as communication protocol between computing environment and orchestrator.

[morellodev]

Hi András, this bug seems to be related to this one.

I hope this helps.

[andras-sereny]

A strange error indeed. For me, running

sudo /usr/bin/java -Xmx20m -Didomaar.url=https://raw.githubusercontent.com/crowdrec/data sets/master/01.MovieTweetings/datasets/snapshots10K/evaluation/test/data.dat -Didomaar.sourceType=file -cp '/vagrant/flume-config/log4j/tes t:/opt/apache/flume/lib/:/opt/apache/flume/plugins.d/idomaar/lib/:/opt/apache/flume/plugins.d/idomaar/libext/:/lib/_' -Djava.library.path = org.apache.flume.node.Application --name a1 --conf-file /vagrant/flume-config/config/generated/idomaar-TO-kafka-test.conf

on the datastream manager sometimes succeeds and sometimes fails with the java.lang.NoClassDefFoundError: Could not initialize class com.yammer.metrics.Metrics error. Apparently, the Metrics class is on the classpath.

Once I hack a call of com.yammer.metrics.Metrics into IdomaarSource initialization, the command succeeds consistently. I'm guessing this is some strange race condition in the Metrics class static initalization. If all is fine, I'll commit the hack on Monday.

[andras-sereny]

Apparently, Metrics is being initialized rather late, when the JVM is already shutting down, thus it fails to add a shutdown hook ... I'm not sure why. Timely initialization should fix this.

[morellodev]

Ok András, can you commit the bug fix when you are ready? Thanks.

However, the bug you have just mentioned is different from the one I've opened this ticket for. Can you please check Flume's log files?

[andras-sereny]

fixed