schedule sink map update in Rx io scheduler, because KafkaProducer.close...

[stevenzwu]

... can potentially block. see

https://issues.apache.org/jira/browse/KAFKA-1660 https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53739782

However, this is still not perfect. if close blocks on waiting sender thread, it haven't close the metrics yet. we can potentially have bad metrics.

this is the thread dump, where poller thread for persisted props got locked up //////////////////////////////////// "pollingConfigurationSource" daemon prio=10 tid=0x0000000003547800 nid=0xfe0 in Object.wait() [0x00007f335f6c8000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method)

• waiting on <0x00000002994dae30> (a org.apache.kafka.common.utils.KafkaThread) at java.lang.Thread.join(Thread.java:1281)
• locked <0x00000002994dae30> (a org.apache.kafka.common.utils.KafkaThread) at java.lang.Thread.join(Thread.java:1355) at org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:422) at com.netflix.suro.sink.kafka.KafkaSink.close(KafkaSink.java:340) at com.netflix.suro.sink.SinkManager.closeSink(SinkManager.java:78) at com.netflix.suro.sink.SinkManager.set(SinkManager.java:118) at com.netflix.chukwa.suro.SuroSinkConfigurator.buildSink(SuroSinkConfigurator.java:66)

[cloudbees-pull-request-builder]

suro-pull-requests #217 FAILURE Looks like there's a problem with this pull request

[cloudbees-pull-request-builder]

NetflixOSS » suro » suro-pull-requests #49 ABORTED

[cloudbees-pull-request-builder]

NetflixOSS » suro » suro-pull-requests #50 FAILURE Looks like there's a problem with this pull request

[cloudbees-pull-request-builder]

suro-pull-requests #218 FAILURE Looks like there's a problem with this pull request