Add support for adding externally defined spouts and bolts to pyleus topologies

[mzbyszynski]

Hello!

Awesome framework! I've really enjoyed hacking around with it so far so thank you for that!

I'm working on integrating a new pyleus topology into an existing application that receives input from amqp. It would be great to have a way to add the storm-amqp spout (and/or others) to a pyleus topology the way that the kafka spout is supported.

I was thinking of being able to add something like this to the pyleus.conf:

[Plugins]
amqp=com.my.custom.AMQPSpoutProvider
amqp_lib=/path/to/my/jars_dir
otherone=com.some.other.SpoutProvider
otherone_lib=/path/to/another/jars_dir

and that referenced class com.my.custom.AMQPSpoutProvider extending some interface like:

public class AMQPSpoutProvider implements SpoutProvider {
    @Overrides
    public IRichSpout provideSpout(final TopologyBuilder builder, final SpoutSpec spec) {
       ...

and would probably look a lot like the handleKafkaSpout() method in PyleusTopologyBuilder.

If this seems like something that you would like to include in your project I would be stoked to submit a PR as a first draft...

[ecanzonieri]

Personally, I like the idea of being able to plug custom java classes to a pyleus topology. I guess we should also have the possibility to pass configuration parameters directly to the plugin, similarly to what we do with the KafkaSpout.