Yelp / pyleus

Pyleus is a Python framework for developing and launching Storm topologies.
Apache License 2.0
404 stars 113 forks source link

Problem for the example of exclamation_topology #79

Open xiaqt78 opened 9 years ago

xiaqt78 commented 9 years ago

I have modified as follows:

 def process_tuple(self, tup):
      log.debug(tup)
      if tup.values != [] :+1: 
         word = tup.values[0] + '!!!'
         log.debug(word)
         self.emit((word,), anchors = tup)

My question: if the class derived from SimpleBolt, it only logged 2 lines. if the class derived from Bolt, it will logged 30 seconds. I love Pyleus to write topology for storm. Pyleus's idea can make me emphasize my work to be solved. But now I don't know why the bolt only work a little time. Please help me!

ecanzonieri commented 9 years ago

did you call self.ack(tup) when deriving from Bolt?

poros commented 9 years ago

@xiaqt78 I have edited the formatting of your comment to better visualize the code, I hope you're fine with that.

I can spot two problems with your code:

  1. @ecanzonieri is right, when you inherit from Bolt you need to call ack() after you are done with processing the tuple. This may be the reason why your bolt blocks after 30 seconds (hangs/out of memory)
  2. You need to anchor non [tup], not tup. This behaviour is inherited by Storm, since you can anchor on more than a tuple at once when emitting, because the output tuple may indeed depend on many tuples at once. It's weird that it didn't break.

Both these two points are covered in this section of the wiki: http://yelp.github.io/pyleus/reliability.html#anchor-tuples-in-your-bolt

Fix that and let us know if it works.

xiaqt78 commented 9 years ago

Now,I run the examples of Pyleus and my own topology built by Pyleus on storm-0.9.2. And it run perfectly. I look forward to Pyleus of next version that can work well with storm-0.9.3. Thanks for your works.