Update Redis Example: Fix AttributeError

cyhsutw commented 7 years ago

Hi,

First, thanks for building streamparse, it's an awesome project!

What Happened?

I encountered the following while running the provided redis example:

28417 [Thread-23] ERROR o.a.s.t.ShellBolt - ShellLog pid:91105, name:count_bolt 2016-11-16 14:35:32,581 - pystorm.component.count_bolt - Exception in RedisWordCountBolt.run()
Traceback (most recent call last):
  File "/Users/cyhsutw/.virtualenvs/streamparse-dev/lib/python2.7/site-packages/pystorm/component.py", line 481, in run
    self._run()
  File "/Users/cyhsutw/.virtualenvs/streamparse-dev/lib/python2.7/site-packages/pystorm/bolt.py", line 197, in _run
    self.process(tup)
  File "/private/var/folders/fl/s44qnfxs6lldrpy8w7k6rd400000gn/T/b34e7e46-7298-4068-89dc-f3f470c61998/supervisor/stormdist/wordcount_redis-1-1479278116/resources/bolts.py", line 24, in process
    self.emit([word, self.counter[word]])
AttributeError: 'RedisWordCountBolt' object has no attribute 'counter'

Full log: https://gist.github.com/cyhsutw/999a37329e79b61943540e588289c622

System Information

Python: 2.7.12, 3.5.2
storm: 1.0.2
streamparse: HEAD@bc49df
Redis: 3.2.5

Proposed Solution

Looking into the source files, I found that RedisWordCountBolt is a subclass of WordCountBolt where the counter attribute is defined. However, the initialize method in RedisWordCountBolt does not call initialize on its superclass.

While adding super(RedisWordCountBolt, self).initialize(conf, ctx) will fix the issue, I am thinking if we could just separate the implementations of these two classes since they are quite different in data storage.

I've drafted a possible implementation and would love to hear your feedback on it.

How to Test

Spin up a Redis server locally
Install dependencies of the example
Run parse run --name wordcount_redis
Watch the activities using top.sh and watch.sh

Thank you! 😄

coveralls commented 7 years ago

Coverage remained the same at 44.534% when pulling bd9a52bdf4d0d2a80467c144b21b13e77a7d92c2 on cyhsutw:cyhsutw/update-redis-example into bc49dfdc5563b93c5a28c8769a25ac6a1555eca2 on Parsely:master.

dan-blanchard commented 7 years ago

Thanks for fixing this. I must not have tried the Redis example in a long time.

pystorm / streamparse