search

Yelp / pyleus

Pyleus is a Python framework for developing and launching Storm topologies.
Apache License 2.0
403 stars 113 forks source link

Joining forces with streamparse devs for common pystorm package #158

Open dan-blanchard opened 9 years ago

dan-blanchard commented 9 years ago

I'm one of the main streamparse devs, and as of about five minutes ago, I pulled out our storm sub-package from streamparse (which is very similar to your storm sub-package) into a new project just called pystorm. Our team's hope is that we can simplify each others' lives when it comes to keeping up with new Storm releases and any changes to the Multi-Lang protocol by collaborating on a joint Multi-Lang-only project.

Both our projects' current storm sub-packages have features that the other's lacks: you have a nice mechanism for supporting custom serializers, whereas we have batching bolts and Storm 0.10.0+ support. Working together and combining the packages seems like a win-win to me.

I created a new neutral GitHub organization, pystorm, in the hopes that we could both be equal contributors to the project, so please let me know if you all are interested, and I'll add you to the organization.

You can check out the pystorm API docs here and compare to yours if you want to see what is different. Since both our projects are Apache-licensed, I'm going to create a PR later today that adds your customer serialization code in, which should make any transitioning much simpler for pyleus.

mzbyszynski commented 9 years ago

Sounds like a great idea to me [a Pyleus user]. I don't really see a downside?

westover commented 8 years ago

+1

dan-blanchard commented 8 years ago

FYI, we're about to release our first streamparse release that relies on pystorm and I did end up getting custom serialization support added in that's based on the pyleus code. I believe it should function as a drop-in replacement for your storm subpackage at this point, but there might be minor differences in the API.

We would still love to have you all contribute to pystorm. Please let me know if you're interested in collaborating.