Open kbruegge opened 6 years ago
What Java implementation of the msgpack serializer are you using? There is no official one yet, and the one from the now-abandoned Pyleus project would cause mysterious issues for us when we used it under load.
Hmm I'm guessed we are a bit confused about how to this at all. We thought changing the serializer in the config.json would suffice. I cant seem to find the docs on how to do that properly. We are trying to serialize small numpy arrays and scalars. We found this PR here issue and thought msgpack would be part of pystorm now. https://github.com/pystorm/pystorm/pull/1
Yeah, the serializer
setting isn't really documented in streamparse (or pystorm) because we were waiting on official upstream support for msgpack that has yet to materialize. When we first added it to pystorm, Pyleus was still under active development and they had created the necessary Java version of things, but that code is very out of date at this point and doesn't work on Storm 1.x.
There's an old Storm PR (apache/Storm#1136), but someone else will probably need to pick up the work there and make it use a more modern version of the Java messagepack library.
Hello,
I'm trying to use the msgpack serializer with streamparse. Here is my current configuration:
I also added a dependency tot he project.clj.
I get an exception when executing the topology using
sparse run
:Thanks for your help.