search

pystorm / streamparse

Run Python in Apache Storm topologies. Pythonic API, CLI tooling, and a topology DSL.
http://streamparse.readthedocs.io/
Apache License 2.0
1.5k stars 218 forks source link

Clojure Topology: Invalid method Code length #337

Closed colingabr closed 7 years ago

colingabr commented 7 years ago

I am writing a compiler that produces code that streamparse submits to Apache storm. With this my topology definition can grow pretty large. I understand that the JVM limit for a Method size is 65kb, which my clojure topology can surpass and causes sparse submit to fail. I've had plans to update to streamparse >3.0 when I get a chance to switch to generating the topology in python. Would this change sidestep the issue I am running into or does the python definition get transpiled into clojure so that I would face it again?

dan-blanchard commented 7 years ago

Clojure is not used under the hood for anything except Leiningen and project.clj being used to build the actual JAR that gets submitted. This should not be an issue with 3.x.