Closed ruivieira closed 6 years ago
@elmiko ptal
lgtm, tested against the rest of jiminy without issue.
i think the only thing we need to be careful of is that each rev bump on the jar file requires a new command to launch the application. we probably need to figure out either a) how to inject the jar file programmatically or b) how to stabilize the name/location of that file.
Since
py4j
method of passing data to the JVM is not very efficient, by avoid passing the model store data to the Scala helper jar for model instantiation and instead creatingJavaRDD
s on the Python side and passing those, model instatiation was reduced from200.1
seconds to0.94
seconds.