search

framed-data / overseer

Overseer is a library for building and running data pipelines in Clojure.
Eclipse Public License 1.0
97 stars 10 forks source link

AOT-compile Overseer.runner #65

Closed andrewberls closed 8 years ago

andrewberls commented 8 years ago

overseer.runner is used by the launcher script included with Overseer. It used to be AOT-compiled, but this was disabled as AOT compilation causes issues with applications that need to modify dependency versions

See https://github.com/onyx-platform/onyx/issues/339. This re-enables AOT-compilation for that namespace, but uses the same fix strategy of requiring namespaces at runtime and resolving symbols which should address the issue. This probably comes at some minor perf cost, but is only run once at runtime.

elliot42 commented 8 years ago

If we're going to do these gymnastics, I'd recommend moving as much of this as possible into a different namespace, and then having the gymnastics only do require resolve and then call one thing from the other ns.

andrewberls commented 8 years ago

I dont know if three dependencies qualifies as over-the-top gymnastics. Besides I'm not sure that would even work given the transitive dependency AOT thing - if overseer.runner just does one call out to overseer.runner.internal or whatever and that namespace requires things, those will all be AOT compiled leading to the original problem

Edit: with a runtime call that would work, but as mentioned I dunno if its worth it

elliot42 commented 8 years ago

Eh? See inline

elliot42 commented 8 years ago

:ok: I think a separate namespace would make this cleaner, but as long as the complexity doesn't increase in here should be fine

andrewberls commented 8 years ago

Yeah I don't think another namespace is the right call yet, but if this does grow I agree that's a good idea