Closed sabbyanandan closed 8 years ago
Ran this and it works as advertised. Concern is additional services to run - admin + zuul + eureka and we'd need two of each for HA. Wonder if they could be combined into single runtime for fewer nodes needed in Ambari cluster config? Also, I think we should name the projects after the functionality rather than implementation - dataflow-zuul -> dataflow-proxy and dataflow-eureka -> dataflow-service-registry
Naming is probably better without zuul/eureka and I think functionality can be combined. Just wanted to have clear features in separated jars in this poc.
If going forward with these ideas, we need to think about a future of a module launcher because the black magic I did with that boot auto-config jar for eureka client kinda relates to how deps are pulled in when modules are run.
closing this poc pr as concept has been reviewed.
As a user, I have
http --server.port 9000 | cassandra
andhttp --server.port 9001 | jdbc
streams running in YARN; however, I'm unable to post data tohttp://localhost:9000
orhttp://localhost:9001
endpoints. The request fails with404
errors.Reason: In a YARN cluster, we do not know where the
http
modules (boot applications) are hosted. They're randomly distributed by YARN'sAppMaster
. It is impossible to guess the node where they're hosted.Possible solution: Leverage spring-cloud components to register where
http
modules are hosted, so they're discoverable for futures.