Apache Spark enhanced with native Kubernetes scheduler back-end: NOTE this repository is being ARCHIVED as all new development for the kubernetes scheduler back-end is now on https://github.com/apache/spark/
In my environment, staging server always starts very slowly, here is the logging below:
2017-12-14 06:32:13 INFO log:192 - Logging initialized @73298ms
2017-12-14 06:32:25 WARN ContextHandler:1444 - o.s.j.s.ServletContextHandler@457c9034{/,null,null} contextPath ends with /
2017-12-14 06:32:57 WARN NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-12-14 06:33:08 INFO SecurityManager:54 - Changing view acls to: root
2017-12-14 06:33:08 INFO SecurityManager:54 - Changing modify acls to: root
2017-12-14 06:33:09 INFO SecurityManager:54 - Changing view acls groups to:
2017-12-14 06:33:09 INFO SecurityManager:54 - Changing modify acls groups to:
2017-12-14 06:33:09 INFO SecurityManager:54 - SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); groups with view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set()
2017-12-14 06:33:12 INFO Server:345 - jetty-9.3.z-SNAPSHOT
2017-12-14 06:34:38 INFO ContextHandler:781 - Started o.s.j.s.ServletContextHandler@457c9034{/api,null,AVAILABLE}
2017-12-14 06:34:40 INFO AbstractConnector:270 - Started ServerConnector@9a2ec9b{HTTP/1.1,[http/1.1]}{0.0.0.0:10000}
2017-12-14 06:34:40 INFO Server:403 - Started @220792ms
2017-12-14 06:34:40 INFO ResourceStagingServer:54 - Resource staging server started on port 10000.
At 06:32:13, the first logging had printed, however, the last logging printed at 06:34:40. It took a long time to start the staging server, why? As far as I know, It's only a http server that provide a stage where users can upload or download their own jars or dependencies. I haven't read the code, can anybody explain this issue? Thanks a lot.
In my environment, staging server always starts very slowly, here is the logging below:
2017-12-14 06:32:13 INFO log:192 - Logging initialized @73298ms 2017-12-14 06:32:25 WARN ContextHandler:1444 - o.s.j.s.ServletContextHandler@457c9034{/,null,null} contextPath ends with / 2017-12-14 06:32:57 WARN NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2017-12-14 06:33:08 INFO SecurityManager:54 - Changing view acls to: root 2017-12-14 06:33:08 INFO SecurityManager:54 - Changing modify acls to: root 2017-12-14 06:33:09 INFO SecurityManager:54 - Changing view acls groups to: 2017-12-14 06:33:09 INFO SecurityManager:54 - Changing modify acls groups to: 2017-12-14 06:33:09 INFO SecurityManager:54 - SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); groups with view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set() 2017-12-14 06:33:12 INFO Server:345 - jetty-9.3.z-SNAPSHOT 2017-12-14 06:34:38 INFO ContextHandler:781 - Started o.s.j.s.ServletContextHandler@457c9034{/api,null,AVAILABLE} 2017-12-14 06:34:40 INFO AbstractConnector:270 - Started ServerConnector@9a2ec9b{HTTP/1.1,[http/1.1]}{0.0.0.0:10000} 2017-12-14 06:34:40 INFO Server:403 - Started @220792ms 2017-12-14 06:34:40 INFO ResourceStagingServer:54 - Resource staging server started on port 10000.
At 06:32:13, the first logging had printed, however, the last logging printed at 06:34:40. It took a long time to start the staging server, why? As far as I know, It's only a http server that provide a stage where users can upload or download their own jars or dependencies. I haven't read the code, can anybody explain this issue? Thanks a lot.