Closed pinguo-xudianyang closed 8 years ago
many these INFO LOG for console output,what does this matter?
I'm not clear what your question is? what is the problem? This looks normal.
ok,I setup all dependency in a manual way, such as hadoop,kafka,zookeeper,spark. I keep on.
kafka-brokers = "hadoop-slave-1:9092,hadoop-slave-2:9092,phpboy:9092" zk-servers = "10.200.143.204:4180,10.47.100.215:4180,10.46.75.247:4180" hdfs-base = "hdfs://hadoop-slave-1:9000/Oryx"
oryx { id = "ALSExample" als { rescorer-provider-class = null } input-topic { broker = ${kafka-brokers} lock = { master = ${zk-servers} } } update-topic { broker = ${kafka-brokers} lock = { master = ${zk-servers} } } batch { streaming { generation-interval-sec = 300 num-executors = 4 executor-cores = 8 executor-memory = "4g" } update-class = "com.cloudera.oryx.app.batch.mllib.als.ALSUpdate" storage { data-dir = ${hdfs-base}"/data/" model-dir = ${hdfs-base}"/model/" } ui { port = 4040 } } speed { model-manager-class = "com.cloudera.oryx.app.speed.als.ALSSpeedModelManager" ui { port = 4041 } } serving { model-manager-class = "com.cloudera.oryx.app.serving.als.model.ALSServingModelManager" application-resources = "com.cloudera.oryx.app.serving,com.cloudera.oryx.app.serving.als" api { port = 8080 } } }
2016-04-11 11:17:54,194 INFO ConsumerFetcherManager:68 [ConsumerFetcherManager-1460344661151] Added fetcher for partitions ArrayBuffer() 2016-04-11 11:17:54,401 INFO VerifiableProperties:68 Verifying properties 2016-04-11 11:17:54,402 INFO VerifiableProperties:68 Property client.id is overridden to OryxGroup-ServingLayer-1460344660862 2016-04-11 11:17:54,402 INFO VerifiableProperties:68 Property metadata.broker.list is overridden to localhost:9092,localhost:9092,localhost:9092 2016-04-11 11:17:54,402 INFO VerifiableProperties:68 Property request.timeout.ms is overridden to 30000 2016-04-11 11:17:54,403 INFO ClientUtils$:68 Fetching metadata from broker id:1,host:localhost,port:9092 with correlation id 56 for 1 topic(s) Set(OryxUpdate) 2016-04-11 11:17:54,403 INFO SyncProducer:68 Connected to localhost:9092 for producing 2016-04-11 11:17:54,407 INFO SyncProducer:68 Disconnecting from localhost:9092 2016-04-11 11:17:54,407 INFO ConsumerFetcherManager:68 [ConsumerFetcherManager-1460344661151] Added fetcher for partitions ArrayBuffer() 2016-04-11 11:17:54,623 INFO VerifiableProperties:68 Verifying properties 2016-04-11 11:17:54,624 INFO VerifiableProperties:68 Property client.id is overridden to OryxGroup-ServingLayer-1460344660862 2016-04-11 11:17:54,624 INFO VerifiableProperties:68 Property metadata.broker.list is overridden to localhost:9092,localhost:9092,localhost:9092 2016-04-11 11:17:54,624 INFO VerifiableProperties:68 Property request.timeout.ms is overridden to 30000 2016-04-11 11:17:54,624 INFO ClientUtils$:68 Fetching metadata from broker id:2,host:localhost,port:9092 with correlation id 57 for 1 topic(s) Set(OryxUpdate) 2016-04-11 11:17:54,626 INFO SyncProducer:68 Connected to localhost:9092 for producing 2016-04-11 11:17:54,628 INFO SyncProducer:68 Disconnecting from localhost:9092 2016-04-11 11:17:54,629 INFO ConsumerFetcherManager:68 [ConsumerFetcherManager-1460344661151] Added fetcher for partitions ArrayBuffer() 2016-04-11 11:17:54,835 INFO VerifiableProperties:68 Verifying properties 2016-04-11 11:17:54,835 INFO VerifiableProperties:68 Property client.id is overridden to OryxGroup-ServingLayer-1460344660862 2016-04-11 11:17:54,835 INFO VerifiableProperties:68 Property metadata.broker.list is overridden to localhost:9092,localhost:9092,localhost:9092 2016-04-11 11:17:54,835 INFO VerifiableProperties:68 Property request.timeout.ms is overridden to 30000 2016-04-11 11:17:54,836 INFO ClientUtils$:68 Fetching metadata from broker id:0,host:localhost,port:9092 with correlation id 58 for 1 topic(s) Set(OryxUpdate) 2016-04-11 11:17:54,836 INFO SyncProducer:68 Connected to localhost:9092 for producing 2016-04-11 11:17:54,838 INFO SyncProducer:68 Disconnecting from localhost:9092 2016-04-11 11:17:54,839 INFO ConsumerFetcherManager:68 [ConsumerFetcherManager-1460344661151] Added fetcher for partitions ArrayBuffer() 2016-04-11 11:17:55,047 INFO VerifiableProperties:68 Verifying properties 2016-04-11 11:17:55,047 INFO VerifiableProperties:68 Property client.id is overridden to OryxGroup-ServingLayer-1460344660862 2016-04-11 11:17:55,047 INFO VerifiableProperties:68 Property metadata.broker.list is overridden to localhost:9092,localhost:9092,localhost:9092 2016-04-11 11:17:55,048 INFO VerifiableProperties:68 Property request.timeout.ms is overridden to 30000 2016-04-11 11:17:55,048 INFO ClientUtils$:68 Fetching metadata from broker id:1,host:localhost,port:9092 with correlation id 59 for 1 topic(s) Set(OryxUpdate) 2016-04-11 11:17:55,048 INFO SyncProducer:68 Connected to localhost:9092 for producing 2016-04-11 11:17:55,051 INFO SyncProducer:68 Disconnecting from localhost:9092