start presto yarn error

[JasonBian]

when I start the presto-yarn, I got error of the server.log: 2016-11-09T15:32:44.132+0800 ERROR Discovery-0 io.airlift.discovery.client.CachingServiceSelector Cannot connect to discovery server for refresh (presto/general): Lookup of presto failed for http://bi-04:1081/v1/service/presto/general

[kokosing]

Do you use labels? Are you sure that coordinator is deployed on bi-04? When labels are not configured then coordinator can be allocated on any node and so it is hard to configure worker to point where it should connect.

[JasonBian]

@kokosing I didn't use labels

[kokosing]

I would strongly advice you to use labels at least for coordinator. That way it will be easy to configure workers and you will know where to connect to issue the query.

@anusudarsan Do you know how to configure workers where no labels are used?

[anusudarsan]

Even if no labels are used, if you are using the default .json slider config files, it should auto-magically set the coordinator host. For eg: if you have this line https://github.com/prestodb/presto-yarn/blob/master/presto-yarn-package/src/main/resources/appConfig.json#L13 in your appConfig.json (also, provided the resources.json has the coordinator named "COORDINATOR" - https://github.com/prestodb/presto-yarn/blob/master/presto-yarn-package/src/main/resources/resources-multinode.json#L11), then site.global.coordinator_host will be set with the actual hostname of the coordinator during runtime. And from there the worker node's configuration will be having the appropriate coordinator host.

[JasonBian]

@anusudarsan yeah, I'm using the default .json slider config files,actually I found that it auto set the coordinator host from the server.log.But I still encounter this issue,can't start presto server.

[JasonBian]

2016-11-10T16:56:19.552+0800 INFO main io.airlift.log.Logging Logging to stderr 2016-11-10T16:56:19.555+0800 INFO main Bootstrap Loading configuration 2016-11-10T16:56:19.696+0800 INFO main Bootstrap Initializing logging 2016-11-10T16:56:20.237+0800 WARN main Bootstrap Warning: Configuration property 'plugin.config-dir' has been replaced. Use 'catalog.config-dir' instead. 2016-11-10T16:56:20.299+0800 INFO main Bootstrap PROPERTY DEFAULT RUNTIME DESCRIPTION 2016-11-10T16:56:20.299+0800 INFO main Bootstrap discovery-server.enabled false false 2016-11-10T16:56:20.299+0800 INFO main Bootstrap event.max-output-stage-size 16MB 16MB 2016-11-10T16:56:20.299+0800 INFO main Bootstrap query.client.timeout 5.00m 5.00m 2016-11-10T16:56:20.299+0800 INFO main Bootstrap query.initial-hash-partitions 100 100 2016-11-10T16:56:20.299+0800 INFO main Bootstrap query.max-concurrent-queries 1000 1000 2016-11-10T16:56:20.299+0800 INFO main Bootstrap query.max-history 100 100 2016-11-10T16:56:20.300+0800 INFO main Bootstrap query.max-queued-queries 5000 5000 2016-11-10T16:56:20.300+0800 INFO main Bootstrap query.min-expire-age 15.00m 15.00m 2016-11-10T16:56:20.300+0800 INFO main Bootstrap query.execution-policy all-at-once all-at-once 2016-11-10T16:56:20.300+0800 INFO main Bootstrap query.manager-executor-pool-size 5 5 2016-11-10T16:56:20.300+0800 INFO main Bootstrap query.max-cpu-time 1000000000.00d 1000000000.00d 2016-11-10T16:56:20.300+0800 INFO main Bootstrap query.max-run-time 100.00d 100.00d 2016-11-10T16:56:20.300+0800 INFO main Bootstrap query.queue-config-file null null 2016-11-10T16:56:20.300+0800 INFO main Bootstrap query.remote-task.max-callback-threads 1000 1000 2016-11-10T16:56:20.300+0800 INFO main Bootstrap query.remote-task.min-error-duration 2.00m 2.00m 2016-11-10T16:56:20.300+0800 INFO main Bootstrap query.schedule-split-batch-size 1000 1000 2016-11-10T16:56:20.300+0800 INFO main Bootstrap task.client.timeout 2.00m 2.00m 2016-11-10T16:56:20.300+0800 INFO main Bootstrap task.http-response-threads 100 100 2016-11-10T16:56:20.300+0800 INFO main Bootstrap task.http-timeout-threads 3 3 2016-11-10T16:56:20.300+0800 INFO main Bootstrap task.info.max-age 15.00m 15.00m 2016-11-10T16:56:20.301+0800 INFO main Bootstrap task.info-update-interval 200.00ms 200.00ms Interval between updating task data 2016-11-10T16:56:20.301+0800 INFO main Bootstrap task.initial-splits-per-node 48 48 2016-11-10T16:56:20.301+0800 INFO main Bootstrap task.max-index-memory 64MB 64MB 2016-11-10T16:56:20.301+0800 INFO main Bootstrap driver.max-page-partitioning-buffer-size 32MB 32MB 2016-11-10T16:56:20.301+0800 INFO main Bootstrap task.max-partial-aggregation-memory 16MB 16MB 2016-11-10T16:56:20.301+0800 INFO main Bootstrap task.max-worker-threads 48 48 2016-11-10T16:56:20.301+0800 INFO main Bootstrap task.min-drivers 96 96 2016-11-10T16:56:20.301+0800 INFO main Bootstrap sink.new-implementation false false Experimental: use new output buffer implementations 2016-11-10T16:56:20.301+0800 INFO main Bootstrap task.operator-pre-allocated-memory 16MB 16MB 2016-11-10T16:56:20.301+0800 INFO main Bootstrap task.share-index-loading false false 2016-11-10T16:56:20.301+0800 INFO main Bootstrap sink.max-buffer-size 32MB 32MB 2016-11-10T16:56:20.301+0800 INFO main Bootstrap task.split-concurrency-adjustment-interval 100.00ms 100.00ms 2016-11-10T16:56:20.302+0800 INFO main Bootstrap task.status-refresh-max-wait 1.00s 1.00s 2016-11-10T16:56:20.302+0800 INFO main Bootstrap task.concurrency 1 1 Default number of local parallel jobs per worker 2016-11-10T16:56:20.302+0800 INFO main Bootstrap task.cpu-timer-enabled true true 2016-11-10T16:56:20.302+0800 INFO main Bootstrap task.task-notification-threads 5 5 Number of threads used for internal task event notifications 2016-11-10T16:56:20.302+0800 INFO main Bootstrap task.verbose-stats false false 2016-11-10T16:56:20.302+0800 INFO main Bootstrap task.writer-count 1 1 Number of writers per task 2016-11-10T16:56:20.302+0800 INFO main Bootstrap node-scheduler.include-coordinator true true 2016-11-10T16:56:20.302+0800 INFO main Bootstrap node-scheduler.max-pending-splits-per-node-per-stage 10 10 2016-11-10T16:56:20.302+0800 INFO main Bootstrap node-scheduler.max-splits-per-node 100 100 2016-11-10T16:56:20.302+0800 INFO main Bootstrap node-scheduler.min-candidates 10 10 2016-11-10T16:56:20.302+0800 INFO main Bootstrap node-scheduler.multiple-tasks-per-node-enabled false false Allow nodes to be selected multiple times by the node scheduler, in a single stage 2016-11-10T16:56:20.302+0800 INFO main Bootstrap node-scheduler.network-topology legacy legacy 2016-11-10T16:56:20.303+0800 INFO main Bootstrap query.low-memory-killer.enabled false false Enable low memory killer 2016-11-10T16:56:20.303+0800 INFO main Bootstrap query.low-memory-killer.delay 5.00m 5.00m Delay between cluster running low on memory and invoking killer 2016-11-10T16:56:20.303+0800 INFO main Bootstrap query.max-memory 20GB 10GB 2016-11-10T16:56:20.303+0800 INFO main Bootstrap query.max-memory-per-node 107374182.40B 600MB 2016-11-10T16:56:20.303+0800 INFO main Bootstrap resources.reserved-system-memory 429496729.60B 429496729.60B 2016-11-10T16:56:20.303+0800 INFO main Bootstrap catalog.config-dir etc/catalog /var/lib/presto/etc/catalog 2016-11-10T16:56:20.303+0800 INFO main Bootstrap catalog.disabled-catalogs null null 2016-11-10T16:56:20.303+0800 INFO main Bootstrap exchange.client-threads 25 25 2016-11-10T16:56:20.303+0800 INFO main Bootstrap exchange.concurrent-request-multiplier 3 3 2016-11-10T16:56:20.303+0800 INFO main Bootstrap exchange.max-buffer-size 32MB 32MB 2016-11-10T16:56:20.303+0800 INFO main Bootstrap exchange.max-response-size 16MB 16MB 2016-11-10T16:56:20.303+0800 INFO main Bootstrap exchange.min-error-duration 1.00m 1.00m 2016-11-10T16:56:20.304+0800 INFO main Bootstrap code-cache-check-interval 20.00s 20.00s How often to check if the code cache need to be collected 2016-11-10T16:56:20.304+0800 INFO main Bootstrap plugin.dir plugin plugin 2016-11-10T16:56:20.304+0800 INFO main Bootstrap maven.repo.local /var/lib/hadoop-yarn/.m2/repository /var/lib/hadoop-yarn/.m2/repository 2016-11-10T16:56:20.304+0800 INFO main Bootstrap maven.repo.remote [http://repo1.maven.org/maven2/] [http://repo1.maven.org/maven2/] 2016-11-10T16:56:20.304+0800 INFO main Bootstrap plugin.config-dir etc /var/lib/presto/etc/catalog 2016-11-10T16:56:20.304+0800 INFO main Bootstrap plugin.bundles null null 2016-11-10T16:56:20.304+0800 INFO main Bootstrap coordinator true false 2016-11-10T16:56:20.304+0800 INFO main Bootstrap datasources null null 2016-11-10T16:56:20.304+0800 INFO main Bootstrap shutdown.grace-period 2.00m 2.00m 2016-11-10T16:56:20.304+0800 INFO main Bootstrap http.include-exception-in-response true true 2016-11-10T16:56:20.304+0800 INFO main Bootstrap presto.version null null 2016-11-10T16:56:20.304+0800 INFO main Bootstrap http.server.authentication.enabled false false 2016-11-10T16:56:20.304+0800 INFO main Bootstrap http.authentication.krb5.config null null 2016-11-10T16:56:20.305+0800 INFO main Bootstrap http.server.authentication.krb5.keytab null null 2016-11-10T16:56:20.305+0800 INFO main Bootstrap http.server.authentication.krb5.service-name null null 2016-11-10T16:56:20.305+0800 INFO main Bootstrap colocated-joins-enabled false false Experimental: Use a colocated join when possible 2016-11-10T16:56:20.305+0800 INFO main Bootstrap optimizer.dictionary-aggregation false false 2016-11-10T16:56:20.305+0800 INFO main Bootstrap distributed-index-joins-enabled false false 2016-11-10T16:56:20.305+0800 INFO main Bootstrap distributed-joins-enabled true true 2016-11-10T16:56:20.305+0800 INFO main Bootstrap experimental-syntax-enabled false false 2016-11-10T16:56:20.305+0800 INFO main Bootstrap deprecated.legacy-array-agg false false 2016-11-10T16:56:20.305+0800 INFO main Bootstrap optimizer.optimize-hash-generation true true 2016-11-10T16:56:20.305+0800 INFO main Bootstrap optimizer.optimize-metadata-queries false false 2016-11-10T16:56:20.305+0800 INFO main Bootstrap optimizer.optimize-single-distinct true true 2016-11-10T16:56:20.305+0800 INFO main Bootstrap optimizer.processing-optimization disabled disabled 2016-11-10T16:56:20.305+0800 INFO main Bootstrap optimizer.push-table-write-through-union true true 2016-11-10T16:56:20.305+0800 INFO main Bootstrap re2j.dfa-retries 5 5 2016-11-10T16:56:20.305+0800 INFO main Bootstrap re2j.dfa-states-limit 2147483647 2147483647 2016-11-10T16:56:20.305+0800 INFO main Bootstrap redistribute-writes true true 2016-11-10T16:56:20.305+0800 INFO main Bootstrap regex-library JONI JONI 2016-11-10T16:56:20.305+0800 INFO main Bootstrap resource-group-manager file file 2016-11-10T16:56:20.306+0800 INFO main Bootstrap experimental.resource-groups-enabled false false 2016-11-10T16:56:20.306+0800 INFO main Bootstrap compiler.interpreter-enabled false false 2016-11-10T16:56:20.306+0800 INFO main Bootstrap transaction.idle-check-interval 1.00m 1.00m Time interval between idle transactions checks 2016-11-10T16:56:20.306+0800 INFO main Bootstrap transaction.idle-timeout 5.00m 5.00m Amount of time before an inactive transaction is considered expired 2016-11-10T16:56:20.306+0800 INFO main Bootstrap transaction.max-finishing-concurrency 1 1 Maximum parallelism for committing or aborting a transaction 2016-11-10T16:56:20.306+0800 INFO main Bootstrap discovery.uri null http://192.168.10.86:9081 Discovery service base URI 2016-11-10T16:56:20.306+0800 INFO main Bootstrap service-inventory.uri null null Service inventory base URI 2016-11-10T16:56:20.306+0800 INFO main Bootstrap service-inventory.update-interval 10.00s 10.00s Service inventory update interval 2016-11-10T16:56:20.306+0800 INFO main Bootstrap discovery.collector.pool general general 2016-11-10T16:56:20.306+0800 INFO main Bootstrap discovery.presto.pool general general 2016-11-10T16:56:20.306+0800 INFO main Bootstrap node-manager.http-client.authentication.enabled false false Enable client authentication 2016-11-10T16:56:20.306+0800 INFO main Bootstrap node-manager.http-client.connect-timeout 1.00s 1.00s 2016-11-10T16:56:20.306+0800 INFO main Bootstrap node-manager.http-client.http2.enabled false false Enable the HTTP/2 transport 2016-11-10T16:56:20.306+0800 INFO main Bootstrap node-manager.http-client.idle-timeout 30.00s 30.00s 2016-11-10T16:56:20.306+0800 INFO main Bootstrap node-manager.http-client.keep-alive-interval null null 2016-11-10T16:56:20.306+0800 INFO main Bootstrap node-manager.http-client.authentication.krb5.principal null null Set kerberos client principal 2016-11-10T16:56:20.306+0800 INFO main Bootstrap node-manager.http-client.authentication.krb5.remote-service-name null null Set kerberos service principal name 2016-11-10T16:56:20.307+0800 INFO main Bootstrap node-manager.http-client.key-store-password null null 2016-11-10T16:56:20.307+0800 INFO main Bootstrap node-manager.http-client.key-store-path null null 2016-11-10T16:56:20.307+0800 INFO main Bootstrap node-manager.http-client.max-connections 200 200 2016-11-10T16:56:20.307+0800 INFO main Bootstrap node-manager.http-client.max-connections-per-server 20 20 2016-11-10T16:56:20.307+0800 INFO main Bootstrap node-manager.http-client.max-content-length 16MB 16MB 2016-11-10T16:56:20.307+0800 INFO main Bootstrap node-manager.http-client.max-requests-queued-per-destination 1024 1024 2016-11-10T16:56:20.307+0800 INFO main Bootstrap node-manager.http-client.request-timeout 10.00s 10.00s 2016-11-10T16:56:20.307+0800 INFO main Bootstrap node-manager.http-client.socks-proxy null null 2016-11-10T16:56:20.307+0800 INFO main Bootstrap node-manager.http-client.trust-store-password null null 2016-11-10T16:56:20.307+0800 INFO main Bootstrap node-manager.http-client.trust-store-path null null 2016-11-10T16:56:20.307+0800 INFO main Bootstrap exchange.http-client.authentication.enabled false false Enable client authentication 2016-11-10T16:56:20.307+0800 INFO main Bootstrap exchange.http-client.connect-timeout 1.00s 1.00s 2016-11-10T16:56:20.307+0800 INFO main Bootstrap exchange.http-client.http2.enabled false false Enable the HTTP/2 transport 2016-11-10T16:56:20.307+0800 INFO main Bootstrap exchange.http-client.idle-timeout 30.00s 30.00s 2016-11-10T16:56:20.307+0800 INFO main Bootstrap exchange.http-client.keep-alive-interval null null 2016-11-10T16:56:20.307+0800 INFO main Bootstrap exchange.http-client.authentication.krb5.principal null null Set kerberos client principal 2016-11-10T16:56:20.307+0800 INFO main Bootstrap exchange.http-client.authentication.krb5.remote-service-name null null Set kerberos service principal name 2016-11-10T16:56:20.307+0800 INFO main Bootstrap exchange.http-client.key-store-password null null 2016-11-10T16:56:20.308+0800 INFO main Bootstrap exchange.http-client.key-store-path null null 2016-11-10T16:56:20.308+0800 INFO main Bootstrap exchange.http-client.max-connections 200 200 2016-11-10T16:56:20.308+0800 INFO main Bootstrap exchange.http-client.max-connections-per-server 250 250 2016-11-10T16:56:20.308+0800 INFO main Bootstrap exchange.http-client.max-content-length 16MB 16MB 2016-11-10T16:56:20.308+0800 INFO main Bootstrap exchange.http-client.max-requests-queued-per-destination 1024 1024 2016-11-10T16:56:20.308+0800 INFO main Bootstrap exchange.http-client.request-timeout 10.00s 10.00s 2016-11-10T16:56:20.308+0800 INFO main Bootstrap exchange.http-client.socks-proxy null null 2016-11-10T16:56:20.308+0800 INFO main Bootstrap exchange.http-client.trust-store-password null null 2016-11-10T16:56:20.308+0800 INFO main Bootstrap exchange.http-client.trust-store-path null null 2016-11-10T16:56:20.308+0800 INFO main Bootstrap discovery.http-client.authentication.enabled false false Enable client authentication 2016-11-10T16:56:20.308+0800 INFO main Bootstrap discovery.http-client.connect-timeout 1.00s 1.00s 2016-11-10T16:56:20.308+0800 INFO main Bootstrap discovery.http-client.http2.enabled false false Enable the HTTP/2 transport 2016-11-10T16:56:20.308+0800 INFO main Bootstrap discovery.http-client.idle-timeout 1.00m 1.00m 2016-11-10T16:56:20.308+0800 INFO main Bootstrap discovery.http-client.keep-alive-interval null null 2016-11-10T16:56:20.308+0800 INFO main Bootstrap discovery.http-client.authentication.krb5.principal null null Set kerberos client principal 2016-11-10T16:56:20.308+0800 INFO main Bootstrap discovery.http-client.authentication.krb5.remote-service-name null null Set kerberos service principal name 2016-11-10T16:56:20.308+0800 INFO main Bootstrap discovery.http-client.key-store-password null null 2016-11-10T16:56:20.309+0800 INFO main Bootstrap discovery.http-client.key-store-path null null 2016-11-10T16:56:20.309+0800 INFO main Bootstrap discovery.http-client.max-connections 200 200 2016-11-10T16:56:20.309+0800 INFO main Bootstrap discovery.http-client.max-connections-per-server 20 20 2016-11-10T16:56:20.309+0800 INFO main Bootstrap discovery.http-client.max-content-length 16MB 16MB 2016-11-10T16:56:20.309+0800 INFO main Bootstrap discovery.http-client.max-requests-queued-per-destination 1024 1024 2016-11-10T16:56:20.309+0800 INFO main Bootstrap discovery.http-client.request-timeout 5.00m 5.00m 2016-11-10T16:56:20.309+0800 INFO main Bootstrap discovery.http-client.socks-proxy null null 2016-11-10T16:56:20.309+0800 INFO main Bootstrap discovery.http-client.trust-store-password null null 2016-11-10T16:56:20.309+0800 INFO main Bootstrap discovery.http-client.trust-store-path null null 2016-11-10T16:56:20.309+0800 INFO main Bootstrap event.http-client.authentication.enabled false false Enable client authentication 2016-11-10T16:56:20.309+0800 INFO main Bootstrap event.http-client.connect-timeout 1.00s 1.00s 2016-11-10T16:56:20.309+0800 INFO main Bootstrap event.http-client.http2.enabled false false Enable the HTTP/2 transport 2016-11-10T16:56:20.309+0800 INFO main Bootstrap event.http-client.idle-timeout 1.00m 1.00m 2016-11-10T16:56:20.309+0800 INFO main Bootstrap event.http-client.keep-alive-interval null null 2016-11-10T16:56:20.309+0800 INFO main Bootstrap event.http-client.authentication.krb5.principal null null Set kerberos client principal 2016-11-10T16:56:20.309+0800 INFO main Bootstrap event.http-client.authentication.krb5.remote-service-name null null Set kerberos service principal name 2016-11-10T16:56:20.309+0800 INFO main Bootstrap event.http-client.key-store-password null null 2016-11-10T16:56:20.309+0800 INFO main Bootstrap event.http-client.key-store-path null null 2016-11-10T16:56:20.309+0800 INFO main Bootstrap event.http-client.max-connections 200 200 2016-11-10T16:56:20.309+0800 INFO main Bootstrap event.http-client.max-connections-per-server 20 20 2016-11-10T16:56:20.310+0800 INFO main Bootstrap event.http-client.max-content-length 16MB 16MB 2016-11-10T16:56:20.310+0800 INFO main Bootstrap event.http-client.max-requests-queued-per-destination 1024 1024 2016-11-10T16:56:20.310+0800 INFO main Bootstrap event.http-client.request-timeout 5.00m 5.00m 2016-11-10T16:56:20.310+0800 INFO main Bootstrap event.http-client.socks-proxy null null 2016-11-10T16:56:20.310+0800 INFO main Bootstrap event.http-client.trust-store-password null null 2016-11-10T16:56:20.310+0800 INFO main Bootstrap event.http-client.trust-store-path null null 2016-11-10T16:56:20.310+0800 INFO main Bootstrap http-client.max-threads 200 200 2016-11-10T16:56:20.310+0800 INFO main Bootstrap http-client.min-threads 8 8 2016-11-10T16:56:20.310+0800 INFO main Bootstrap http.authentication.krb5.config null null Set kerberos configuration path 2016-11-10T16:56:20.310+0800 INFO main Bootstrap http.authentication.krb5.credential-cache null null Set kerberos credential cache path 2016-11-10T16:56:20.310+0800 INFO main Bootstrap http.authentication.krb5.keytab null null Set kerberos key table path 2016-11-10T16:56:20.310+0800 INFO main Bootstrap http.authentication.krb5.use-canonical-hostname true true Canonicalize service hostname using the DNS reverse lookup 2016-11-10T16:56:20.310+0800 INFO main Bootstrap http-server.admin.enabled true true 2016-11-10T16:56:20.310+0800 INFO main Bootstrap http-server.admin.threads.max 200 200 2016-11-10T16:56:20.310+0800 INFO main Bootstrap http-server.admin.threads.min 2 2 2016-11-10T16:56:20.310+0800 INFO main Bootstrap http-server.admin.port 0 0 2016-11-10T16:56:20.310+0800 INFO main Bootstrap http-server.http2.max-concurrent-streams 16384 16384 Maximum concurrent streams per connection for HTTP/2 2016-11-10T16:56:20.310+0800 INFO main Bootstrap http-server.accept-queue-size 8000 8000 2016-11-10T16:56:20.310+0800 INFO main Bootstrap http-server.http.acceptor-threads null null 2016-11-10T16:56:20.310+0800 INFO main Bootstrap http-server.http.enabled true true 2016-11-10T16:56:20.310+0800 INFO main Bootstrap http-server.http.port 8080 9081 2016-11-10T16:56:20.310+0800 INFO main Bootstrap http-server.http.selector-threads null null 2016-11-10T16:56:20.310+0800 INFO main Bootstrap http-server.https.acceptor-threads null null 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.https.enabled false false 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.https.excluded-cipher [] [] 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.https.included-cipher [] [] 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.https.port 8443 8443 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.https.selector-threads null null 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.https.keystore.key null null 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.https.keystore.path null null 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.log.enabled true true 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.log.max-history 15 15 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.log.max-size 9223372036854775807B 9223372036854775807B 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.log.path var/log/http-request.log var/log/http-request.log 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.max-request-header-size null null 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.threads.max 200 200 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.threads.min 2 2 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.net.max-idle-time 200.00s 200.00s 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.show-stack-trace true true Show the stack trace when generating an error response 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.threads.max-idle-time 1.00m 1.00m 2016-11-10T16:56:20.311+0800 INFO main Bootstrap http-server.auth.users-file null null 2016-11-10T16:56:20.311+0800 INFO main Bootstrap jmx.rmiregistry.port null null 2016-11-10T16:56:20.311+0800 INFO main Bootstrap jmx.rmiserver.port null null 2016-11-10T16:56:20.311+0800 INFO main Bootstrap node.binary-spec null null 2016-11-10T16:56:20.312+0800 INFO main Bootstrap node.config-spec null null 2016-11-10T16:56:20.312+0800 INFO main Bootstrap node.environment null test 2016-11-10T16:56:20.312+0800 INFO main Bootstrap node.location null null 2016-11-10T16:56:20.312+0800 INFO main Bootstrap node.bind-ip null null 2016-11-10T16:56:20.312+0800 INFO main Bootstrap node.external-address null null 2016-11-10T16:56:20.312+0800 INFO main Bootstrap node.id null 8badb144-a723-11e6-aafd-00e0ed5716dd 2016-11-10T16:56:20.312+0800 INFO main Bootstrap node.ip null null 2016-11-10T16:56:20.312+0800 INFO main Bootstrap node.pool general general 2016-11-10T16:56:20.312+0800 INFO main Bootstrap
2016-11-10T16:56:21.361+0800 WARN main io.airlift.jmx.JmxAgent Cannot determine if JMX agent is already running (not an Oracle JVM?). Will try to start it manually. 2016-11-10T16:56:21.385+0800 INFO main io.airlift.jmx.JmxAgent JMX agent started and listening on bi-03:4515 Already running as 30697 2016-11-10T16:56:22.058+0800 ERROR Discovery-0 io.airlift.discovery.client.CachingServiceSelector Cannot connect to discovery server for refresh (presto/general): Lookup of presto failed for http://192.168.10.86:9081/v1/service/presto/general 2016-11-10T16:56:22.265+0800 ERROR Discovery-0 io.airlift.discovery.client.CachingServiceSelector Cannot connect to discovery server for refresh (collector/general): Lookup of collector failed for http://192.168.10.86:9081/v1/service/collector/general 2016-11-10T16:56:22.425+0800 INFO main org.eclipse.jetty.server.Server jetty-9.3.11.v20160721 2016-11-10T16:56:22.432+0800 WARN main org.eclipse.jetty.server.handler.AbstractHandler No Server set for org.eclipse.jetty.server.handler.ErrorHandler@4527f70a 2016-11-10T16:56:22.902+0800 INFO main org.eclipse.jetty.server.handler.ContextHandler Started o.e.j.s.ServletContextHandler@633cc6b5{/,null,AVAILABLE,@http} 2016-11-10T16:56:22.913+0800 INFO main org.eclipse.jetty.server.AbstractConnector Started http@697a34af{HTTP/1.1,[http/1.1, h2c, h2c-17, h2c-16, h2c-15, h2c-14]}{0.0.0.0:9081} 2016-11-10T16:56:22.913+0800 INFO main org.eclipse.jetty.server.Server Started @3792ms 2016-11-10T16:56:23.260+0800 INFO main io.airlift.bootstrap.LifeCycleManager Life cycle starting... 2016-11-10T16:56:23.260+0800 INFO main io.airlift.bootstrap.LifeCycleManager Life cycle startup complete. System ready. 2016-11-10T16:56:23.261+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/atop -- 2016-11-10T16:56:23.275+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.atop.AtopPlugin 2016-11-10T16:56:23.420+0800 INFO main com.facebook.presto.server.PluginManager Registering connector atop 2016-11-10T16:56:23.421+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/atop -- 2016-11-10T16:56:23.421+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/blackhole -- 2016-11-10T16:56:23.423+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.plugin.blackhole.BlackHolePlugin 2016-11-10T16:56:23.430+0800 INFO main com.facebook.presto.server.PluginManager Registering connector blackhole 2016-11-10T16:56:23.431+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/blackhole -- 2016-11-10T16:56:23.431+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/cassandra -- 2016-11-10T16:56:23.453+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.cassandra.CassandraPlugin 2016-11-10T16:56:23.463+0800 INFO main com.facebook.presto.server.PluginManager Registering connector cassandra 2016-11-10T16:56:23.464+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/cassandra -- 2016-11-10T16:56:23.464+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/example-http -- 2016-11-10T16:56:23.476+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.example.ExamplePlugin 2016-11-10T16:56:23.485+0800 INFO main com.facebook.presto.server.PluginManager Registering connector example-http 2016-11-10T16:56:23.486+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/example-http -- 2016-11-10T16:56:23.486+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/hive-cdh4 -- 2016-11-10T16:56:23.506+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.hive.HiveCdh4Plugin 2016-11-10T16:56:23.514+0800 INFO main com.facebook.presto.server.PluginManager Registering connector hive-cdh4 2016-11-10T16:56:23.515+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/hive-cdh4 -- 2016-11-10T16:56:23.515+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/hive-cdh5 -- 2016-11-10T16:56:23.537+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.hive.HiveCdh5Plugin 2016-11-10T16:56:23.546+0800 INFO main com.facebook.presto.server.PluginManager Registering connector hive-cdh5 2016-11-10T16:56:23.547+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/hive-cdh5 -- 2016-11-10T16:56:23.547+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/hive-hadoop1 -- 2016-11-10T16:56:23.567+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.hive.HiveHadoop1Plugin 2016-11-10T16:56:23.577+0800 INFO main com.facebook.presto.server.PluginManager Registering connector hive-hadoop1 2016-11-10T16:56:23.578+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/hive-hadoop1 -- 2016-11-10T16:56:23.578+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/hive-hadoop2 -- 2016-11-10T16:56:23.602+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.hive.HiveHadoop2Plugin 2016-11-10T16:56:23.612+0800 INFO main com.facebook.presto.server.PluginManager Registering connector hive-hadoop2 2016-11-10T16:56:23.613+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/hive-hadoop2 -- 2016-11-10T16:56:23.613+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/jmx -- 2016-11-10T16:56:23.619+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.connector.jmx.JmxPlugin 2016-11-10T16:56:23.627+0800 INFO main com.facebook.presto.server.PluginManager Registering connector jmx 2016-11-10T16:56:23.627+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/jmx -- 2016-11-10T16:56:23.627+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/kafka -- 2016-11-10T16:56:23.640+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.kafka.KafkaPlugin 2016-11-10T16:56:23.650+0800 INFO main com.facebook.presto.server.PluginManager Registering connector kafka 2016-11-10T16:56:23.650+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/kafka -- 2016-11-10T16:56:23.650+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/localfile -- 2016-11-10T16:56:23.658+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.localfile.LocalFilePlugin 2016-11-10T16:56:23.667+0800 INFO main com.facebook.presto.server.PluginManager Registering connector localfile 2016-11-10T16:56:23.668+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/localfile -- 2016-11-10T16:56:23.668+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/ml -- 2016-11-10T16:56:23.679+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.ml.MLPlugin 2016-11-10T16:56:23.688+0800 INFO main com.facebook.presto.server.PluginManager Registering type Model 2016-11-10T16:56:23.688+0800 INFO main com.facebook.presto.server.PluginManager Registering type Regressor 2016-11-10T16:56:23.689+0800 INFO main com.facebook.presto.server.PluginManager Registering parametric type Classifier 2016-11-10T16:56:23.698+0800 INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.ml.LearnClassifierAggregation 2016-11-10T16:56:23.703+0800 INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.ml.LearnVarcharClassifierAggregation 2016-11-10T16:56:23.706+0800 INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.ml.LearnRegressorAggregation 2016-11-10T16:56:23.708+0800 INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.ml.LearnLibSvmClassifierAggregation 2016-11-10T16:56:23.711+0800 INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.ml.LearnLibSvmVarcharClassifierAggregation 2016-11-10T16:56:23.713+0800 INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.ml.LearnLibSvmRegressorAggregation 2016-11-10T16:56:23.715+0800 INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.ml.EvaluateClassifierPredictionsAggregation 2016-11-10T16:56:23.842+0800 INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.ml.MLFunctions 2016-11-10T16:56:23.855+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/ml -- 2016-11-10T16:56:23.855+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/mongodb -- 2016-11-10T16:56:23.866+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.mongodb.MongoPlugin 2016-11-10T16:56:23.874+0800 INFO main com.facebook.presto.server.PluginManager Registering type ObjectId 2016-11-10T16:56:23.875+0800 INFO main com.facebook.presto.server.PluginManager Registering legacy connector mongodb 2016-11-10T16:56:23.876+0800 INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.mongodb.ObjectIdFunctions 2016-11-10T16:56:23.883+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/mongodb -- 2016-11-10T16:56:23.883+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/mysql -- 2016-11-10T16:56:23.893+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.plugin.mysql.MySqlPlugin 2016-11-10T16:56:23.902+0800 INFO main com.facebook.presto.server.PluginManager Registering connector mysql 2016-11-10T16:56:23.902+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/mysql -- 2016-11-10T16:56:23.902+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/postgresql -- 2016-11-10T16:56:23.913+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.plugin.postgresql.PostgreSqlPlugin 2016-11-10T16:56:23.921+0800 INFO main com.facebook.presto.server.PluginManager Registering connector postgresql 2016-11-10T16:56:23.921+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/postgresql -- 2016-11-10T16:56:23.921+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/raptor -- 2016-11-10T16:56:23.948+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.raptor.RaptorPlugin 2016-11-10T16:56:23.953+0800 INFO main com.facebook.presto.server.PluginManager Registering connector raptor 2016-11-10T16:56:23.954+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/raptor -- 2016-11-10T16:56:23.954+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/redis -- 2016-11-10T16:56:23.965+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.redis.RedisPlugin 2016-11-10T16:56:23.975+0800 INFO main com.facebook.presto.server.PluginManager Registering legacy connector redis 2016-11-10T16:56:23.975+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/redis -- 2016-11-10T16:56:23.976+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/teradata-functions -- 2016-11-10T16:56:23.977+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.teradata.functions.TeradataFunctionsPlugin 2016-11-10T16:56:23.978+0800 INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.teradata.functions.TeradataDateFunctions 2016-11-10T16:56:23.981+0800 INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.teradata.functions.TeradataStringFunctions 2016-11-10T16:56:23.985+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/teradata-functions -- 2016-11-10T16:56:23.985+0800 INFO main com.facebook.presto.server.PluginManager -- Loading plugin /var/lib/presto/data/plugin/tpch -- 2016-11-10T16:56:23.986+0800 INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.tpch.TpchPlugin 2016-11-10T16:56:23.993+0800 INFO main com.facebook.presto.server.PluginManager Registering connector tpch 2016-11-10T16:56:23.993+0800 INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /var/lib/presto/data/plugin/tpch -- 2016-11-10T16:56:24.007+0800 INFO main com.facebook.presto.metadata.CatalogManager -- Loading catalog /var/lib/presto/etc/catalog/hive.properties -- 2016-11-10T16:56:24.007+0800 INFO main com.facebook.presto.connector.ConnectorManager catalog:system 2016-11-10T16:56:24.396+0800 INFO main Bootstrap PROPERTY DEFAULT RUNTIME DESCRIPTION 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.allow-add-column false false Allow Hive connector to add column 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.allow-corrupt-writes-for-testing false false Allow Hive connector to write data even when data will likely be corrupt 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.allow-drop-table false false Allow Hive connector to drop table 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.allow-rename-column false false Allow hive connector to rename column 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.allow-rename-table false false Allow hive connector to rename table 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.assume-canonical-partition-keys false false 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.bucket-execution true true Use bucketing to speed up execution 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.bucket-writing true true Enable writing to bucketed tables 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.dfs.connect.max-retries 5 5 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.dfs.connect.timeout 500.00ms 500.00ms 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.dfs-timeout 60.00s 60.00s 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.domain-compaction-threshold 100 100 Maximum ranges to allow in a tuple domain without compacting it 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.dfs.domain-socket-path null null 2016-11-10T16:56:24.396+0800 INFO main Bootstrap deprecated.hive.integral-types-as-bigint false false Convert all hive integral types to BIGINT 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.force-local-scheduling false false 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.hdfs.authentication.type NONE NONE HDFS authentication type 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.hdfs.impersonation.enabled false false Should Presto user be impersonated when communicating with HDFS 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.hdfs.presto.keytab null null Presto keytab used to access HDFS 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.hdfs.presto.principal null null Presto principal used to access HDFS 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.compression-codec GZIP GZIP 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.metastore.authentication.type NONE NONE Hive Metastore authentication type 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.metastore.client.keytab null null Hive Metastore client keytab location 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.metastore.client.principal null null Hive Metastore client principal 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.metastore.service.principal null null Hive Metastore service principal 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.storage-format RCBINARY RCBINARY 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.immutable-partitions false false Can new data be inserted into existing partitions or existing unpartitioned tables 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.dfs.ipc-ping-interval 10.00s 10.00s 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.max-concurrent-file-renames 20 20 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.max-initial-split-size 32MB 32MB 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.max-initial-splits 200 200 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.metastore-refresh-max-threads 100 100 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.max-outstanding-splits 1000 1000 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.metastore.partition-batch-size.max 100 100 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.max-partitions-per-writers 100 100 Maximum number of partitions per writer 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.max-split-iterator-threads 1000 1000 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.max-split-size 64MB 64MB 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.metastore-cache-ttl 1.00h 1.00h 2016-11-10T16:56:24.396+0800 INFO main Bootstrap hive.metastore-refresh-interval 1.00s 1.00s 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.metastore.thrift.client.socks-proxy null null 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.metastore-timeout 10.00s 10.00s 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.metastore.partition-batch-size.min 10 10 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.orc.max-buffer-size 8MB 8MB 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.orc.max-merge-distance 1MB 1MB 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.orc.stream-buffer-size 8MB 8MB 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.parquet-optimized-reader.enabled false false 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.parquet-predicate-pushdown.enabled false false 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.s3.pin-client-to-current-region false false Should the S3 client be pinned to the current EC2 region 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.recursive-directories false false 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.config.resources null [/opt/cloudera/parcels/CDH/lib/hadoop/etc/hadoop/core-site.xml, /opt/cloudera/parcels/CDH/lib/hadoop/etc/hadoop/hdfs-site.xml] 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.respect-table-format true true Should new partitions be written using the existing table format or the default Presto format 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.s3.aws-access-key null null 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.s3.aws-secret-key null null 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.s3.connect-timeout 5.00s 5.00s 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.s3.max-backoff-time 10.00m 10.00m 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.s3.max-client-retries 3 3 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.s3.max-connections 500 500 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.s3.max-error-retries 10 10 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.s3.max-retry-time 10.00m 10.00m 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.s3.multipart.min-file-size 16MB 16MB Minimum file size for an S3 multipart upload 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.s3.multipart.min-part-size 5MB 5MB Minimum part size for an S3 multipart upload 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.s3.socket-timeout 5.00s 5.00s 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.s3.sse.enabled false false Enable S3 server side encryption 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.s3.ssl.enabled true true 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.s3.staging-directory /tmp /tmp Temporary directory for staging files before uploading to S3 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.s3.use-instance-credentials true true 2016-11-10T16:56:24.397+0800 INFO main Bootstrap hive.time-zone PRC PRC 2016-11-10T16:56:24.398+0800 INFO main Bootstrap hive.orc.use-column-names false false Access ORC columns using names from the file 2016-11-10T16:56:24.398+0800 INFO main Bootstrap hive.parquet.use-column-names false false Access Parquet columns using names from the file 2016-11-10T16:56:24.398+0800 INFO main Bootstrap hive.dfs.verify-checksum true true 2016-11-10T16:56:24.398+0800 INFO main Bootstrap hive.security allow-all allow-all 2016-11-10T16:56:24.398+0800 INFO main Bootstrap hive.metastore.uri null [thrift://bi-01:9083, thrift://bi-03:9083, thrift://bi-04:9083] Hive metastore URIs (comma separated) 2016-11-10T16:56:24.398+0800 INFO main Bootstrap node.binary-spec null null 2016-11-10T16:56:24.398+0800 INFO main Bootstrap node.config-spec null null 2016-11-10T16:56:24.398+0800 INFO main Bootstrap node.environment null test 2016-11-10T16:56:24.398+0800 INFO main Bootstrap node.location null null 2016-11-10T16:56:24.398+0800 INFO main Bootstrap node.bind-ip null null 2016-11-10T16:56:24.398+0800 INFO main Bootstrap node.external-address null null 2016-11-10T16:56:24.398+0800 INFO main Bootstrap node.id null 8badb144-a723-11e6-aafd-00e0ed5716dd 2016-11-10T16:56:24.398+0800 INFO main Bootstrap node.ip null null 2016-11-10T16:56:24.398+0800 INFO main Bootstrap node.pool general general 2016-11-10T16:56:24.398+0800 INFO main Bootstrap
2016-11-10T16:56:25.136+0800 INFO main io.airlift.bootstrap.LifeCycleManager Life cycle starting... 2016-11-10T16:56:25.137+0800 INFO main io.airlift.bootstrap.LifeCycleManager Life cycle startup complete. System ready. 2016-11-10T16:56:25.140+0800 INFO main com.facebook.presto.metadata.CatalogManager -- Added catalog hive using connector hive-hadoop2 -- 2016-11-10T16:56:25.141+0800 INFO Thread-26 com.facebook.presto.metadata.CatalogManager -- Catalog watcher thread start -- 2016-11-10T16:56:25.142+0800 INFO main com.facebook.presto.security.AccessControlManager -- Loading system access control -- 2016-11-10T16:56:25.142+0800 INFO main com.facebook.presto.security.AccessControlManager -- Loaded system access control allow-all -- 2016-11-10T16:56:25.195+0800 INFO main com.facebook.presto.server.PrestoServer ======== SERVER STARTED ======== 2016-11-10T16:56:25.196+0800 ERROR Announcer-0 io.airlift.discovery.client.Announcer Cannot connect to discovery server for announce: Announcement failed with status code 404: 2016-11-10T16:56:25.197+0800 ERROR Announcer-0 io.airlift.discovery.client.Announcer Service announcement failed after 54.29ms. Next request will happen within 0.00s 2016-11-10T16:56:25.206+0800 ERROR Announcer-1 io.airlift.discovery.client.Announcer Service announcement failed after 4.97ms. Next request will happen within 1.00ms 2016-11-10T16:56:25.217+0800 ERROR Announcer-2 io.airlift.discovery.client.Announcer Service announcement failed after 8.44ms. Next request will happen within 2.00ms 2016-11-10T16:56:25.226+0800 ERROR Announcer-3 io.airlift.discovery.client.Announcer Service announcement failed after 5.23ms. Next request will happen within 4.00ms 2016-11-10T16:56:25.240+0800 ERROR Announcer-4 io.airlift.discovery.client.Announcer Service announcement failed after 4.98ms. Next request will happen within 8.00ms 2016-11-10T16:56:25.263+0800 ERROR Announcer-2 io.airlift.discovery.client.Announcer Service announcement failed after 6.75ms. Next request will happen within 16.00ms 2016-11-10T16:56:25.300+0800 ERROR Announcer-3 io.airlift.discovery.client.Announcer Service announcement failed after 4.31ms. Next request will happen within 32.00ms 2016-11-10T16:56:25.369+0800 ERROR Announcer-4 io.airlift.discovery.client.Announcer Service announcement failed after 4.45ms. Next request will happen within 64.00ms 2016-11-10T16:56:25.502+0800 ERROR Announcer-2 io.airlift.discovery.client.Announcer Service announcement failed after 5.04ms. Next request will happen within 128.00ms 2016-11-10T16:56:25.764+0800 ERROR Announcer-3 io.airlift.discovery.client.Announcer Service announcement failed after 5.64ms. Next request will happen within 256.00ms 2016-11-10T16:56:26.282+0800 ERROR Announcer-4 io.airlift.discovery.client.Announcer Service announcement failed after 5.63ms. Next request will happen within 512.00ms 2016-11-10T16:56:27.290+0800 ERROR Announcer-2 io.airlift.discovery.client.Announcer Service announcement failed after 6.45ms. Next request will happen within 1000.00ms 2016-11-10T16:56:28.295+0800 ERROR Announcer-3 io.airlift.discovery.client.Announcer Service announcement failed after 4.89ms. Next request will happen within 1000.00ms 2016-11-10T16:56:29.300+0800 ERROR Announcer-4 io.airlift.discovery.client.Announcer Service announcement failed after 4.65ms. Next request will happen within 1000.00ms 2016-11-10T16:56:30.305+0800 ERROR Announcer-2 io.airlift.discovery.client.Announcer Service announcement failed after 4.05ms. Next request will happen within 1000.00ms 2016-11-10T16:56:31.312+0800 ERROR Announcer-3 io.airlift.discovery.client.Announcer Service announcement failed after 6.66ms. Next request will happen within 1000.00ms 2016-11-10T16:56:32.316+0800 ERROR Announcer-4 io.airlift.discovery.client.Announcer Service announcement failed after 3.53ms. Next request will happen within 1000.00ms 2016-11-10T16:56:33.321+0800 ERROR Announcer-2 io.airlift.discovery.client.Announcer Service announcement failed after 4.51ms. Next request will happen within 1000.00ms 2016-11-10T16:56:34.326+0800 ERROR Announcer-0 io.airlift.discovery.client.Announcer Service announcement failed after 4.41ms. Next request will happen within 1000.00ms 2016-11-10T16:56:35.330+0800 ERROR Announcer-1 io.airlift.discovery.client.Announcer Service announcement failed after 3.85ms. Next request will happen within 1000.00ms 2016-11-10T16:56:36.334+0800 ERROR Announcer-2 io.airlift.discovery.client.Announcer Service announcement failed after 3.20ms. Next request will happen within 1000.00ms 2016-11-10T16:56:37.339+0800 ERROR Announcer-0 io.airlift.discovery.client.Announcer Service announcement failed after 3.54ms. Next request will happen within 1000.00ms 2016-11-10T16:56:38.342+0800 ERROR Announcer-1 io.airlift.discovery.client.Announcer Service announcement failed after 2.67ms. Next request will happen within 1000.00ms 2016-11-10T16:56:39.346+0800 ERROR Announcer-2 io.airlift.discovery.client.Announcer Service announcement failed after 3.39ms. Next request will happen within 1000.00ms 2016-11-10T16:56:40.349+0800 ERROR Announcer-0 io.airlift.discovery.client.Announcer Service announcement failed after 2.70ms. Next request will happen within 1000.00ms 2016-11-10T16:56:41.353+0800 ERROR Announcer-1 io.airlift.discovery.client.Announcer Service announcement failed after 3.33ms. Next request will happen within 1000.00ms 2016-11-10T16:56:42.357+0800 ERROR Announcer-2 io.airlift.discovery.client.Announcer Service announcement failed after 3.28ms. Next request will happen within 1000.00ms 2016-11-10T16:56:43.361+0800 ERROR Announcer-0 io.airlift.discovery.client.Announcer Service announcement failed after 3.83ms. Next request will happen within 1000.00ms 2016-11-10T16:56:44.365+0800 ERROR Announcer-1 io.airlift.discovery.client.Announcer Service announcement failed after 3.17ms. Next request will happen within 1000.00ms 2016-11-10T16:56:45.369+0800 ERROR Announcer-2 io.airlift.discovery.client.Announcer Service announcement failed after 4.06ms. Next request will happen within 1000.00ms 2016-11-10T16:56:46.373+0800 ERROR Announcer-0 io.airlift.discovery.client.Announcer Service announcement failed after 3.03ms. Next request will happen within 1000.00ms 2016-11-10T16:56:47.377+0800 ERROR Announcer-1 io.airlift.discovery.client.Announcer Service announcement failed after 3.78ms. Next request will happen within 1000.00ms 2016-11-10T16:56:48.381+0800 ERROR Announcer-2 io.airlift.discovery.client.Announcer Service announcement failed after 3.29ms. Next request will happen within 1000.00ms 2016-11-10T16:56:49.385+0800 ERROR Announcer-0 io.airlift.discovery.client.Announcer Service announcement failed after 3.37ms. Next request will happen within 1000.00ms 2016-11-10T16:56:50.387+0800 ERROR Announcer-1 io.airlift.discovery.client.Announcer Service announcement failed after 2.14ms. Next request will happen within 1000.00ms 2016-11-10T16:56:51.392+0800 ERROR Announcer-2 io.airlift.discovery.client.Announcer Service announcement failed after 3.71ms. Next request will happen within 1000.00ms 2016-11-10T16:56:52.395+0800 ERROR Announcer-0 io.airlift.discovery.client.Announcer Service announcement failed after 3.24ms. Next request will happen within 1000.00ms 2016-11-10T16:56:53.399+0800 ERROR Announcer-1 io.airlift.discovery.client.Announcer Service announcement failed after 3.32ms. Next request will happen within 1000.00ms 2016-11-10T16:56:54.403+0800 ERROR Announcer-2 io.airlift.discovery.client.Announcer Service announcement failed after 3.41ms. Next request will happen within 1000.00ms 2016-11-10T16:56:55.407+0800 ERROR Announcer-0 io.airlift.discovery.client.Announcer Service announcement failed after 3.52ms. Next request will happen within 1000.00ms 2016-11-10T16:56:56.411+0800 ERROR Announcer-1 io.airlift.discovery.client.Announcer Service announcement failed after 3.19ms. Next request will happen within 1000.00ms 2016-11-10T16:56:57.415+0800 ERROR Announcer-2 io.airlift.discovery.client.Announcer Service announcement failed after 3.13ms. Next request will happen within 1000.00ms 2016-11-10T16:56:58.419+0800 ERROR Announcer-0 io.airlift.discovery.client.Announcer Service announcement failed after 3.96ms. Next request will happen within 1000.00ms 2016-11-10T16:56:59.423+0800 ERROR Announcer-1 io.airlift.discovery.client.Announcer Service announcement failed after 3.65ms. Next request will happen within 1000.00ms 2016-11-10T16:57:00.426+0800 ERROR Announcer-2 io.airlift.discovery.client.Announcer Service announcement failed after 2.73ms. Next request will happen within 1000.00ms 2016-11-10T16:57:01.430+0800 ERROR Announcer-0 io.airlift.discovery.client.Announcer Service announcement failed after 2.68ms. Next request will happen within 1000.00ms 2016-11-10T16:57:02.433+0800 ERROR Announcer-1 io.airlift.discovery.client.Announcer Service announcement failed after 3.23ms. Next request will happen within 1000.00ms 2016-11-10T16:57:02.487+0800 INFO Thread-25 io.airlift.bootstrap.LifeCycleManager Life cycle stopping... 2016-11-10T16:57:02.488+0800 INFO Thread-23 io.airlift.bootstrap.LifeCycleManager Life cycle stopping...

[anusudarsan]

Did you verify the config.properties file for Presto? It should be available at /var/lib/presto/etc directory if you are using the default appConfig.json property site.global.config_dir once the YARN app starts. Can you paste the contents of node.properties and config.properties from the coordinator and one of the worker nodes?

[JasonBian]

@anusudarsan appConfig.json `{ "schema": "http://example.org/specification/v2.0.0", "metadata": { }, "global": { "site.global.app_user": "root", "site.global.user_group": "hadoop", "site.global.data_dir": "/var/lib/presto/data", "site.global.config_dir": "/var/lib/presto/etc", "site.global.app_name": "presto-server-0.152", "site.global.app_pkg_plugin": "${AGENT_WORK_ROOT}/app/definition/package/plugins/", "site.global.singlenode": "false", "site.global.coordinator_host": "${COORDINATOR_HOST}", "site.global.presto_query_max_memory": "1GB", "site.global.presto_query_max_memory_per_node": "60MB", "site.global.presto_server_port": "9081", "site.global.catalog": "{'hive': ['connector.name=hive-hadoop2', 'hive.metastore.uri=thrift://bi-01:9083,thrift://bi-03:9083,thrift://bi-04:9083', 'hive.config.resources=/opt/cloudera/parcels/CD H/lib/hadoop/etc/hadoop/core-site.xml,/opt/cloudera/parcels/CDH/lib/hadoop/etc/hadoop/hdfs-site.xml', 'hive.allow-drop-table=false']}", "site.global.jvm_args": "['-server', '-Xmx1024M', '-XX:+UseG1GC', '-XX:G1HeapRegionSize=32M', '-XX:+UseGCOverheadLimit', '-XX:+ExplicitGCInvokesConcurrent', '-XX:+HeapDumpOnOutOfMemoryError', '- XX:OnOutOfMemoryError=kill -9 %p']",

"application.def": ".slider/package/PRESTO/presto-yarn-package-1.4-SNAPSHOT-0.152.zip",
"java_home": "/usr/java/jdk1.8.0_60/jre/bin/java"

}, "components": { "slider-appmaster": { "jvm.heapsize": "128M" } } }`

coordinator config.properties: coordinator=true node-scheduler.include-coordinator=False discovery-server.enabled=true http-server.http.port=9081 query.max-memory=50GB query.max-memory-per-node=600MB query.queue-config-file=/var/lib/presto/etc/queues.json discovery.uri=http://bi-03:9081

coordinator node.properties: node.environment=test node.id=b89e6f14-a7ae-11e6-b35d-00e0ed5716dd node.data-dir=/var/lib/presto/data plugin.config-dir=/var/lib/presto/etc/catalog

worker config.properties: coordinator=false http-server.http.port=9081 query.max-memory=50GB query.max-memory-per-node=600MB discovery.uri=http://bi-03:9081

worker node.properties: ode.environment=test node.id=b647b14e-a7ae-11e6-8d3f-6c92bf2dddb0 node.data-dir=/var/lib/presto/data plugin.config-dir=/var/lib/presto/etc/catalog

[anusudarsan]

@JasonBian your configuration looks good to me. Do you think this could be a DNS problem?

[JasonBian]

@anusudarsan I really don't know what cause this problem.When I reinstall apache slider and build presto on yarn with presto-0.153,it works well.