search

prestodb / presto-admin

A tool to install, configure and manage Presto installations
http://prestodb.github.io/presto-admin/
Apache License 2.0
171 stars 100 forks source link

Presto-admin should not swallow errors when it does not get a valid port #243

Open mattsfuller opened 8 years ago

mattsfuller commented 8 years ago

when lookup_in_config() from remote_config_util.py encounters an error, it swallows and hides any errors. Additionally, if field validation fails for lookup_port, a warning gets printed to the logs, but the query still tries to continue without a valid port. That means that instead of getting an error with the actual issue when you run server status and get an error, you just get a blank status Server Status: 39.1.192.3(IP: Unknown, Roles: coordinator): Running No information available: unable to query coordinator Server Status: 39.1.184.31(IP: Unknown, Roles: worker): Running No information available: unable to query coordinator Server Status: 39.1.192.20(IP: Unknown, Roles: worker): Running No information available: unable to query coordinator Server Status: 39.1.192.21(IP: Unknown, Roles: worker): Running No information available: unable to query coordinator Instead the query should fail fast with the appropriate error message

col15 commented 8 years ago

I'm trying to setup presto-admin 1.5.2 on EMR but get the same error you mentioned above:

./presto-admin -i key.pem server status

Server Status: 11.19.3.12(IP: Unknown, Roles: coordinator): Not Running No information available: unable to query coordinator Server Status: 11.19.3.11(IP: Unknown, Roles: worker): Not Running No information available: unable to query coordinator

The config.json and pem key are in the same directory? The IPs are valid and I am able to SSH into these machines. Should be something simple but I'm out of hair to pull. Anyone know what the issue is?

col15 commented 8 years ago

@mattsfuller - were you able to get this working with EMR?

cawallin commented 8 years ago

Check what error you get in /var/log/prestoadmin/presto-admin.log; this issue could happen for a few different reasons.

cawallin commented 8 years ago

Also, presto-admin won't necessarily work with the EMR version of Presto, because of differences in directories, you may have to install the RPM from the Presto project, most easily accessible at www.teradata.com/presto.

col15 commented 8 years ago

Hi @cawallin

Log posted below. It looks like one issue is it can't find the file /etc/presto/config.properties. On EMR, the path is /etc/presto/conf/config.properties

Is there an easy way set that path?

STDERR: 2016-11-15 18:30:19,421|8484|140067122726720|prestoadmin.server|WARNING|Could not access config file /etc/presto/config.properties on host 1.2.3.4 2016-11-15 18:30:19,431|8525|140066882877184|paramiko.transport|DEBUG|starting thread (client mode): 0xdcf8b990L 2016-11-15 18:30:19,436|8525|140066882877184|paramiko.transport|INFO|Connected (version 2.0, client OpenSSH_6.6.1) 2016-11-15 18:30:19,437|8525|140066882877184|paramiko.transport|DEBUG|kex algos:[u'curve25519-sha256@libssh.org', u'ecdh-sha2-nistp256', u'ecdh-sha2-nistp384', u'ecdh-sha2-nistp521', u'diffie-hellman-group-exchange-sha256', u'diffie-hellman-group-exchange-sha1', u'diffie-hellman-group14-sha1', u'diffie-hellman-group1-sha1'] server key:[u'ssh-rsa', u'ecdsa-sha2-nistp256', u'ssh-ed25519'] client encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'arcfour256', u'arcfour128', u'aes128-gcm@openssh.com', u'aes256-gcm@openssh.com', u'chacha20-poly1305@openssh.com', u'aes128-cbc', u'3des-cbc', u'blowfish-cbc', u'cast128-cbc', u'aes192-cbc', u'aes256-cbc', u'arcfour', u'rijndael-cbc@lysator.liu.se'] server encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'arcfour256', u'arcfour128', u'aes128-gcm@openssh.com', u'aes256-gcm@openssh.com', u'chacha20-poly1305@openssh.com', u'aes128-cbc', u'3des-cbc', u'blowfish-cbc', u'cast128-cbc', u'aes192-cbc', u'aes256-cbc', u'arcfour', u'rijndael-cbc@lysator.liu.se'] client mac:[u'hmac-md5-etm@openssh.com', u'hmac-sha1-etm@openssh.com', u'umac-64-etm@openssh.com', u'umac-128-etm@openssh.com', u'hmac-sha2-256-etm@openssh.com', u'hmac-sha2-512-etm@openssh.com', u'hmac-ripemd160-etm@openssh.com', u'hmac-sha1-96-etm@openssh.com', u'hmac-md5-96-etm@openssh.com', u'hmac-md5', u'hmac-sha1', u'umac-64@openssh.com', u'umac-128@openssh.com', u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-ripemd160', u'hmac-ripemd160@openssh.com', u'hmac-sha1-96', u'hmac-md5-96'] server mac:[u'hmac-md5-etm@openssh.com', u'hmac-sha1-etm@openssh.com', u'umac-64-etm@openssh.com', u'umac-128-etm@openssh.com', u'hmac-sha2-256-etm@openssh.com', u'hmac-sha2-512-etm@openssh.com', u'hmac-ripemd160-etm@openssh.com', u'hmac-sha1-96-etm@openssh.com', u'hmac-md5-96-etm@openssh.com', u'hmac-md5', u'hmac-sha1', u'umac-64@openssh.com', u'umac-128@openssh.com', u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-ripemd160', u'hmac-ripemd160@openssh.com', u'hmac-sha1-96', u'hmac-md5-96'] client compress:[u'none', u'zlib@openssh.com'] server compress:[u'none', u'zlib@openssh.com'] client lang:[u''] server lang:[u''] kex follows?False 2016-11-15 18:30:19,438|8525|140066882877184|paramiko.transport|DEBUG|Ciphers agreed: local=aes128-ctr, remote=aes128-ctr 2016-11-15 18:30:19,438|8525|140066882877184|paramiko.transport|DEBUG|using kex diffie-hellman-group14-sha1; server key type ssh-rsa; cipher: local aes128-ctr, remote aes128-ctr; mac: local hmac-sha1, remote hmac-sha1; compression: local none, remote none 2016-11-15 18:30:19,481|8525|140066882877184|paramiko.transport|DEBUG|Switch to new keys ... 2016-11-15 18:30:19,496|8525|140067122726720|paramiko.transport|DEBUG|Adding ssh-rsa host key for 1.2.3.4: f74be8e5416d4977021be73377f88c97 2016-11-15 18:30:19,497|8525|140067122726720|paramiko.transport|DEBUG|Trying key c7fa8f1599eea839cd37b1a1b4dfb51f from abc.pem 2016-11-15 18:30:19,519|8525|140066882877184|paramiko.transport|DEBUG|userauth is OK 2016-11-15 18:30:19,736|8525|140066882877184|paramiko.transport|INFO|Authentication (publickey) successful! 2016-11-15 18:30:19,738|8525|140067122726720|paramiko.transport|DEBUG|[chan 0] Max packet in: 32768 bytes 2016-11-15 18:30:19,740|8525|140066882877184|paramiko.transport|DEBUG|[chan 0] Max packet out: 32768 bytes 2016-11-15 18:30:19,740|8525|140066882877184|paramiko.transport|DEBUG|Secsh channel 0 opened. 2016-11-15 18:30:19,743|8525|140066882877184|paramiko.transport|DEBUG|[chan 0] Sesch channel 0 request ok 2016-11-15 18:30:19,744|8525|140066882877184|paramiko.transport|DEBUG|[chan 0] Sesch channel 0 request ok 2016-11-15 18:30:19,815|8525|140066882877184|paramiko.transport|DEBUG|[chan 0] EOF received (0) 2016-11-15 18:30:19,816|8525|140066882877184|paramiko.transport|DEBUG|[chan 0] EOF sent (0) 2016-11-15 18:30:19,847|8525|140067122726720|prestoadmin.fabric_patches|INFO| COMMAND: rpm -q presto FULL COMMAND: /bin/bash -l -c "rpm -q presto" STDOUT: presto-0.150-1.amzn1.x86_64 STDERR: 2016-11-15 18:30:19,848|8525|140067122726720|paramiko.transport|DEBUG|[chan 1] Max packet in: 32768 bytes 2016-11-15 18:30:19,855|8525|140066882877184|paramiko.transport|DEBUG|[chan 1] Max packet out: 32768 bytes 2016-11-15 18:30:19,856|8525|140066882877184|paramiko.transport|DEBUG|Secsh channel 1 opened. 2016-11-15 18:30:19,865|8525|140066882877184|paramiko.transport|DEBUG|[chan 1] Sesch channel 1 request ok 2016-11-15 18:30:19,866|8525|140066882877184|paramiko.transport|DEBUG|[chan 1] Sesch channel 1 request ok 2016-11-15 18:30:19,906|8525|140066882877184|paramiko.transport|DEBUG|[chan 1] EOF received (1) 2016-11-15 18:30:19,906|8525|140066882877184|paramiko.transport|DEBUG|[chan 1] EOF sent (1) 2016-11-15 18:30:19,932|8525|140067122726720|paramiko.transport|DEBUG|[chan 2] Max packet in: 32768 bytes 2016-11-15 18:30:19,943|8525|140066882877184|paramiko.transport|DEBUG|[chan 2] Max packet out: 32768 bytes 2016-11-15 18:30:19,944|8525|140066882877184|paramiko.transport|DEBUG|Secsh channel 2 opened. 2016-11-15 18:30:19,949|8525|140066882877184|paramiko.transport|DEBUG|[chan 2] Sesch channel 2 request ok 2016-11-15 18:30:19,950|8525|140066882877184|paramiko.transport|DEBUG|[chan 2] Sesch channel 2 request ok 2016-11-15 18:30:20,017|8525|140066882877184|paramiko.transport|DEBUG|[chan 2] EOF received (2) 2016-11-15 18:30:20,018|8525|140066882877184|paramiko.transport|DEBUG|[chan 2] EOF sent (2) 2016-11-15 18:30:20,051|8525|140067122726720|prestoadmin.fabric_patches|INFO| COMMAND: rpm -q presto FULL COMMAND: /bin/bash -l -c "rpm -q presto" STDOUT: presto-0.150-1.amzn1.x86_64 STDERR: 2016-11-15 18:30:20,052|8525|140067122726720|prestoadmin.server|INFO|Executing status on presto server 2016-11-15 18:30:20,052|8525|140067122726720|paramiko.transport|DEBUG|[chan 3] Max packet in: 32768 bytes 2016-11-15 18:30:20,055|8525|140066882877184|paramiko.transport|DEBUG|[chan 3] Max packet out: 32768 bytes 2016-11-15 18:30:20,056|8525|140066882877184|paramiko.transport|DEBUG|Secsh channel 3 opened. 2016-11-15 18:30:20,058|8525|140066882877184|paramiko.transport|DEBUG|[chan 3] Sesch channel 3 request ok 2016-11-15 18:30:20,059|8525|140066882877184|paramiko.transport|DEBUG|[chan 3] Sesch channel 3 request ok 2016-11-15 18:30:20,098|8525|140066882877184|paramiko.transport|DEBUG|[chan 3] EOF received (3) 2016-11-15 18:30:20,099|8525|140066882877184|paramiko.transport|DEBUG|[chan 3] EOF sent (3) 2016-11-15 18:30:20,146|8484|140067122726720|__main__|DEBUG|Exiting normally

cawallin commented 8 years ago

No, that can't be changed other than a code change. The RPM that's used on EMR unfortunately is different from the presto-server-rpm from the Presto project.

col15 commented 8 years ago

Hmm, so basically it is not supported with EMR? Surprised on one else has complained yet. This should definitely be a configurable path and along with any hardcoded paths.

cawallin commented 8 years ago

Yeah, I agree with you -- there is an issue for it: https://github.com/prestodb/presto-admin/issues/229. We haven't gotten to it yet, but if you wanted to jump in you'd be welcome to!