kiteco / issue-tracker

User-reported issues for Kite
https://Kite.com
1.75k stars 136 forks source link

Can't install to internal ec2 #181

Closed yelled1 closed 4 years ago

yelled1 commented 4 years ago

I've been trying to get Kite installed to our company internal EC2. I tried proxies already & I know it works, as I can pickup the installation script already. I've also tried hacking the json file but Kite overwrites it during installation. I tried to get Kite support but, as you can see below, he referred to this github for help. Thank u in advance for any lead,

 "$(wget -q -O - https://linux.kite.com/dls/linux/current)"

 This script will install Kite!
 We hope you enjoy! If you run into any issues, please reach out at support@kite.com or feedback@kite.com
 - The Kite Team

 Press enter to continue...
 Downloading /home/MySweetHome/Temp/kite-installer binary using wget...
 Kite watches your workspace to be notified when python files change on disk. This allows us to
 provide the latest information & completions from your coding environment. However, for larger
 workspaces, Kite can exceed the default limits on inotify watches, which can result in a degraded experience. 

 We can fix this by placing a file in /etc/sysctl.d/ to increase this limit.
 Configure inotify now? (you might be asked for your sudo password) [Y/n] n

 If you'd like to configure this yourself, you can run:
     sudo echo fs.inotify.max_user_watches = 524288 > /etc/sysctl.d/30-kite.conf && sysctl -q --system

     Checking to see if all dependencies are installed....
     Running /home/MySweetHome/Temp/kite-installer install 
     [installer] no previous kite installation found
     [installer] latest version is 2.20200310.0, downloading now...
     [installer] Downloading Kite:   99.9% of 370MiB
     [installer] verifying checksum
     [installer] validating signature
     [installer] installing version 2.20200310.0
     [installer] installed ~/.config/autostart/kite-autostart.desktop
     [installer] installed ~/.config/systemd/user/kite-updater.service
     [installer] installed ~/.config/systemd/user/kite-updater.timer
     [installer] installed ~/.local/share/applications/kite-copilot.desktop
     [installer] installed ~/.local/share/applications/kite.desktop
     [installer] installed ~/.local/share/icons/hicolor/128x128/apps/kite.png
     [installer] installed ~/.local/share/kite/kited
     [installer] installed ~/.local/share/kite/uninstall
     [installer] installed ~/.local/share/kite/update
     [installer] activating kite-updater systemd service
     [installer] registering kite:// protocol handler
     [installer] error installing kite protocol handler. exec: "xdg-mime": executable file not found in $PATH, output: 
     [installer] Rolling back installed data...
     [installer] terminating kite processes
     [installer] removing kite-updater systemd service
     [installer] removed ~/.config/autostart/kite-autostart.desktop
     [installer] removed ~/.config/systemd/user/kite-updater.service
     [installer] removed ~/.config/systemd/user/kite-updater.timer
     [installer] removed ~/.local/share/applications/kite-copilot.desktop
     [installer] removed ~/.local/share/applications/kite.desktop
     [installer] removed ~/.local/share/icons/hicolor/128x128/apps/kite.png
     [installer] removed ~/.local/share/kite/kited
     [installer] removed ~/.local/share/kite/uninstall
     [installer] removed ~/.local/share/kite/update
     [installer] removed ~/.local/share/kite
     Rollbar error: received response: 429 Too Many Requests
     Rollbar error: received response: 429 Too Many Requests
     Rollbar error: received response: 429 Too Many Requests
     Rollbar error: received response: 429 Too Many Requests
     Rollbar item failed to send: map[access_token:cd9c587fdee94832b047d7a497605dec data:map[platform:linux custom:map[data0:install data4:Linux ec2-MySweetHome 4.15.0-1060-aws #62-Ubuntu SMP Tue Feb 11 21:23:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
      machine_id: data2:exec: "xdg-mime": executable file not found in $PATH data3: data1:error running 'xdg-mime default kite-copilot.desktop x-scheme-handler/kite' user_id:0 install_id: metrics_id:] body:map[trace_chain:[map[frames:[{github.com/kiteco/kiteco/vendor/github.com/rollbar/rollbar-go/rollbar.go rollbar-go.ErrorWithStackSkipWithExtras 431} {github.com/kiteco/kiteco/kite-golib/rollbar/rollbar.go rollbar.send 163} {github.com/kiteco/kiteco/kite-golib/rollbar/rollbar.go rollbar.Error 94} {github.com/kiteco/kiteco/linux/cmds/kite-installer/utils.go main.rollbarError 157} {github.com/kiteco/kiteco/linux/cmds/kite-installer/utils.go main.installSystemData 148} {github.com/kiteco/kiteco/linux/cmds/kite-installer/install.go main.(*installCommand).Run 110} {github.com/kiteco/kiteco/vendor/github.com/mitchellh/cli/cli.go cli.(*CLI).Run 153} {github.com/kiteco/kiteco/linux/cmds/kite-installer/main.go main.main 50} {/usr/lib/go-1.11/src/runtime/proc.go runtime.main 201} {/usr/lib/go-1.11/src/runtime/asm_amd64.s runtime.goexit 1333}] exception:map[class:{34d10ee5} message:failed to enable kite:// protocol handler]]]] environment:production title:failed to enable kite:// protocol handler language:go code_version:2.20200310.0 server:map[root: host:ec2-MySweetHome] notifier:map[name:rollbar/rollbar-go version:1.0.1] level:error timestamp:1584211105]]
------------- ERROR TEXT -------------------------------------------------------------------

Hi Hoon,

Sorry I couldn't figure this out for you. Please feel free to open up a GitHub issue at our repo and a more qualified engineer will be able to help you out. Just be sure to provide all the useful and relevant info you gave me so that they'll be able to debug effectively. Hope we can fix this for you!

On Tue, Mar 24, 2020 at 6:28 PM S. Hoon my_fake_email@gmail wrote: Daniel:

Good thought, but I have already set those prior to attempting the installation. So, that does not do it. Think we might need an old-fashioned script solution or an offline way to get it working. I can get to GUI but GUI can't get to registration without proxy assigned. It obviously ignores the existing settings below. Thank you for trying though,

$ set | grep proxy
ftp_proxy=http://MUTAL%5Cny1111:myPwdS%25@nyproxy.mutal.local:80
http_proxy=http://MUTAL%5Cny1111:myPwdS%25@nyproxy.mutal.local:80
https_proxy=http://MUTAL%5Cny1111:myPwd%25@nyproxy.mutal.local:80

On Tue, Mar 24, 2020 at 8:15 PM Daniel Hung daniel@kite.com wrote: Hi Hoon,

Can you try running this command from your shell: export http_proxy="http://host:port" https_proxy="https://host:port" Then start Kite from the command line from the same shell session.

On Sun, Mar 22, 2020 at 4:28 PM S. Hoon my_fake_email@gmail wrote: Pls make it work. Really like the product. When it works.

Thank you much

On Fri, Mar 20, 2020, 11:11 PM Daniel Hung daniel@kite.com wrote: Thanks for the info. I'll forward this question to our engineers and get back to you ASAP.

On Fri, Mar 20, 2020 at 4:10 AM S. Hoon my_fake_email@gmail wrote: Hi:

Retrying. Is there a way to set the proxy for Kite pre-install via hacking json file or as an argument to launching the bash script? Or, go into proxy settings just prior to connecting the web to verify? Just popup or button to access the proxy menu when Kite installer realizes that after a full min, it can't reach the destination. Seems like that's all it may take, as my desktop Ubuntu experience was extremely smooth. Thanks,

Hoon,

On Wed, Mar 18, 2020 at 5:40 AM S. Hoon my_fake_email@gmail wrote: Daniel:

I got Kite installed again via MobaXterm X11 fwding, but I am getting "wait msg" over & over again from the GUI. Note that this is 4th time that I have download & attempted to install. I tried to overwrite the setting.json in .kite directory to put our company proxy settings which I know it works via Atom editor, but the app continues to overwrite my edits. I killed -9 kite instances & . kited as well & edited users & settings.json. Kite really has a revolutionary product that I & many want. Do you really want to tell the EC2 market that Kite is NOT supporting it? If anyone has NOT noticed. The world just went cloud after #WuhanVirus19. Please work with me to get this done. After Kite, neither deoplete nor YCM really does the job. Thanks,

Hoon,

On Tue, Mar 17, 2020 at 9:17 PM Daniel Hung daniel@kite.com wrote: Hi Hoon,

Thanks for taking the time to write in. Unfortunately Kite does not support remote coding sessions on e.g. EC2 instances. Apologies for the inconvenience!

On Sat, Mar 14, 2020 at 12:04 PM S. Hoon my_fake_email@gmail wrote: Hi:

I get error as following:

xdg-mime
     Rollbar error: received response: 429 Too Many Requests
     Rollbar item failed to send: map[access_token:cdxxxxxxxx data:map[platform:linux custom:map[data0:install data4:Linux ec2-ny11559 4.15.0-1060-aws #62-Ubuntu SMP Tue Feb 11 21:23:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
      machine_id: data2:exec: "xdg-mime": executable file not found in $PATH data3: data1:error running 'xdg-mime default kite-copilot.desktop x-scheme-handler/kite' user_id:0 install_id: metrics_id:] body:map[trace_chain:[map[frames:[{github.com/kiteco/kiteco/vendor/github.com/rollbar/rollbar-go/rollbar.go rollbar-go.ErrorWithStackSkipWithExtras 431} {github.com/kiteco/kiteco/kite-golib/rollbar/rollbar.go rollbar.send 163} {github.com/kiteco/kiteco/kite-golib/rollbar/rollbar.go rollbar.Error 94} {github.com/kiteco/kiteco/linux/cmds/kite-installer/utils.go main.rollbarError 157} {github.com/kiteco/kiteco/linux/cmds/kite-installer/utils.go main.installSystemData 148} {github.com/kiteco/kiteco/linux/cmds/kite-installer/install.go main.(*installCommand).Run 110} {github.com/kiteco/kiteco/vendor/github.com/mitchellh/cli/cli.go cli.(*CLI).Run 153} {github.com/kiteco/kiteco/linux/cmds/kite-installer/main.go main.main 50} {/usr/lib/go-1.11/src/runtime/proc.go runtime.main 201} {/usr/lib/go-1.11/src/runtime/asm_amd64.s runtime.goexit 1333}] exception:map[class:{34d10ee5} message:failed to enable kite:// protocol handler]]]] environment:production title:failed to enable kite:// protocol handler language:go code_version:2.20200310.0 server:map[root: host:ec2-ny11559] notifier:map[name:rollbar/rollbar-go version:1.0.1] level:error timestamp:1584211105]]
My machine is configured as follows on Amazon EC2
=============================================================================
       __|  __|_  )
       _|  (     /   Deep Learning AMI (Ubuntu 18.04) Version 27.0
      ___|\___|___|
=============================================================================

Welcome to Ubuntu 18.04.4 LTS (GNU/Linux 4.15.0-1060-aws x86_64v)

Please use one of the following commands to start the required environment with the framework of your choice:
for MXNet(+Keras2) with Python3 (CUDA 10.1 and Intel MKL-DNN) ____________________________________ source activate mxnet_p36
for MXNet(+Keras2) with Python2 (CUDA 10.1 and Intel MKL-DNN) ____________________________________ source activate mxnet_p27
for MXNet(+AWS Neuron) with Python3 ___________________________________________________ source activate aws_neuron_mxnet_p36
for TensorFlow(+Keras2) with Python3 (CUDA 10.0 and Intel MKL-DNN) __________________________ source activate tensorflow_p36
for TensorFlow(+Keras2) with Python2 (CUDA 10.0 and Intel MKL-DNN) __________________________ source activate tensorflow_p27
for Tensorflow(+AWS Neuron) with Python3 _________________________________________ source activate aws_neuron_tensorflow_p36
for TensorFlow 2(+Keras2) with Python3 (CUDA 10.1 and Intel MKL-DNN) _______________________ source activate tensorflow2_p36
for TensorFlow 2(+Keras2) with Python2 (CUDA 10.1 and Intel MKL-DNN) _______________________ source activate tensorflow2_p27
for PyTorch with Python3 (CUDA 10.1 and Intel MKL) _____________________________________________ source activate pytorch_p36
for PyTorch with Python2 (CUDA 10.1 and Intel MKL) _____________________________________________ source activate pytorch_p27
for PyTorch (+AWS Neuron) with Python3 ______________________________________________ source activate aws_neuron_pytorch_p36
for Chainer with Python2 (CUDA 10.0 and Intel iDeep) ___________________________________________ source activate chainer_p27
for Chainer with Python3 (CUDA 10.0 and Intel iDeep) ___________________________________________ source activate chainer_p36
for base Python2 (CUDA 10.0) _______________________________________________________________________ source activate python2
for base Python3 (CUDA 10.0) _______________________________________________________________________ source activate python3

Official Conda User Guide: https://docs.conda.io/projects/conda/en/latest/user-guide/
AWS Deep Learning AMI Homepage: https://aws.amazon.com/machine-learning/amis/
Developer Guide and Release Notes: https://docs.aws.amazon.com/dlami/latest/devguide/what-is-dlami.html
Support: https://forums.aws.amazon.com/forum.jspa?forumID=263
For a fully managed experience, check out Amazon SageMaker at https://aws.amazon.com/sagemaker
When using INF1 type instances, please update regularly using the instructions at: https://github.com/aws/aws-neuron-sdk/tree/master/release-notes
=============================================================================

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Sat Mar 14 14:53:59 EDT 2020

  System load:  0.0                Processes:              135
  Usage of /:   72.8% of 87.18GB   Users logged in:        2
  Memory usage: 19%                IP address for ens5:    1x.1xx.1x.xxxx
  Swap usage:   0%                 IP address for docker0: 172.xx.x.x

 * Canonical Livepatch is available for installation.
   - Reduce system reboots and improve kernel security. Activate at:
     https://ubuntu.com/livepatch
0 packages can be updated.
0 updates are security updates.
Failed to connect to https://changelogs.ubuntu.com/meta-release-lts. Check your Internet connection or proxy settings

The full text of the error is in the attached file. (which is now in this msg top) I have no graphical access to this machine as far as I know. Trying to use it for neovim v0.3.8 I am quite excited about using Kite for my coding. I can't believe how wonderful it is on my windows. Thanks much,

Hoon,

jansorg commented 4 years ago

@yelled1 Hi, the installation seems to fail due to a missing xdg-mime command. On Ubuntu this is usually istalled via sudo apt-get install xdg-utils. Could you install the package and try the installation again?

yelled1 commented 4 years ago

Actually, I should have mentioned that xgd-mime had been installed & that problem was solved. I was able to get nvim-qt xhosted & display. So, afterward, I managed to get into GUI, but GUI will just ask me to wait.... & wait.... went no where. I tried 3 different times & tried to hack json files to add proxy settings. Thanks for the response,

jansorg commented 4 years ago

Kite should automatically pick up your proxy settings by default. Is the GUI of nvim-qt blocking or the Kite UI?

Could you execute wget -S http://localhost:46624/clientapi/ping on the EC2 machine and post the output?

Is vim on the console working when you log into the machine via ssh? Instructions for manual installation are at https://github.com/kiteco/vim-plugin/blob/master/DEVELOPMENT.md.

yelled1 commented 4 years ago

Hi:

Here's the wget output

~$ wget -S http://localhost:46624/clientapi/ping
--2020-03-27 08:33:13--  http://localhost:46624/clientapi/ping
Resolving nyproxy.mycompany.local (nyproxy.mycompany.local)... 172.30.10.16
Connecting to nyproxy.mycompany.local (nyproxy.mycompany.local)|172.30.10.16|:80... connected.
Proxy request sent, awaiting response...
  HTTP/1.1 301 Moved Permanently
  Server: BlueCoat-Security-Appliance
  Location:http://10.175.15.216
  Connection: Close
Location: http://10.175.15.216 [following]
--2020-03-27 08:33:13--  http://10.175.15.216/
Connecting to nyproxy.mycompany.local (nyproxy.mycompany.local)|172.30.10.16|:80... connected.
Proxy request sent, awaiting response...
  HTTP/1.1 503 Service Unavailable
  Cache-Control: no-cache
  Pragma: no-cache
  Content-Type: text/html; charset=utf-8
  Proxy-Connection: Keep-Alive
  Connection: Keep-Alive
  Content-Length: 787
2020-03-27 08:34:28 ERROR 503: Service Unavailable.

I closed nvim-qt prior to launching Kite installer & I do see Kite installer GUI up (with dark background) & keep going initializing, but can't get past beyond that. I have all my machines running Kite. So, I know that this should only be few sec & next splash the login screen. I might be getting timed out on ssh. While installing as the internal network speed is not exactly Lode Runner. I do see after a while ERROR 503: Service Unavailable. Below is the latest installation attempt this morning. No errors. Thank you much,

Hoon,

myUser@ec2-nyme: $ bash -c "$(wget -q -O - https://linux.kite.com/dls/linux/current)"
This script will install Kite!
We hope you enjoy! If you run into any issues, please reach out at support@kite.com or feedback@kite.com
.. (snip) ...
Running /home/myUser/kite-installer install
[installer] no previous kite installation found
[installer] latest version is 2.20200324.3, downloading now...
[installer] Downloading Kite:   99.9% of 370MiB
[installer] verifying checksum
[installer] validating signature
[installer] installing version 2.20200324.3
[installer] installed ~/.config/autostart/kite-autostart.desktop
[installer] installed ~/.config/systemd/user/kite-updater.service
[installer] installed ~/.config/systemd/user/kite-updater.timer
[installer] installed ~/.local/share/applications/kite-copilot.desktop
[installer] installed ~/.local/share/applications/kite.desktop
[installer] installed ~/.local/share/icons/hicolor/128x128/apps/kite.png
[installer] installed ~/.local/share/kite/kited
[installer] installed ~/.local/share/kite/uninstall
[installer] installed ~/.local/share/kite/update
[installer] activating kite-updater systemd service
[installer] registering kite:// protocol handler
[installer] kite is installed! launching now! happy coding! :)
[installer] run /home/ny11559/.local/share/kite/kited
[installer]     or launch it using the Applications Menu
Removing kite-installer
myUser@ec2-nyme:~$
yelled1 commented 4 years ago

Also, I killed all kite session & reran as below. The Xhosted GUI comes up again & it keeps on displaying Initializing... with blue dots keep on running on the GUI.

:~$ ps -ef | grep kite
myUserId   7792     1  0 08:25 pts/0    00:00:00 /bin/sh /home/myUserId/.local/share/kite/kited
myUserId   7798  7792  0 08:25 pts/0    00:00:19 /home/myUserId/.local/share/kite/kite-v2.20200324.3/kited
myUserId   7813  7798  1 08:25 pts/0    00:00:35 [kite] <defunct>
myUserId   9231  5046  0 09:19 pts/0    00:00:00 grep --color=auto kite
myUserId@ec2-myUserId:~$ kill -9 7792
myUserId@ec2-myUserId:~$ kill -9 7798
myUserId@ec2-myUserId:~$ ps -ef | grep kite
myUserId   9233  5046  0 09:19 pts/0    00:00:00 grep --color=auto kite
myUserId@ec2-myUserId:~$ /bin/sh /home/myUserId/.local/share/kite/kited
starting kited 2.20200324.3
[region= release=] 2020/03/27 09:19:50.377891 lib.go:14: loading tensorflow dynamically...
[region= release=] 2020/03/27 09:19:50.377978 loader_linux.go:17: Loading tensorflow libary libtensorflow.so.1.15.0
[region= release=] 2020/03/27 09:19:50.453673 lib.go:11: loading tensorflow took 75.767097ms
2020-03-27 09:19:53.914101: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-03-27 09:19:53.920682: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2199850000 Hz
2020-03-27 09:19:53.920909: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f51900d7090 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-03-27 09:19:53.920943: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
jansorg commented 4 years ago

The kite is opening port 46624 on the local machine (localhost). It seems that the proxy is used for that network connection. A (external) proxy must not be used to talk to the local machine. The Kite UI is supposed to not use the proxy for localhost, but this seems to fail here.

Could you try unsetting the proxy vars and then run kite?

# kill the kite processes first
export http_proxy="" https_proxy=""
/bin/sh /home/myUserId/.local/share/kite/kited
yelled1 commented 4 years ago

Awesome! That did the job & my ec2 is flying. Thank you so much. Two more quick Qs. 1) Whenever I need to update for Kite. I need to: export http_proxy="" https_proxy=""? 2) Also, should I set Proxy Custom to equal my browser's proxy? Thank you again!

I got the welcome msg & that Kite is disabling deoplete msg. It has not kicked in yet (not autocompleting yet), but I am sure it will soon. Will keep you posted!

jansorg commented 4 years ago

Great! Either use a custom script with the commands above or try to set the proxy to None. Kite is installed on the local machine and doesn't need external hosts to work, but this setting isn't fully supported. But just try that and it should hopefully work well enough :) Proxy Custom won't help, I think, because it's still having the problem that it's used for the local machine.

yelled1 commented 4 years ago

It's finally kicked in after a little tweak to init.vim *(may be just need a moment or two). Flying with nvim & Kite!

yelled1 commented 4 years ago

Changed as below. I think this would be just local to daemon process that launches Kite. Since "" worked I won't bother with export http_proxy=none. Pls note that this is only partial snipped script file.

~$ head ~/.local/share/kite/kited
#!/bin/sh

# This is to implement: https://github.com/kiteco/plugins/issues/181
export http_proxy=""
export https_proxy=""

ROOT="$HOME/.local/share/kite"
CURRENT="$ROOT/current"

CURRENT_VERSION="$(basename "$(readlink -f "$CURRENT")" | grep "kite-v" | sed -e 's/^kite-v//g')"

... (sniped) ....