fail to install JSX on a fresh MacOS machine

GlenDC commented 5 years ago

I have no previous installation, on my host machine, no scripts left from old installations, and no containers.

I follow the 3-steps of the "Interactive Container" installation process as described on https://github.com/threefoldtech/jumpscaleX/blob/development_jumpscale/docs/Installation/README.md#interactive-container

It fails always at the same step:

 glendc@gdcwmbp  ~  curl https://raw.githubusercontent.com/threefoldtech/jumpscaleX/development_jumpscale/install/jsx.py\?$RANDOM > /tmp/jsx ;
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 19121  100 19121    0     0   209k      0 --:--:-- --:--:-- --:--:--  209k
 glendc@gdcwmbp  ~  chmod +x /tmp/jsx;
 glendc@gdcwmbp  ~  /tmp/jsx container_install
DOWNLOADED INSTALLTOOLS TO /tmp/InstallTools.py
provide secret to use for encrypting private key: 
provide secret to use for encrypting private key (confirm): 
provide secret to use for encrypting private key: 
provide secret to use for encrypting private key (confirm): 
 - Docker machine gets created: 
af23f54815c5281268dcc02828f4fbcd54ce2528e9ca4a43c3b7174594e4ebb7
 - Docker machine OK
 - Start SSH server
cat: /root/.BASEINSTALL_OK: No such file or directory
 * Starting OpenBSD Secure Shell server sshd                             [ OK ] 
 - Upgrade ubuntu
Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Hit:2 http://ppa.launchpad.net/wireguard/wireguard/ubuntu bionic InRelease     
Hit:3 http://archive.ubuntu.com/ubuntu bionic InRelease                        
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]      
Hit:5 http://mirror.unix-solutions.be/ubuntu bionic InRelease                  
Get:6 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]    
Get:7 http://security.ubuntu.com/ubuntu bionic-security/universe Sources [190 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Sources [4867 B]
Get:9 http://security.ubuntu.com/ubuntu bionic-security/main Sources [150 kB]  
Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main Sources [368 kB]   
Get:11 http://security.ubuntu.com/ubuntu bionic-security/multiverse Sources [2776 B]
Get:12 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [4172 B]
Get:13 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [725 kB]
Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/universe Sources [331 kB]
Get:15 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [580 kB]
Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1245 kB]
Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [889 kB]
Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [7235 B]
Fetched 4748 kB in 3s (1610 kB/s)                      
Reading package lists... Done
Building dependency tree       
Reading state information... Done
11 packages can be upgraded. Run 'apt list --upgradable' to see them.
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
  linux-headers-generic
The following packages will be upgraded:
  base-files bash iputils-ping libglib2.0-0 libssl-dev libssl1.1 openssl
  redis-server redis-tools tmux
10 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
Need to get 6177 kB of archives.
After this operation, 24.6 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 base-files amd64 10.1ubuntu2.5 [60.2 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 bash amd64 4.4.18-2ubuntu1.2 [614 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 iputils-ping amd64 3:20161105-1ubuntu3 [54.2 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libglib2.0-0 amd64 2.56.4-0ubuntu0.18.04.4 [1169 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.4 [1566 kB]
Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl1.1 amd64 1.1.1-1ubuntu2.1~18.04.4 [1300 kB]
Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 openssl amd64 1.1.1-1ubuntu2.1~18.04.4 [613 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 redis-server amd64 5:4.0.9-1ubuntu0.2 [35.4 kB]
Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 redis-tools amd64 5:4.0.9-1ubuntu0.2 [516 kB]
Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 tmux amd64 2.6-3ubuntu0.2 [248 kB]
Fetched 6177 kB in 3s (2298 kB/s)
Preconfiguring packages ...
(Reading database ... 55691 files and directories currently installed.)
Preparing to unpack .../base-files_10.1ubuntu2.5_amd64.deb ...
Unpacking base-files (10.1ubuntu2.5) over (10.1ubuntu2.4) ...
Setting up base-files (10.1ubuntu2.5) ...
(Reading database ... 55691 files and directories currently installed.)
Preparing to unpack .../bash_4.4.18-2ubuntu1.2_amd64.deb ...
Unpacking bash (4.4.18-2ubuntu1.2) over (4.4.18-2ubuntu1.1) ...
Setting up bash (4.4.18-2ubuntu1.2) ...
update-alternatives: error: alternative path /usr/share/man/man7/bash-builtins.7.gz doesn't exist
(Reading database ... 55691 files and directories currently installed.)
Preparing to unpack .../0-iputils-ping_3%3a20161105-1ubuntu3_amd64.deb ...
Unpacking iputils-ping (3:20161105-1ubuntu3) over (3:20161105-1ubuntu2) ...
Preparing to unpack .../1-libglib2.0-0_2.56.4-0ubuntu0.18.04.4_amd64.deb ...
Unpacking libglib2.0-0:amd64 (2.56.4-0ubuntu0.18.04.4) over (2.56.4-0ubuntu0.18.04.3) ...
Preparing to unpack .../2-libssl-dev_1.1.1-1ubuntu2.1~18.04.4_amd64.deb ...
Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.4) over (1.1.1-1ubuntu2.1~18.04.3) ...
Preparing to unpack .../3-libssl1.1_1.1.1-1ubuntu2.1~18.04.4_amd64.deb ...
Unpacking libssl1.1:amd64 (1.1.1-1ubuntu2.1~18.04.4) over (1.1.1-1ubuntu2.1~18.04.3) ...
Preparing to unpack .../4-openssl_1.1.1-1ubuntu2.1~18.04.4_amd64.deb ...
Unpacking openssl (1.1.1-1ubuntu2.1~18.04.4) over (1.1.1-1ubuntu2.1~18.04.3) ...
Preparing to unpack .../5-redis-server_5%3a4.0.9-1ubuntu0.2_amd64.deb ...
Unpacking redis-server (5:4.0.9-1ubuntu0.2) over (5:4.0.9-1ubuntu0.1) ...
Preparing to unpack .../6-redis-tools_5%3a4.0.9-1ubuntu0.2_amd64.deb ...
Unpacking redis-tools (5:4.0.9-1ubuntu0.2) over (5:4.0.9-1ubuntu0.1) ...
Preparing to unpack .../7-tmux_2.6-3ubuntu0.2_amd64.deb ...
Unpacking tmux (2.6-3ubuntu0.2) over (2.6-3ubuntu0.1) ...
Setting up iputils-ping (3:20161105-1ubuntu3) ...
Setting up redis-tools (5:4.0.9-1ubuntu0.2) ...
Setting up libglib2.0-0:amd64 (2.56.4-0ubuntu0.18.04.4) ...
No schema files found: doing nothing.
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Setting up libssl1.1:amd64 (1.1.1-1ubuntu2.1~18.04.4) ...
Setting up openssl (1.1.1-1ubuntu2.1~18.04.4) ...
Setting up tmux (2.6-3ubuntu0.2) ...
Setting up redis-server (5:4.0.9-1ubuntu0.2) ...
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of restart.
Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.4) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
 - Upgrade ubuntu ended
Reading package lists... Done
Building dependency tree       
Reading state information... Done
mc is already the newest version (3:4.8.19-1).
git is already the newest version (1:2.17.1-1ubuntu0.4).
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
getaddrinfo 3bot: nodename nor servname provided, or not known
getaddrinfo 3bot: nodename nor servname provided, or not known
getaddrinfo 3bot: nodename nor servname provided, or not known
copy installer over from where I install from
Warning: Permanently added '[localhost]:9122' (ECDSA) to the list of known hosts.
Warning: Permanently added '[localhost]:9122' (ECDSA) to the list of known hosts.
 - Installing jumpscaleX 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
python3-click is already the newest version (6.7-3).
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
Connection to localhost closed.
DOWNLOADED INSTALLTOOLS TO /tmp/InstallTools.py
Fri 19 08:50:29 InstallTools.py  -2999 - repos_get                          : get code:jumpscaleX:threefoldtech (['development_jumpscale'])
Fri 19 08:50:29 InstallTools.py  -2999 - repos_get                          : get code [git] (first time): jumpscaleX

EXCEPTION

        jsx                                                          : 562 : cli()
        /root/.local/lib/python3.6/site-packages/click/core.py       : 764 : return self.main(*args, **kwargs)
        /root/.local/lib/python3.6/site-packages/click/core.py       : 717 : rv = self.invoke(ctx)
        /root/.local/lib/python3.6/site-packages/click/core.py       : 1137: return _process_result(sub_ctx.command.invoke(sub_ctx))
        /root/.local/lib/python3.6/site-packages/click/core.py       : 956 : return ctx.invoke(self.callback, **ctx.params)
        /root/.local/lib/python3.6/site-packages/click/core.py       : 555 : return callback(*args, **kwargs)
        jsx                                                          : 299 : installer.install(sandboxed=False, force=force, gitpull=pull)
        /tmp/InstallTools.py                                         : 2944: self.repos_get(pull=gitpull)
        /tmp/InstallTools.py                                         : 3002: Tools.code_github_get(repo=sourceName, account=self.account, branch=self.branch, pull=pull)
        /tmp/InstallTools.py                                         : 1742: rc, _, _ = Tools.execute(C, args=args, die=True, showout=False)
        /tmp/InstallTools.py                                         : 1255: original_command=command,
        /tmp/InstallTools.py                                         : 1417: raise RuntimeError(msg)

Fri 19 08:51:06 core.py          - 762 -                                    : EXCEPTION: 
Could not execute: 'bash /tmp/jumpscale/scripts/62544.sh'stderr:
    Cloning into 'jumpscaleX'...
    ssh_exchange_identification: Connection closed by remote host
    fatal: Could not read from remote repository.

    Please make sure you have the correct access rights
    and the repository exists.

CANNOT CONTINUE
Connection to localhost closed.

EXCEPTION

        /tmp/jsx                                                     : 562 : cli()
        /usr/local/lib/python3.7/site-packages/click/core.py         : 764 : return self.main(*args, **kwargs)
        /usr/local/lib/python3.7/site-packages/click/core.py         : 717 : rv = self.invoke(ctx)
        /usr/local/lib/python3.7/site-packages/click/core.py         : 1137: return _process_result(sub_ctx.command.invoke(sub_ctx))
        /usr/local/lib/python3.7/site-packages/click/core.py         : 956 : return ctx.invoke(self.callback, **ctx.params)
        /usr/local/lib/python3.7/site-packages/click/core.py         : 555 : return callback(*args, **kwargs)
        /tmp/jsx                                                     : 242 : docker.jumpscale_install(branch=branch, redo=reinstall, pull=pull, web=web)
        /tmp/InstallTools.py                                         : 3558: self.sshexec(cmd)
        /tmp/InstallTools.py                                         : 3400: Tools.execute(cmd2, interactive=True, showout=False, replace=False, asfile=True)
        /tmp/InstallTools.py                                         : 1255: original_command=command,
        /tmp/InstallTools.py                                         : 1262: res = Tools._execute_interactive(cmd=command, die=die, original_command=original_command)
        /tmp/InstallTools.py                                         : 667 : % (original_command, returncode)

Fri 19 10:51:06 core.py          - 762 -                                    : EXCEPTION: ***ERROR EXECUTE INTERACTIVE:
Could not execute:ssh -oStrictHostKeyChecking=no -t root@localhost -A -p 9122 'cd /tmp;python3 jsx install'
returncode:1

CANNOT CONTINUE

Note that I do have an ssh-key loaded in my host system.

GlenDC commented 5 years ago

@zgorizzo69 helped me, and allowed me to install JSX as follows from my host machine (with the docker container already created by my failed installation, but manually stopped it prior to running these commands):

$ cd ~/sandbox/code/github/threefoldtech
$ git clone git@github.com:threefoldtech/jumpscaleX.git
$ cd jumpscaleX
$ git checkout development_jumpscale
$ cd install
$ python3 jsx.py container_install

After that is finished I am able to go into the kosmos from my host machine using:

$ python3 jsx.py container_kosmos

despiegk commented 5 years ago

we should know what the issue was. The installer should be smart enough to figure out how to clone and which credentials to use.

are we sure the right branch was used for the jsx install tool?

GlenDC commented 5 years ago

are we sure the right branch was used for the jsx install tool?

100% sure it was the correct branch, if the correct branch is development_jumpscale that is, as is confirmed by the logs above

waleedhammam commented 5 years ago

Now, Checking installation on a freash MacOS Mojave

waleedhammam commented 5 years ago

On a fresh MacOS user:

ssh-keygen
ssh-add
ssh-add -L
added the key to GitHub

execute the following from docs

# download
curl https://raw.githubusercontent.com/threefoldtech/jumpscaleX/development_jumpscale/install/jsx.py?$RANDOM > /tmp/jsx ;\
# change permission
chmod +x /tmp/jsx; \
# install
/tmp/jsx container-install

and installation works Ok

Stacktrace from the point in the ticket


opy installer over from where I install from
Warning: Permanently added '[localhost]:9122' (ECDSA) to the list of known hosts.
Warning: Permanently added '[localhost]:9122' (ECDSA) to the list of known hosts.
- Installing jumpscaleX 
Warning: Permanently added '[localhost]:9122' (ECDSA) to the list of known hosts.
Reading package lists... Done
Building dependency tree       
Reading state information... Done
python3-click is already the newest version (6.7-3).
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
Connection to localhost closed.
DOWNLOADED INSTALLTOOLS TO /tmp/InstallTools.py
Ok to use key: 'test_jsx_installer@MacBookPro' as your default key?
make your choice (y,n): y
Mon 22 14:46:10 InstallTools.py  -3003 - repos_get                          : get code:jumpscaleX:threefoldtech (['development_jumpscale'])
Mon 22 14:46:10 InstallTools.py  -3003 - repos_get                          : get code [git] (first time): jumpscaleX
The authenticity of host 'github.com (140.82.118.3)' can't be established.
RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8.
Are you sure you want to continue connecting (yes/no)? yes
development_jumpscale
Mon 22 14:47:14 InstallTools.py  -1789 - checkoutbranch                     : Found branch: development_jumpscale
Mon 22 14:47:14 InstallTools.py  -3003 - repos_get                          : get code:digitalmeX:threefoldtech (['development_jumpscale'])
Mon 22 14:47:14 InstallTools.py  -3003 - repos_get                          : get code [git] (first time): digitalmeX
development_jumpscale
Mon 22 14:48:26 InstallTools.py  -1789 - checkoutbranch                     : Found branch: development_jumpscale
Mon 22 14:48:26 InstallTools.py  -2937 - install                            : link /sandbox/code/github/threefoldtech/jumpscaleX/Jumpscale to Jumpscale
Mon 22 14:48:26 InstallTools.py  -2937 - install                            : link /sandbox/code/github/threefoldtech/digitalmeX/DigitalMe to DigitalMe
Mon 22 14:48:28 JSBase.py        - 103 - systemfs                           : ***CLASS INIT 1: systemfs

process /sandbox/lib/jumpscale/DigitalMe/graphql.py
process /sandbox/lib/jumpscale/DigitalMe/init.py
process /sandbox/lib/jumpscale/DigitalMe/clients/init.py
process /sandbox/lib/jumpscale/DigitalMe/clients/gedis/init.py
process /sandbox/lib/jumpscale/DigitalMe/clients/gedis/GedisClientFactory.py
process /sandbox/lib/jumpscale/DigitalMe/clients/gedis/GedisClient.py
process /sandbox/lib/jumpscale/DigitalMe/clients/gedis/templates/GedisClientGenerated.py
process /sandbox/lib/jumpscale/DigitalMe/clients/gedis/templates/_archive/ModelBase.py
process /sandbox/lib/jumpscale/DigitalMe/clients/multicast/MulticastFactory.py
process /sandbox/lib/jumpscale/DigitalMe/clients/multicast/init.py
process /sandbox/lib/jumpscale/DigitalMe/clients/multicast/MulticastClient.py
process /sandbox/lib/jumpscale/DigitalMe/clients/gedis_backends/GunClient.py
process /sandbox/lib/jumpscale/DigitalMe/clients/gedis_backends/GedisBackendClientFactory.py
process /sandbox/lib/jumpscale/DigitalMe/clients/gedis_backends/init.py
process /sandbox/lib/jumpscale/DigitalMe/clients/gedis_backends/utils.py
```
## possible reasons for that issue
```
wifi and ethernet both connected so ssh conflict happens
issue with GitHub cloning with ssh
ssh-key is not added to GitHub authorized key or conflicted
multiple ssh keys loaded

Pishoy commented 5 years ago

steps as mentioned above in addition remove nay old files in case old installation like ~/sandbox before installation

jsx version inside docker

3BOTDEVEL:3bot:jumpscaleX: git log -1
commit f1d139fb46c3ee7dd7933db781ae628ac7ce047b (grafted, HEAD -> development_jumpscale, origin/development_jumpscale)
Author: despiegk <kristof@incubaid.com>
Date:   Tue Jul 23 18:11:26 2019 +0200

specs for metadata

threefoldtecharchive / jumpscaleX_archived

fail to install JSX on a fresh MacOS machine #734

Stacktrace from the point in the ticket