eclipse-che / che

Kubernetes based Cloud Development Environments for Enterprise Teams
http://eclipse.org/che
Eclipse Public License 2.0
6.96k stars 1.19k forks source link

Experiencing error when creating/starting a workspace. #8507

Closed MatthewC closed 6 years ago

MatthewC commented 6 years ago

Description

I had just installed Eclipe CHE on my Ubuntu Server, and was creating my first workspace, after a while I got an error mentioning that the workspace wasn't running, and when I clicked the Start button I got the error mention below.

Reproduction Steps

On Web GUI I get a message saying "Working is not running" and when clicking Start I get the error: Could not start workspace gCyborg. Reason: Start of environment 'default' failed. Error: Error occurs while initializing command [/bin/sh, -c, trap '[ -z "$(jobs -p)" ] || kill $(jobs -p); [ -e /tmp/docker-exec-34.pid ] && rm /tmp/docker-exec-34.pid' EXIT; echo $$>/tmp/docker-exec-34.pid; command -v pidof >/dev/null 2>&1 && { pidof che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1 } || { ps -fC che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1 }] in docker container ebbdff02acb6214e845b9902f98799313aeee5b039531f60798910e1c3931ab4: Error response from docker API, status: 409, message: Container ebbdff02acb6214e845b9902f98799313aeee5b039531f60798910e1c3931ab4 is not running

OS and version:
Ubuntu 16.04.3 LTS

Docker Version:

Client:
 Version:       18.01.0-ce
 API version:   1.35
 Go version:    go1.9.2
 Git commit:    03596f5
 Built: Wed Jan 10 20:11:05 2018
 OS/Arch:       linux/amd64
 Experimental:  false
 Orchestrator:  swarm

Server:
 Engine:
  Version:      18.01.0-ce
  API version:  1.35 (minimum version 1.12)
  Go version:   go1.9.2
  Git commit:   03596f5
  Built:        Wed Jan 10 20:09:37 2018
  OS/Arch:      linux/amd64
  Experimental: false

Diagnostics:

---------------  S Y S T E M  ---------------

OS:NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.6.2
PRETTY_NAME="Alpine Linux v3.6"
HOME_URL="http://alpinelinux.org"
BUG_REPORT_URL="http://bugs.alpinelinux.org"

uname:Linux 4.4.0-109-generic #132-Ubuntu SMP Tue Jan 9 19:52:39 UTC 2018 x86_64
libc:glibc 2.9 NPTL
rlimit: STACK 8192k, CORE infinity, NPROC infinity, NOFILE 1048576, AS infinity
load average:0.02 0.09 0.05

/proc/meminfo:
MemTotal:        1014212 kB
MemFree:          105164 kB
MemAvailable:     356048 kB
Buffers:           33924 kB
Cached:           336484 kB
SwapCached:            0 kB
Active:           610400 kB
Inactive:         165428 kB
Active(anon):     411724 kB
Inactive(anon):    15016 kB
Active(file):     198676 kB
Inactive(file):   150412 kB
Unevictable:        3652 kB
Mlocked:            3652 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:             40704 kB
Writeback:          5148 kB
AnonPages:        409056 kB
Mapped:            68660 kB
Shmem:             18872 kB
Slab:              82728 kB
SReclaimable:      58844 kB
SUnreclaim:        23884 kB
KernelStack:        5008 kB
PageTables:        12408 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      507104 kB
Committed_AS:    1784828 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:    260096 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       86016 kB
DirectMap2M:      962560 kB

CPU:total 1 (initial active 1) (1 cores per cpu, 1 threads per core) family 6 model 63 stepping 2, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, lzcnt, tsc, bmi1, bmi2

/proc/cpuinfo:
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 63
model name      : Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GHz
stepping        : 2
microcode       : 0x3b
cpu MHz         : 2400.072
cache size      : 30720 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single kaiser fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt
bugs            :
bogomips        : 4800.14
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

Memory: 4k page, physical 1014212k(105164k free), swap 0k(0k free)

vm_info: OpenJDK 64-Bit Server VM (25.131-b11) for linux-amd64 JRE (1.8.0_131-b11), built on Jun 16 2017 13:51:29 by "buildozer" with gcc 6.3.0

time: Mon Jan 29 20:23:40 2018
elapsed time: 0 seconds (0d 0h 0m 0s)

Using embedded assembly...
Found custom che.properties...
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000085330000, 449642496, 0) failed; error='Out of memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 449642496 bytes for committing reserved memory.
# Can not save log file, dump to screen..
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 449642496 bytes for committing reserved memory.
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Use 64 bit Java on a 64 bit OS
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (os_linux.cpp:2658), pid=38, tid=0x00007fb9d53dfae8
#
# JRE version:  (8.0_131-b11) (build )
# Java VM: OpenJDK 64-Bit Server VM (25.131-b11 mixed mode linux-amd64 compressed oops)
# Derivative: IcedTea 3.4.0
# Distribution: Custom build (Fri Jun 16 13:41:54 GMT 2017)
# Core dump written. Default location: //core or core.38
#

---------------  T H R E A D  ---------------

Current thread (0x0000563914b51000):  JavaThread "Unknown thread" [_thread_in_vm, id=42, stack(0x00007fb9d52df000,0x00007fb9d53dfaa8)]

Stack: [0x00007fb9d52df000,0x00007fb9d53dfaa8]

---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )

Other Threads:

=>0x0000563914b51000 (exited) JavaThread "Unknown thread" [_thread_in_vm, id=42, stack(0x00007fb9d52df000,0x00007fb9d53dfaa8)]

VM state:not at safepoint (not fully initialized)

VM Mutex/Monitor currently owned by a thread: None

GC Heap History (0 events):
No events

Deoptimization events (0 events):
No events

Internal exceptions (0 events):
No events

Events (0 events):
No events

Dynamic libraries:
80000000-85330000 rw-p 00000000 00:00 0
a0000000-100000000 ---p 00000000 00:00 0
5639142e7000-5639142e8000 r-xp 00000000 00:2b 82                         /usr/lib/jvm/java-1.8-openjdk/jre/bin/java
5639144e7000-5639144e8000 r--p 00000000 00:2b 82                         /usr/lib/jvm/java-1.8-openjdk/jre/bin/java
5639144e8000-5639144e9000 rw-p 00001000 00:2b 82                         /usr/lib/jvm/java-1.8-openjdk/jre/bin/java
563914b39000-563914b68000 rw-p 00000000 00:00 0                          [heap]
7fb9c3cc0000-7fb9c3deb000 rw-p 00000000 00:00 0
7fb9c3deb000-7fb9c41c1000 ---p 00000000 00:00 0
7fb9c41c1000-7fb9c41c2000 rw-p 00000000 00:00 0
7fb9c41c2000-7fb9c41c3000 ---p 00000000 00:00 0
7fb9c41c3000-7fb9c42ce000 rw-p 00000000 00:00 0
7fb9c42ce000-7fb9c4684000 ---p 00000000 00:00 0
7fb9c4684000-7fb9c48f4000 rwxp 00000000 00:00 0
7fb9c48f4000-7fb9d3684000 ---p 00000000 00:00 0
7fb9d3684000-7fb9d388b000 r-xp 00000000 00:2b 103                        /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/libzip.so
7fb9d388b000-7fb9d388c000 r--p 00007000 00:2b 103                        /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/libzip.so
7fb9d388c000-7fb9d388d000 rw-p 00008000 00:2b 103                        /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/libzip.so
7fb9d388d000-7fb9d3ab9000 r-xp 00000000 00:2b 89                         /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/libjava.so
7fb9d3ab9000-7fb9d3aba000 r--p 0002c000 00:2b 89                         /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/libjava.so
7fb9d3aba000-7fb9d3abb000 rw-p 0002d000 00:2b 89                         /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/libjava.so
7fb9d3abb000-7fb9d3abc000 rw-p 00000000 00:00 0
7fb9d3abc000-7fb9d3cc8000 r-xp 00000000 00:2b 98                         /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/libverify.so
7fb9d3cc8000-7fb9d3cca000 r--p 0000c000 00:2b 98                         /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/libverify.so
7fb9d3cca000-7fb9d3ccb000 rw-p 0000e000 00:2b 98                         /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/libverify.so
7fb9d3ccb000-7fb9d3edb000 r-xp 00000000 00:2b 97                         /usr/lib/libgcc_s.so.1
7fb9d3edb000-7fb9d3edc000 r--p 00010000 00:2b 97                         /usr/lib/libgcc_s.so.1
7fb9d3edc000-7fb9d3edd000 rw-p 00011000 00:2b 97                         /usr/lib/libgcc_s.so.1
7fb9d3edd000-7fb9d421d000 r-xp 00000000 00:2b 96                         /usr/lib/libstdc++.so.6.0.22
7fb9d421d000-7fb9d422b000 r--p 00140000 00:2b 96                         /usr/lib/libstdc++.so.6.0.22
7fb9d422b000-7fb9d422c000 rw-p 0014e000 00:2b 96                         /usr/lib/libstdc++.so.6.0.22
7fb9d422c000-7fb9d422f000 rw-p 00000000 00:00 0
7fb9d422f000-7fb9d4c46000 r-xp 00000000 00:2b 92                         /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server/libjvm.so
7fb9d4c46000-7fb9d4cdb000 r--p 00817000 00:2b 92                         /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server/libjvm.so
7fb9d4cdb000-7fb9d4d03000 rw-p 008ac000 00:2b 92                         /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server/libjvm.so
7fb9d4d03000-7fb9d4d33000 rw-p 00000000 00:00 0
7fb9d4d33000-7fb9d4f48000 r-xp 00000000 00:2b 88                         /lib/libz.so.1.2.11
7fb9d4f48000-7fb9d4f49000 r--p 00015000 00:2b 88                         /lib/libz.so.1.2.11
7fb9d4f49000-7fb9d4f4a000 rw-p 00016000 00:2b 88                         /lib/libz.so.1.2.11
7fb9d4f4a000-7fb9d5156000 r-xp 00000000 00:2b 86                         /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/jli/libjli.so
7fb9d5156000-7fb9d5157000 r--p 0000c000 00:2b 86                         /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/jli/libjli.so
7fb9d5157000-7fb9d5158000 rw-p 0000d000 00:2b 86                         /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/jli/libjli.so
7fb9d5158000-7fb9d51e1000 r-xp 00000000 00:2b 30                         /lib/ld-musl-x86_64.so.1
7fb9d52d3000-7fb9d52db000 rw-s 00000000 00:2b 102                        /tmp/hsperfdata_root/38
7fb9d52db000-7fb9d52dc000 rw-p 00000000 00:00 0
7fb9d52dc000-7fb9d52dd000 r--p 00000000 00:00 0
7fb9d52dd000-7fb9d52df000 ---p 00000000 00:00 0
7fb9d52df000-7fb9d52e2000 ---p 00000000 00:00 0
7fb9d52e2000-7fb9d53e0000 rw-p 00000000 00:00 0
7fb9d53e0000-7fb9d53e1000 r--p 00088000 00:2b 30                         /lib/ld-musl-x86_64.so.1
7fb9d53e1000-7fb9d53e2000 rw-p 00089000 00:2b 30                         /lib/ld-musl-x86_64.so.1
7fb9d53e2000-7fb9d53e5000 rw-p 00000000 00:00 0
7ffe1560d000-7ffe1562f000 rw-p 00000000 00:00 0                          [stack]
7ffe15674000-7ffe15677000 r--p 00000000 00:00 0                          [vvar]
7ffe15677000-7ffe15679000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

VM Arguments:
jvm_args: -Djava.util.logging.config.file=/home/user/eclipse-che//tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xms512m -Xmx2048m -Djava.security.egd=file:/dev/./urandom -XX:+HeapDumpOnOutOfMemoryError -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+UseCMSInitiatingOccupancyOnly -XX:+ScavengeBeforeFullGC -XX:+CMSScavengeBeforeRemark -Dche.local.conf.dir=/conf -Dche.docker.network=bridge -Dport.http=6868 -Dche.home=/home/user/eclipse-che/ -Dche.logs.dir=/logs -Dche.logs.level=INFO -Djuli-logback.configurationFile=file:/home/user/eclipse-che//tomcat/conf/tomcat-logger.xml -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dche.local.conf.dir=/conf -Dcatalina.base=/home/user/eclipse-che//tomcat -Dcatalina.home=/home/user/eclipse-che//tomcat -Djava.io.tmpdir=/home/user/eclipse-che//tomcat/temp
java_command: org.apache.catalina.startup.Bootstrap start
java_class_path (initial): /home/user/eclipse-che//tomcat/conf/:/usr/lib/jvm/java-1.8-openjdk/jre/lib/tools.jar:/home/user/eclipse-che//tomcat/bin/bootstrap.jar:/home/user/eclipse-che//tomcat/bin/tomcat-juli.jar
Launcher Type: SUN_STANDARD

Environment Variables:
JAVA_HOME=/usr/lib/jvm/java-1.8-openjdk/jre
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/jvm/java-1.8-openjdk/jre/bin:/usr/lib/jvm/java-1.8-openjdk/bin
LD_LIBRARY_PATH=/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64

Signal Handlers:
SIGSEGV: [libjvm.so+0x63094c], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGBUS: [libjvm.so+0x63094c], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGFPE: [libjvm.so+0x55ffac], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGPIPE: [libjvm.so+0x55ffac], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGXFSZ: [libjvm.so+0x55ffac], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGILL: [libjvm.so+0x55ffac], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGUSR1: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGUSR2: [libjvm.so+0x56014b], sa_mask[0]=00000000000000000000000000000000, sa_flags=SA_RESTART|SA_SIGINFO
SIGHUP: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGINT: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGTERM: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGQUIT: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none

---------------  S Y S T E M  ---------------

OS:NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.6.2
PRETTY_NAME="Alpine Linux v3.6"
HOME_URL="http://alpinelinux.org"
BUG_REPORT_URL="http://bugs.alpinelinux.org"

uname:Linux 4.4.0-109-generic #132-Ubuntu SMP Tue Jan 9 19:52:39 UTC 2018 x86_64
libc:glibc 2.9 NPTL
rlimit: STACK 8192k, CORE infinity, NPROC infinity, NOFILE 1048576, AS infinity
load average:0.00 0.07 0.05

/proc/meminfo:
MemTotal:        1014212 kB
MemFree:          104672 kB
MemAvailable:     355896 kB
Buffers:           33992 kB
Cached:           336776 kB
SwapCached:            0 kB
Active:           611876 kB
Inactive:         164248 kB
Active(anon):     411680 kB
Inactive(anon):    15016 kB
Active(file):     200196 kB
Inactive(file):   149232 kB
Unevictable:        3652 kB
Mlocked:            3652 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:             40676 kB
Writeback:          5132 kB
AnonPages:        409020 kB
Mapped:            68468 kB
Shmem:             18872 kB
Slab:              82732 kB
SReclaimable:      58844 kB
SUnreclaim:        23888 kB
KernelStack:        4992 kB
PageTables:        12408 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      507104 kB
Committed_AS:    1783420 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:    268288 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       86016 kB
DirectMap2M:      962560 kB

CPU:total 1 (initial active 1) (1 cores per cpu, 1 threads per core) family 6 model 63 stepping 2, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, lzcnt, tsc, bmi1, bmi2

/proc/cpuinfo:
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 63
model name      : Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GHz
stepping        : 2
microcode       : 0x3b
cpu MHz         : 2400.072
cache size      : 30720 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single kaiser fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt
bugs            :
bogomips        : 4800.14
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

Memory: 4k page, physical 1014212k(104672k free), swap 0k(0k free)

vm_info: OpenJDK 64-Bit Server VM (25.131-b11) for linux-amd64 JRE (1.8.0_131-b11), built on Jun 16 2017 13:51:29 by "buildozer" with gcc 6.3.0

time: Mon Jan 29 20:24:40 2018
elapsed time: 0 seconds (0d 0h 0m 0s)

Using embedded assembly...
Found custom che.properties...
2018-01-29 20:26:20,592[main]             [INFO ] [o.a.c.s.VersionLoggerListener 89]    - Server version:        Apache Tomcat/8.5.11
2018-01-29 20:26:20,599[main]             [INFO ] [o.a.c.s.VersionLoggerListener 91]    - Server built:          Jan 10 2017 21:02:52 UTC
2018-01-29 20:26:20,600[main]             [INFO ] [o.a.c.s.VersionLoggerListener 93]    - Server number:         8.5.11.0
2018-01-29 20:26:20,600[main]             [INFO ] [o.a.c.s.VersionLoggerListener 95]    - OS Name:               Linux
2018-01-29 20:26:20,601[main]             [INFO ] [o.a.c.s.VersionLoggerListener 97]    - OS Version:            4.4.0-112-generic
2018-01-29 20:26:20,602[main]             [INFO ] [o.a.c.s.VersionLoggerListener 99]    - Architecture:          amd64
2018-01-29 20:26:20,604[main]             [INFO ] [o.a.c.s.VersionLoggerListener 101]   - Java Home:             /usr/lib/jvm/java-1.8-openjdk/jre
2018-01-29 20:26:20,605[main]             [INFO ] [o.a.c.s.VersionLoggerListener 103]   - JVM Version:           1.8.0_131-b11
2018-01-29 20:26:20,605[main]             [INFO ] [o.a.c.s.VersionLoggerListener 105]   - JVM Vendor:            Oracle Corporation
2018-01-29 20:26:20,606[main]             [INFO ] [o.a.c.s.VersionLoggerListener 107]   - CATALINA_BASE:         /home/user/eclipse-che/tomcat
2018-01-29 20:26:20,609[main]             [INFO ] [o.a.c.s.VersionLoggerListener 109]   - CATALINA_HOME:         /home/user/eclipse-che/tomcat
2018-01-29 20:26:20,614[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Djava.util.logging.config.file=/home/user/eclipse-che//tomcat/conf/logging.properties
2018-01-29 20:26:20,614[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
2018-01-29 20:26:20,615[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Xms512m
2018-01-29 20:26:20,616[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Xmx2048m
2018-01-29 20:26:20,617[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Djava.security.egd=file:/dev/./urandom
2018-01-29 20:26:20,617[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -XX:+HeapDumpOnOutOfMemoryError
2018-01-29 20:26:20,617[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -XX:+UseConcMarkSweepGC
2018-01-29 20:26:20,620[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -XX:+CMSParallelRemarkEnabled
2018-01-29 20:26:20,620[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -XX:+UseCMSInitiatingOccupancyOnly
2018-01-29 20:26:20,621[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -XX:+ScavengeBeforeFullGC
2018-01-29 20:26:20,621[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -XX:+CMSScavengeBeforeRemark
2018-01-29 20:26:20,621[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dche.local.conf.dir=/conf
2018-01-29 20:26:20,622[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dche.docker.network=bridge
2018-01-29 20:26:20,623[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dport.http=6868
2018-01-29 20:26:20,624[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dche.home=/home/user/eclipse-che/
2018-01-29 20:26:20,628[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dche.logs.dir=/logs
2018-01-29 20:26:20,630[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dche.logs.level=INFO
2018-01-29 20:26:20,630[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Djuli-logback.configurationFile=file:/home/user/eclipse-che//tomcat/conf/tomcat-logger.xml
2018-01-29 20:26:20,630[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Djdk.tls.ephemeralDHKeySize=2048
2018-01-29 20:26:20,631[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
2018-01-29 20:26:20,631[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dcom.sun.management.jmxremote
2018-01-29 20:26:20,632[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dcom.sun.management.jmxremote.ssl=false
2018-01-29 20:26:20,633[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dcom.sun.management.jmxremote.authenticate=false
2018-01-29 20:26:20,634[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dche.local.conf.dir=/conf
2018-01-29 20:26:20,634[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dcatalina.base=/home/user/eclipse-che//tomcat
2018-01-29 20:26:20,634[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dcatalina.home=/home/user/eclipse-che//tomcat
2018-01-29 20:26:20,635[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Djava.io.tmpdir=/home/user/eclipse-che//tomcat/temp
2018-01-29 20:26:20,841[main]             [INFO ] [o.a.c.http11.Http11NioProtocol 525]  - Initializing ProtocolHandler ["http-nio-6868"]
2018-01-29 20:26:20,880[main]             [INFO ] [o.a.t.util.net.NioSelectorPool 67]   - Using a shared selector for servlet write/read
2018-01-29 20:26:20,894[main]             [INFO ] [o.a.catalina.startup.Catalina 617]   - Initialization processed in 1616 ms
2018-01-29 20:26:20,931[main]             [INFO ] [c.m.JmxRemoteLifecycleListener 336]  - The JMX Remote Listener has configured the registry on port 32001 and the server on port 32101 for the Platform server
2018-01-29 20:26:20,932[main]             [INFO ] [o.a.c.core.StandardService 416]      - Starting service Catalina
2018-01-29 20:26:20,932[main]             [INFO ] [o.a.c.core.StandardEngine 259]       - Starting Servlet Engine: Apache Tomcat/8.5.11
2018-01-29 20:26:22,152[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 923]        - Deploying web application archive /home/user/eclipse-che/tomcat/webapps/dashboard.war
2018-01-29 20:26:22,663[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 987]        - Deployment of web application archive /home/user/eclipse-che/tomcat/webapps/dashboard.war has finished in 510 ms
2018-01-29 20:26:22,667[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 923]        - Deploying web application archive /home/user/eclipse-che/tomcat/webapps/docs.war
2018-01-29 20:26:23,528[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 987]        - Deployment of web application archive /home/user/eclipse-che/tomcat/webapps/docs.war has finished in 861 ms
2018-01-29 20:26:23,534[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 923]        - Deploying web application archive /home/user/eclipse-che/tomcat/webapps/ROOT.war
2018-01-29 20:26:24,305[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 987]        - Deployment of web application archive /home/user/eclipse-che/tomcat/webapps/ROOT.war has finished in 771 ms
2018-01-29 20:26:24,310[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 923]        - Deploying web application archive /home/user/eclipse-che/tomcat/webapps/wsmaster.war
2018-01-29 20:26:25,884[ost-startStop-1]  [INFO ] [.e.c.c.d.JNDIDataSourceFactory 59]   - This=org.eclipse.che.core.db.h2.H2SQLJndiDataSourceFactory@5b4c08bf obj=ResourceRef[className=javax.sql.DataSource,factoryClassLocation=null,factoryClassName=org.apache.naming.factory.ResourceFactory,{type=scope,content=Shareable},{type=auth,content=Container},{type=singleton,content=true},{type=factory,content=org.eclipse.che.core.db.h2.H2SQLJndiDataSourceFactory}] name=che Context=org.apache.naming.NamingContext@1179a32 environment={}
2018-01-29 20:26:30,253[ost-startStop-1]  [WARN ] [i.f.kubernetes.client.Config 338]    - Error reading service account token from: [/var/run/secrets/kubernetes.io/serviceaccount/token]. Ignoring.
2018-01-29 20:26:30,579[ost-startStop-1]  [INFO ] [o.e.c.p.o.c.OpenShiftConnector 306]  - openshiftWorkspaceEnvironmentProvider = class org.eclipse.che.plugin.openshift.client.OpenshiftWorkspaceEnvironmentProvider
2018-01-29 20:26:31,521[ost-startStop-1]  [INFO ] [o.f.c.i.d.DbSupportFactory 44]       - Database: jdbc:h2:che (H2 1.4)
2018-01-29 20:26:31,620[ost-startStop-1]  [INFO ] [o.f.c.i.util.VersionPrinter 44]      - Flyway 4.0.3 by Boxfuse
2018-01-29 20:26:31,626[ost-startStop-1]  [INFO ] [o.f.c.i.d.DbSupportFactory 44]       - Database: jdbc:h2:che (H2 1.4)
2018-01-29 20:26:31,668[ost-startStop-1]  [INFO ] [i.f.CustomSqlMigrationResolver 157]  - Searching for sql scripts in locations [classpath:che-schema]
2018-01-29 20:26:31,704[ost-startStop-1]  [INFO ] [o.f.c.i.command.DbValidate 44]       - Successfully validated 11 migrations (execution time 00:00.046s)
2018-01-29 20:26:31,738[ost-startStop-1]  [INFO ] [o.f.c.i.m.MetaDataTableImpl 44]      - Creating Metadata table: "PUBLIC"."schema_version"
2018-01-29 20:26:31,777[ost-startStop-1]  [INFO ] [o.f.c.i.command.DbMigrate 44]        - Current version of schema "PUBLIC": << Empty Schema >>
2018-01-29 20:26:31,787[ost-startStop-1]  [INFO ] [o.f.c.i.command.DbMigrate 44]        - Migrating schema "PUBLIC" to version 5.0.0.8.1 - 1__init.sql
2018-01-29 20:26:32,169[ost-startStop-1]  [INFO ] [o.f.c.i.command.DbMigrate 44]        - Migrating schema "PUBLIC" to version 5.0.0.9.1 - 1__add_index_on_workspace_temporary.sql
2018-01-29 20:26:32,191[ost-startStop-1]  [INFO ] [o.f.c.i.command.DbMigrate 44]        - Migrating schema "PUBLIC" to version 5.0.0.9.2 - 2__update_local_links_in_environments.sql
2018-01-29 20:26:32,217[ost-startStop-1]  [INFO ] [o.f.c.i.command.DbMigrate 44]        - Migrating schema "PUBLIC" to version 5.2.0.1 - 1__increase_project_attributes_values_length.sql
2018-01-29 20:26:32,260[ost-startStop-1]  [INFO ] [o.f.c.i.command.DbMigrate 44]        - Migrating schema "PUBLIC" to version 5.4.0.1 - 1__drop_user_to_account_relation.sql
2018-01-29 20:26:32,344[ost-startStop-1]  [INFO ] [o.f.c.i.command.DbMigrate 44]        - Migrating schema "PUBLIC" to version 5.4.0.2 - 2__create_missed_account_indexes.sql
2018-01-29 20:26:32,372[ost-startStop-1]  [INFO ] [o.f.c.i.command.DbMigrate 44]        - Migrating schema "PUBLIC" to version 5.6.0.1 - 1__add_exec_agent_where_terminal_agent_is_present.sql
2018-01-29 20:26:32,387[ost-startStop-1]  [INFO ] [o.f.c.i.command.DbMigrate 44]        - Migrating schema "PUBLIC" to version 5.7.0.1 - 1__add_factory.sql
2018-01-29 20:26:32,478[ost-startStop-1]  [INFO ] [o.f.c.i.command.DbMigrate 44]        - Migrating schema "PUBLIC" to version 5.7.0.2 - 2__remove_match_policy.sql
2018-01-29 20:26:32,533[ost-startStop-1]  [INFO ] [o.f.c.i.command.DbMigrate 44]        - Migrating schema "PUBLIC" to version 5.8.0.1 - 1__add_foreigh_key_indexes.sql
2018-01-29 20:26:32,573[ost-startStop-1]  [INFO ] [o.f.c.i.command.DbMigrate 44]        - Migrating schema "PUBLIC" to version 5.11.0.1 - 1__optimize_user_search.sql
2018-01-29 20:26:32,593[ost-startStop-1]  [INFO ] [o.f.c.i.command.DbMigrate 44]        - Successfully applied 11 migrations to schema "PUBLIC" (execution time 00:00.860s).
2018-01-29 20:26:35,210[ost-startStop-1]  [INFO ] [o.e.c.a.m.s.r.RecipeLoader 74]       - Recipes initialization finished
2018-01-29 20:26:35,235[ost-startStop-1]  [INFO ] [ogDirSetterEnvVariableProvider 38]   - WsAgentLogDirProvider = org.eclipse.che.plugin.docker.machine.WsAgentLogDirProvider@54553196
2018-01-29 20:26:35,238[ost-startStop-1]  [INFO ] [ogDirSetterEnvVariableProvider 37]   - ExecAgentLogDirProvider = org.eclipse.che.plugin.docker.machine.ExecAgentLogDirProvider@7ac776e4
2018-01-29 20:26:35,378[ost-startStop-1]  [INFO ] [e.c.a.w.s.i.ServerIdleDetector 58]   - Idling che server scheduled [timeout=1800] seconds]
2018-01-29 20:26:35,500[ted-scheduler-2]  [INFO ] [ockerAbandonedResourcesCleaner 126]  - List containers registered in the api: []
2018-01-29 20:26:36,422[ost-startStop-1]  [WARN ] [o.e.c.a.w.s.stack.StackLoader 134]   - No configured image found for stack che-in-che
2018-01-29 20:26:36,440[ost-startStop-1]  [INFO ] [o.e.c.a.w.s.stack.StackLoader 103]   - Stacks initialization finished
2018-01-29 20:26:37,116[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 987]        - Deployment of web application archive /home/user/eclipse-che/tomcat/webapps/wsmaster.war has finished in 12,805 ms
2018-01-29 20:26:37,120[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 923]        - Deploying web application archive /home/user/eclipse-che/tomcat/webapps/swagger.war
2018-01-29 20:26:37,382[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 987]        - Deployment of web application archive /home/user/eclipse-che/tomcat/webapps/swagger.war has finished in 262 ms
2018-01-29 20:26:37,394[main]             [INFO ] [o.a.c.http11.Http11NioProtocol 570]  - Starting ProtocolHandler [http-nio-6868]
2018-01-29 20:26:37,410[main]             [INFO ] [o.a.catalina.startup.Catalina 668]   - Server startup in 16515 ms
2018-01-29 20:46:26,450[nio-6868-exec-3]  [INFO ] [o.a.c.http11.Http11Processor 710]    - Error parsing HTTP request header
 Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: Invalid character found in method name. HTTP method names must be tokens
        at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:421)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:667)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:798)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1434)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)
2018-01-29 20:46:26,451[nio-6868-exec-4]  [INFO ] [o.a.c.http11.Http11Processor 710]    - Error parsing HTTP request header
 Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: Invalid character found in method name. HTTP method names must be tokens
        at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:421)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:667)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:798)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1434)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)
2018-01-29 20:46:43,928[nio-6868-exec-2]  [INFO ] [o.e.c.a.f.s.jpa.JpaFactoryDao 111]   - FactoryDao#getByAttributes #maxItems: 15 #skipCount: 0, #attributes: [{first=creator.userId, second=che}]
2018-01-29 20:47:12,014[io-6868-exec-10]  [INFO ] [o.e.c.a.w.s.WorkspaceManager 807]    - Workspace 'che/gCyborg' with id 'workspacebsetwzqtonmg1guy' created by user 'che'
2018-01-29 20:47:32,019[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 20:47:32,026[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 20:47:32,027[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 20:47:32,033[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 20:47:32,036[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 20:47:32,224[aceSharedPool-0]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 676]   - Launching 'org.eclipse.che.exec' agent at workspace workspacebsetwzqtonmg1guy
2018-01-29 20:47:34,783[aceSharedPool-0]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 676]   - Launching 'org.eclipse.che.terminal' agent at workspace workspacebsetwzqtonmg1guy
2018-01-29 20:51:05,091[nio-6868-exec-8]  [INFO ] [o.e.c.a.f.s.jpa.JpaFactoryDao 111]   - FactoryDao#getByAttributes #maxItems: 15 #skipCount: 0, #attributes: [{first=creator.userId, second=che}]
2018-01-29 20:51:51,952[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 20:51:51,960[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 20:51:51,960[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 20:51:51,960[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 20:51:51,961[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 20:51:52,144[aceSharedPool-0]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 676]   - Launching 'org.eclipse.che.exec' agent at workspace workspacebsetwzqtonmg1guy
2018-01-29 20:51:54,516[aceSharedPool-0]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 676]   - Launching 'org.eclipse.che.terminal' agent at workspace workspacebsetwzqtonmg1guy
2018-01-29 20:51:56,750[aceSharedPool-0]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 676]   - Launching 'org.eclipse.che.ws-agent' agent at workspace workspacebsetwzqtonmg1guy
2018-01-29 20:54:57,094[aceSharedPool-0]  [ERROR] [o.e.c.a.agent.WsAgentLauncher 147]   - Fail pinging ws agent with http://172.31.1.176:32780/api/ url in workspacebsetwzqtonmg1guy workspace in machine2t9v7livavokgw3t machine on localhost node.
2018-01-29 20:54:57,921[aceSharedPool-0]  [ERROR] [o.e.c.a.w.s.WorkspaceManager 683]    - Timeout. The Che server is unable to ping your workspace. This implies a network configuration issue, workspace boot failure, or an unusually slow workspace boot.
org.eclipse.che.api.core.ServerException: Timeout. The Che server is unable to ping your workspace. This implies a network configuration issue, workspace boot failure, or an unusually slow workspace boot.
        at org.eclipse.che.api.agent.WsAgentLauncher.launch(WsAgentLauncher.java:153)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.launchAgents(WorkspaceRuntimes.java:681)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$MachineAgentsLauncher.started(WorkspaceRuntimes.java:1048)
        at org.eclipse.che.api.environment.server.CheEnvironmentEngine.startEnvironmentQueue(CheEnvironmentEngine.java:800)
        at org.eclipse.che.api.environment.server.CheEnvironmentEngine.start(CheEnvironmentEngine.java:263)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.startEnvironmentAndPublishEvents(WorkspaceRuntimes.java:694)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.access$100(WorkspaceRuntimes.java:103)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartTask.call(WorkspaceRuntimes.java:986)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartTask.call(WorkspaceRuntimes.java:948)
        at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalCallable.call(CopyThreadLocalCallable.java:31)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
2018-01-29 20:56:10,572[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 20:56:10,573[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 20:56:10,574[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 20:56:10,574[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 20:56:10,575[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 20:56:10,643[aceSharedPool-1]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 676]   - Launching 'org.eclipse.che.exec' agent at workspace workspacebsetwzqtonmg1guy
2018-01-29 20:56:12,849[aceSharedPool-1]  [ERROR] [.a.a.s.l.AbstractAgentLauncher 165]  - An error occurs while starting 'Exec' agent in 'workspacebsetwzqtonmg1guy' workspace in 'machinehkey58e3g59rqi2u' machine on 'localhost' node. Detailed log:
[STDOUT] 2018/01/29 20:56:11 Exec-agent configuration
[STDOUT] 2018/01/29 20:56:11   Server
[STDOUT] 2018/01/29 20:56:11     - Address: :4412
[STDOUT] 2018/01/29 20:56:11     - Base path: ''
[STDOUT] 2018/01/29 20:56:11   Process executor
[STDOUT] 2018/01/29 20:56:11     - Logs dir: /root/che/exec-agent/logs
[STDOUT] 2018/01/29 20:56:11
[STDOUT] 2018/01/29 20:56:11 ⇩ Registered HTTPRoutes:
[STDOUT]
[STDOUT] 2018/01/29 20:56:11 Process Routes:
[STDOUT] 2018/01/29 20:56:11 ✓ Start Process ........................... POST   /process
[STDOUT] 2018/01/29 20:56:11 ✓ Get Process ............................. GET    /process/:pid
[STDOUT] 2018/01/29 20:56:11 ✓ Kill Process ............................ DELETE /process/:pid
[STDOUT] 2018/01/29 20:56:11 ✓ Get Process Logs ........................ GET    /process/:pid/logs
[STDOUT] 2018/01/29 20:56:11 ✓ Get Processes ........................... GET    /process
[STDOUT] 2018/01/29 20:56:11
[STDOUT] 2018/01/29 20:56:11 Exec-Agent WebSocket routes:
[STDOUT] 2018/01/29 20:56:11 ✓ Connect to Exec-Agent(websocket) ........ GET    /connect
[STDOUT] 2018/01/29 20:56:11
[STDOUT] 2018/01/29 20:56:11 ⇩ Registered RPCRoutes:
[STDOUT]
[STDOUT] 2018/01/29 20:56:11 Process Routes:
[STDOUT] 2018/01/29 20:56:11 ✓ process.start
[STDOUT] 2018/01/29 20:56:11 ✓ process.kill
[STDOUT] 2018/01/29 20:56:11 ✓ process.subscribe
[STDOUT] 2018/01/29 20:56:11 ✓ process.unsubscribe
[STDOUT] 2018/01/29 20:56:11 ✓ process.updateSubscriber
[STDOUT] 2018/01/29 20:56:11 ✓ process.getLogs
[STDOUT] 2018/01/29 20:56:11 ✓ process.getProcess
[STDOUT] 2018/01/29 20:56:11 ✓ process.getProcesses
2018-01-29 20:56:12,934[aceSharedPool-1]  [ERROR] [o.e.c.a.w.s.WorkspaceManager 683]    - Error occurs while initializing command [/bin/sh, -c, trap '[ -z "$(jobs -p)" ] || kill $(jobs -p); [ -e /tmp/docker-exec-16.pid ] && rm /tmp/docker-exec-16.pid' EXIT; echo $$>/tmp/docker-exec-16.pid; command -v pidof >/dev/null 2>&1 && {
    pidof che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
} || {
    ps -fC che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
}] in docker container 8f877d704481bc68bd75b7d5f31140109f05437336e4e79382fa04695789cc7b: Error response from docker API, status: 409, message: Container 8f877d704481bc68bd75b7d5f31140109f05437336e4e79382fa04695789cc7b is not running

org.eclipse.che.api.core.ServerException: Error occurs while initializing command [/bin/sh, -c, trap '[ -z "$(jobs -p)" ] || kill $(jobs -p); [ -e /tmp/docker-exec-16.pid ] && rm /tmp/docker-exec-16.pid' EXIT; echo $$>/tmp/docker-exec-16.pid; command -v pidof >/dev/null 2>&1 && {
    pidof che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
} || {
    ps -fC che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
}] in docker container 8f877d704481bc68bd75b7d5f31140109f05437336e4e79382fa04695789cc7b: Error response from docker API, status: 409, message: Container 8f877d704481bc68bd75b7d5f31140109f05437336e4e79382fa04695789cc7b is not running

        at org.eclipse.che.api.agent.server.launcher.AbstractAgentLauncher.launch(AbstractAgentLauncher.java:115)
        at org.eclipse.che.api.agent.ExecAgentLauncher.launch(ExecAgentLauncher.java:46)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.launchAgents(WorkspaceRuntimes.java:681)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$MachineAgentsLauncher.started(WorkspaceRuntimes.java:1048)
        at org.eclipse.che.api.environment.server.CheEnvironmentEngine.startEnvironmentQueue(CheEnvironmentEngine.java:800)
        at org.eclipse.che.api.environment.server.CheEnvironmentEngine.start(CheEnvironmentEngine.java:263)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.startEnvironmentAndPublishEvents(WorkspaceRuntimes.java:694)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.access$100(WorkspaceRuntimes.java:103)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartTask.call(WorkspaceRuntimes.java:986)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartTask.call(WorkspaceRuntimes.java:948)
        at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalCallable.call(CopyThreadLocalCallable.java:31)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
2018-01-29 20:56:23,514[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 20:56:23,514[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 20:56:23,515[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 20:56:23,515[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 20:56:23,515[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 20:56:23,593[aceSharedPool-0]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 676]   - Launching 'org.eclipse.che.exec' agent at workspace workspacebsetwzqtonmg1guy
2018-01-29 20:56:25,835[aceSharedPool-0]  [ERROR] [.a.a.s.l.AbstractAgentLauncher 173]  - An error occurs while starting 'Exec' agent in 'workspacebsetwzqtonmg1guy' workspace in 'machinepzzxoc14b15kjtrk' machine on 'localhost' node. The agent didn't produce any logs.
2018-01-29 20:56:25,899[aceSharedPool-0]  [ERROR] [o.e.c.a.w.s.WorkspaceManager 683]    - Error occurs while initializing command [/bin/sh, -c, trap '[ -z "$(jobs -p)" ] || kill $(jobs -p); [ -e /tmp/docker-exec-19.pid ] && rm /tmp/docker-exec-19.pid' EXIT; echo $$>/tmp/docker-exec-19.pid; command -v pidof >/dev/null 2>&1 && {
    pidof che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
} || {
    ps -fC che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
}] in docker container 8c8c45b94296399a680afccb513301f7eaa2b84322955916e45439bd440a058e: Error response from docker API, status: 409, message: Container 8c8c45b94296399a680afccb513301f7eaa2b84322955916e45439bd440a058e is not running

org.eclipse.che.api.core.ServerException: Error occurs while initializing command [/bin/sh, -c, trap '[ -z "$(jobs -p)" ] || kill $(jobs -p); [ -e /tmp/docker-exec-19.pid ] && rm /tmp/docker-exec-19.pid' EXIT; echo $$>/tmp/docker-exec-19.pid; command -v pidof >/dev/null 2>&1 && {
    pidof che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
} || {
    ps -fC che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
}] in docker container 8c8c45b94296399a680afccb513301f7eaa2b84322955916e45439bd440a058e: Error response from docker API, status: 409, message: Container 8c8c45b94296399a680afccb513301f7eaa2b84322955916e45439bd440a058e is not running

        at org.eclipse.che.api.agent.server.launcher.AbstractAgentLauncher.launch(AbstractAgentLauncher.java:115)
        at org.eclipse.che.api.agent.ExecAgentLauncher.launch(ExecAgentLauncher.java:46)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.launchAgents(WorkspaceRuntimes.java:681)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$MachineAgentsLauncher.started(WorkspaceRuntimes.java:1048)
        at org.eclipse.che.api.environment.server.CheEnvironmentEngine.startEnvironmentQueue(CheEnvironmentEngine.java:800)
        at org.eclipse.che.api.environment.server.CheEnvironmentEngine.start(CheEnvironmentEngine.java:263)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.startEnvironmentAndPublishEvents(WorkspaceRuntimes.java:694)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.access$100(WorkspaceRuntimes.java:103)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartTask.call(WorkspaceRuntimes.java:986)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartTask.call(WorkspaceRuntimes.java:948)
        at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalCallable.call(CopyThreadLocalCallable.java:31)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
2018-01-29 20:56:35,968[ool-12-thread-1]  [WARN ] [i.f.kubernetes.client.Config 338]    - Error reading service account token from: [/var/run/secrets/kubernetes.io/serviceaccount/token]. Ignoring.
2018-01-29 20:56:36,220[ool-12-thread-1]  [ERROR] [o.e.c.a.c.n.EventService 109]        - Operation: [get]  for kind: [DeploymentConfig]  with name: [che]  in namespace: [eclipse-che]  failed.
io.fabric8.kubernetes.client.KubernetesClientException: Operation: [get]  for kind: [DeploymentConfig]  with name: [che]  in namespace: [eclipse-che]  failed.
        at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:62)
        at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:71)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:217)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:181)
        at org.eclipse.che.plugin.openshift.client.OpenShiftConnector.idleCheServer(OpenShiftConnector.java:391)
        at org.eclipse.che.plugin.openshift.client.OpenShiftConnector.access$000(OpenShiftConnector.java:179)
        at org.eclipse.che.plugin.openshift.client.OpenShiftConnector$1.onEvent(OpenShiftConnector.java:303)
        at org.eclipse.che.plugin.openshift.client.OpenShiftConnector$1.onEvent(OpenShiftConnector.java:299)
        at org.eclipse.che.api.core.notification.EventService.publish(EventService.java:107)
        at org.eclipse.che.api.workspace.server.idle.ServerIdleDetector.run(ServerIdleDetector.java:93)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.UnknownHostException: kubernetes.default.svc: Name does not resolve
        at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
        at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
        at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
        at java.net.InetAddress.getAllByName0(InetAddress.java:1276)
        at java.net.InetAddress.getAllByName(InetAddress.java:1192)
        at java.net.InetAddress.getAllByName(InetAddress.java:1126)
        at okhttp3.Dns$1.lookup(Dns.java:39)
        at okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:170)
        at okhttp3.internal.connection.RouteSelector.nextProxy(RouteSelector.java:136)
        at okhttp3.internal.connection.RouteSelector.next(RouteSelector.java:81)
        at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:171)
        at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:121)
        at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:100)
        at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
        at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
        at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
        at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
        at io.fabric8.openshift.client.internal.OpenShiftOAuthInterceptor.intercept(OpenShiftOAuthInterceptor.java:64)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
        at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:185)
        at okhttp3.RealCall.execute(RealCall.java:69)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:377)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:343)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:312)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:295)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleGet(BaseOperation.java:753)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:206)
        ... 14 common frames omitted
2018-01-29 20:56:45,314[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 20:56:45,315[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 20:56:45,316[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 20:56:45,317[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 20:56:45,318[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 20:56:45,418[aceSharedPool-1]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 676]   - Launching 'org.eclipse.che.exec' agent at workspace workspacebsetwzqtonmg1guy
2018-01-29 20:56:47,713[aceSharedPool-1]  [ERROR] [.a.a.s.l.AbstractAgentLauncher 173]  - An error occurs while starting 'Exec' agent in 'workspacebsetwzqtonmg1guy' workspace in 'machine0g7g1140bvhjnrxh' machine on 'localhost' node. The agent didn't produce any logs.
2018-01-29 20:56:47,776[aceSharedPool-1]  [ERROR] [o.e.c.a.w.s.WorkspaceManager 683]    - Error occurs while initializing command [/bin/sh, -c, trap '[ -z "$(jobs -p)" ] || kill $(jobs -p); [ -e /tmp/docker-exec-22.pid ] && rm /tmp/docker-exec-22.pid' EXIT; echo $$>/tmp/docker-exec-22.pid; command -v pidof >/dev/null 2>&1 && {
    pidof che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
} || {
    ps -fC che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
}] in docker container 0cd04ada71eff1004a38c77c41db63094cc95bef51cdfc2ad206dc54966592a0: Error response from docker API, status: 409, message: Container 0cd04ada71eff1004a38c77c41db63094cc95bef51cdfc2ad206dc54966592a0 is not running

org.eclipse.che.api.core.ServerException: Error occurs while initializing command [/bin/sh, -c, trap '[ -z "$(jobs -p)" ] || kill $(jobs -p); [ -e /tmp/docker-exec-22.pid ] && rm /tmp/docker-exec-22.pid' EXIT; echo $$>/tmp/docker-exec-22.pid; command -v pidof >/dev/null 2>&1 && {
    pidof che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
} || {
    ps -fC che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
}] in docker container 0cd04ada71eff1004a38c77c41db63094cc95bef51cdfc2ad206dc54966592a0: Error response from docker API, status: 409, message: Container 0cd04ada71eff1004a38c77c41db63094cc95bef51cdfc2ad206dc54966592a0 is not running

        at org.eclipse.che.api.agent.server.launcher.AbstractAgentLauncher.launch(AbstractAgentLauncher.java:115)
        at org.eclipse.che.api.agent.ExecAgentLauncher.launch(ExecAgentLauncher.java:46)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.launchAgents(WorkspaceRuntimes.java:681)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$MachineAgentsLauncher.started(WorkspaceRuntimes.java:1048)
        at org.eclipse.che.api.environment.server.CheEnvironmentEngine.startEnvironmentQueue(CheEnvironmentEngine.java:800)
        at org.eclipse.che.api.environment.server.CheEnvironmentEngine.start(CheEnvironmentEngine.java:263)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.startEnvironmentAndPublishEvents(WorkspaceRuntimes.java:694)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.access$100(WorkspaceRuntimes.java:103)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartTask.call(WorkspaceRuntimes.java:986)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartTask.call(WorkspaceRuntimes.java:948)
        at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalCallable.call(CopyThreadLocalCallable.java:31)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
2018-01-29 21:00:33,434[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 21:00:33,435[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 21:00:33,435[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 21:00:33,437[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 21:00:33,437[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 21:00:33,542[aceSharedPool-0]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 676]   - Launching 'org.eclipse.che.exec' agent at workspace workspacebsetwzqtonmg1guy
2018-01-29 21:00:35,881[aceSharedPool-0]  [ERROR] [.a.a.s.l.AbstractAgentLauncher 165]  - An error occurs while starting 'Exec' agent in 'workspacebsetwzqtonmg1guy' workspace in 'machineqek4xah9afrvkgpl' machine on 'localhost' node. Detailed log:
[STDOUT] 2018/01/29 21:00:34 Exec-agent configuration
[STDOUT] 2018/01/29 21:00:34   Server
[STDOUT] 2018/01/29 21:00:34     - Address: :4412
[STDOUT] 2018/01/29 21:00:34     - Base path: ''
[STDOUT] 2018/01/29 21:00:34   Process executor
[STDOUT] 2018/01/29 21:00:34     - Logs dir: /root/che/exec-agent/logs
[STDOUT] 2018/01/29 21:00:34
[STDOUT] 2018/01/29 21:00:34 ⇩ Registered HTTPRoutes:
[STDOUT]
[STDOUT] 2018/01/29 21:00:34 Process Routes:
[STDOUT] 2018/01/29 21:00:34 ✓ Start Process ........................... POST   /process
[STDOUT] 2018/01/29 21:00:34 ✓ Get Process ............................. GET    /process/:pid
[STDOUT] 2018/01/29 21:00:34 ✓ Kill Process ............................ DELETE /process/:pid
[STDOUT] 2018/01/29 21:00:34 ✓ Get Process Logs ........................ GET    /process/:pid/logs
[STDOUT] 2018/01/29 21:00:34 ✓ Get Processes ........................... GET    /process
[STDOUT] 2018/01/29 21:00:34
[STDOUT] 2018/01/29 21:00:34 Exec-Agent WebSocket routes:
[STDOUT] 2018/01/29 21:00:34 ✓ Connect to Exec-Agent(websocket) ........ GET    /connect
[STDOUT] 2018/01/29 21:00:34
[STDOUT] 2018/01/29 21:00:34 ⇩ Registered RPCRoutes:
[STDOUT]
[STDOUT] 2018/01/29 21:00:34 Process Routes:
[STDOUT] 2018/01/29 21:00:34 ✓ process.start
[STDOUT] 2018/01/29 21:00:34 ✓ process.kill
[STDOUT] 2018/01/29 21:00:34 ✓ process.subscribe
[STDOUT] 2018/01/29 21:00:34 ✓ process.unsubscribe
[STDOUT] 2018/01/29 21:00:34 ✓ process.updateSubscriber
[STDOUT] 2018/01/29 21:00:34 ✓ process.getLogs
[STDOUT] 2018/01/29 21:00:34 ✓ process.getProcess
[STDOUT] 2018/01/29 21:00:34 ✓ process.getProcesses
2018-01-29 21:00:35,965[aceSharedPool-0]  [ERROR] [o.e.c.a.w.s.WorkspaceManager 683]    - Error occurs while initializing command [/bin/sh, -c, trap '[ -z "$(jobs -p)" ] || kill $(jobs -p); [ -e /tmp/docker-exec-25.pid ] && rm /tmp/docker-exec-25.pid' EXIT; echo $$>/tmp/docker-exec-25.pid; command -v pidof >/dev/null 2>&1 && {
    pidof che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
} || {
    ps -fC che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
}] in docker container 9e59ff20ef9eaedbdfff60948328bc79ee17fb629e66e79abd3cba4471cf4f60: Error response from docker API, status: 409, message: Container 9e59ff20ef9eaedbdfff60948328bc79ee17fb629e66e79abd3cba4471cf4f60 is not running

org.eclipse.che.api.core.ServerException: Error occurs while initializing command [/bin/sh, -c, trap '[ -z "$(jobs -p)" ] || kill $(jobs -p); [ -e /tmp/docker-exec-25.pid ] && rm /tmp/docker-exec-25.pid' EXIT; echo $$>/tmp/docker-exec-25.pid; command -v pidof >/dev/null 2>&1 && {
    pidof che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
} || {
    ps -fC che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
}] in docker container 9e59ff20ef9eaedbdfff60948328bc79ee17fb629e66e79abd3cba4471cf4f60: Error response from docker API, status: 409, message: Container 9e59ff20ef9eaedbdfff60948328bc79ee17fb629e66e79abd3cba4471cf4f60 is not running

        at org.eclipse.che.api.agent.server.launcher.AbstractAgentLauncher.launch(AbstractAgentLauncher.java:115)
        at org.eclipse.che.api.agent.ExecAgentLauncher.launch(ExecAgentLauncher.java:46)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.launchAgents(WorkspaceRuntimes.java:681)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$MachineAgentsLauncher.started(WorkspaceRuntimes.java:1048)
        at org.eclipse.che.api.environment.server.CheEnvironmentEngine.startEnvironmentQueue(CheEnvironmentEngine.java:800)
        at org.eclipse.che.api.environment.server.CheEnvironmentEngine.start(CheEnvironmentEngine.java:263)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.startEnvironmentAndPublishEvents(WorkspaceRuntimes.java:694)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.access$100(WorkspaceRuntimes.java:103)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartTask.call(WorkspaceRuntimes.java:986)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartTask.call(WorkspaceRuntimes.java:948)
        at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalCallable.call(CopyThreadLocalCallable.java:31)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
2018-01-29 21:00:59,629[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 21:00:59,630[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 21:00:59,630[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 21:00:59,631[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 21:00:59,631[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 21:00:59,727[aceSharedPool-1]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 676]   - Launching 'org.eclipse.che.exec' agent at workspace workspacebsetwzqtonmg1guy
2018-01-29 21:01:02,023[aceSharedPool-1]  [ERROR] [.a.a.s.l.AbstractAgentLauncher 173]  - An error occurs while starting 'Exec' agent in 'workspacebsetwzqtonmg1guy' workspace in 'machinerncltl2vy8vh2kzq' machine on 'localhost' node. The agent didn't produce any logs.
2018-01-29 21:01:02,080[aceSharedPool-1]  [ERROR] [o.e.c.a.w.s.WorkspaceManager 683]    - Error occurs while initializing command [/bin/sh, -c, trap '[ -z "$(jobs -p)" ] || kill $(jobs -p); [ -e /tmp/docker-exec-28.pid ] && rm /tmp/docker-exec-28.pid' EXIT; echo $$>/tmp/docker-exec-28.pid; command -v pidof >/dev/null 2>&1 && {
    pidof che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
} || {
    ps -fC che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
}] in docker container 9a1f143521a958edc4e11664503b771dfea978324bed3b69fab456fdd5bc2850: Error response from docker API, status: 409, message: Container 9a1f143521a958edc4e11664503b771dfea978324bed3b69fab456fdd5bc2850 is not running

org.eclipse.che.api.core.ServerException: Error occurs while initializing command [/bin/sh, -c, trap '[ -z "$(jobs -p)" ] || kill $(jobs -p); [ -e /tmp/docker-exec-28.pid ] && rm /tmp/docker-exec-28.pid' EXIT; echo $$>/tmp/docker-exec-28.pid; command -v pidof >/dev/null 2>&1 && {
    pidof che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
} || {
    ps -fC che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
}] in docker container 9a1f143521a958edc4e11664503b771dfea978324bed3b69fab456fdd5bc2850: Error response from docker API, status: 409, message: Container 9a1f143521a958edc4e11664503b771dfea978324bed3b69fab456fdd5bc2850 is not running

        at org.eclipse.che.api.agent.server.launcher.AbstractAgentLauncher.launch(AbstractAgentLauncher.java:115)
        at org.eclipse.che.api.agent.ExecAgentLauncher.launch(ExecAgentLauncher.java:46)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.launchAgents(WorkspaceRuntimes.java:681)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$MachineAgentsLauncher.started(WorkspaceRuntimes.java:1048)
        at org.eclipse.che.api.environment.server.CheEnvironmentEngine.startEnvironmentQueue(CheEnvironmentEngine.java:800)
        at org.eclipse.che.api.environment.server.CheEnvironmentEngine.start(CheEnvironmentEngine.java:263)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.startEnvironmentAndPublishEvents(WorkspaceRuntimes.java:694)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.access$100(WorkspaceRuntimes.java:103)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartTask.call(WorkspaceRuntimes.java:986)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartTask.call(WorkspaceRuntimes.java:948)
        at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalCallable.call(CopyThreadLocalCallable.java:31)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
2018-01-29 21:01:07,606[io-6868-exec-10]  [INFO ] [o.e.c.a.f.s.jpa.JpaFactoryDao 111]   - FactoryDao#getByAttributes #maxItems: 15 #skipCount: 0, #attributes: [{first=creator.userId, second=che}]
2018-01-29 21:01:12,572[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 21:01:12,574[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 21:01:12,575[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 21:01:12,576[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 21:01:12,578[aceSharedPool-0]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 21:01:12,682[aceSharedPool-0]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 676]   - Launching 'org.eclipse.che.exec' agent at workspace workspacebsetwzqtonmg1guy
2018-01-29 21:01:14,927[aceSharedPool-0]  [ERROR] [.a.a.s.l.AbstractAgentLauncher 173]  - An error occurs while starting 'Exec' agent in 'workspacebsetwzqtonmg1guy' workspace in 'machinebxaboz4wn78o5n6r' machine on 'localhost' node. The agent didn't produce any logs.
2018-01-29 21:01:14,990[aceSharedPool-0]  [ERROR] [o.e.c.a.w.s.WorkspaceManager 683]    - Error occurs while initializing command [/bin/sh, -c, trap '[ -z "$(jobs -p)" ] || kill $(jobs -p); [ -e /tmp/docker-exec-31.pid ] && rm /tmp/docker-exec-31.pid' EXIT; echo $$>/tmp/docker-exec-31.pid; command -v pidof >/dev/null 2>&1 && {
    pidof che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
} || {
    ps -fC che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
}] in docker container 8b9f4344a0e3657b8c3efa2b61d31669d4162c9fe8a0390de65c564085f5159d: Error response from docker API, status: 409, message: Container 8b9f4344a0e3657b8c3efa2b61d31669d4162c9fe8a0390de65c564085f5159d is not running

org.eclipse.che.api.core.ServerException: Error occurs while initializing command [/bin/sh, -c, trap '[ -z "$(jobs -p)" ] || kill $(jobs -p); [ -e /tmp/docker-exec-31.pid ] && rm /tmp/docker-exec-31.pid' EXIT; echo $$>/tmp/docker-exec-31.pid; command -v pidof >/dev/null 2>&1 && {
    pidof che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
} || {
    ps -fC che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
}] in docker container 8b9f4344a0e3657b8c3efa2b61d31669d4162c9fe8a0390de65c564085f5159d: Error response from docker API, status: 409, message: Container 8b9f4344a0e3657b8c3efa2b61d31669d4162c9fe8a0390de65c564085f5159d is not running

        at org.eclipse.che.api.agent.server.launcher.AbstractAgentLauncher.launch(AbstractAgentLauncher.java:115)
        at org.eclipse.che.api.agent.ExecAgentLauncher.launch(ExecAgentLauncher.java:46)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.launchAgents(WorkspaceRuntimes.java:681)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$MachineAgentsLauncher.started(WorkspaceRuntimes.java:1048)
        at org.eclipse.che.api.environment.server.CheEnvironmentEngine.startEnvironmentQueue(CheEnvironmentEngine.java:800)
        at org.eclipse.che.api.environment.server.CheEnvironmentEngine.start(CheEnvironmentEngine.java:263)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.startEnvironmentAndPublishEvents(WorkspaceRuntimes.java:694)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.access$100(WorkspaceRuntimes.java:103)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartTask.call(WorkspaceRuntimes.java:986)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartTask.call(WorkspaceRuntimes.java:948)
        at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalCallable.call(CopyThreadLocalCallable.java:31)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
2018-01-29 21:01:19,712[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 21:01:19,712[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = null
2018-01-29 21:01:19,712[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 21:01:19,719[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 21:01:19,720[aceSharedPool-1]  [INFO ] [m.BaseServerEvaluationStrategy 141]  - cheWorkspacesRoutingSuffixProvider = org.eclipse.che.plugin.docker.client.WorkspacesRoutingSuffixProvider@2c549906
2018-01-29 21:01:19,839[aceSharedPool-1]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 676]   - Launching 'org.eclipse.che.exec' agent at workspace workspacebsetwzqtonmg1guy
2018-01-29 21:01:21,987[aceSharedPool-1]  [ERROR] [.a.a.s.l.AbstractAgentLauncher 165]  - An error occurs while starting 'Exec' agent in 'workspacebsetwzqtonmg1guy' workspace in 'machinepes1c6xee84eo0j9' machine on 'localhost' node. Detailed log:
[STDOUT] OCI runtime exec failed: exec failed: container_linux.go:296: starting container process caused "process_linux.go:108: writing config to pipe caused \"write init-p: broken pipe\"": unknown
2018-01-29 21:01:22,041[aceSharedPool-1]  [ERROR] [o.e.c.a.w.s.WorkspaceManager 683]    - Error occurs while initializing command [/bin/sh, -c, trap '[ -z "$(jobs -p)" ] || kill $(jobs -p); [ -e /tmp/docker-exec-34.pid ] && rm /tmp/docker-exec-34.pid' EXIT; echo $$>/tmp/docker-exec-34.pid; command -v pidof >/dev/null 2>&1 && {
    pidof che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
} || {
    ps -fC che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
}] in docker container ebbdff02acb6214e845b9902f98799313aeee5b039531f60798910e1c3931ab4: Error response from docker API, status: 409, message: Container ebbdff02acb6214e845b9902f98799313aeee5b039531f60798910e1c3931ab4 is not running

org.eclipse.che.api.core.ServerException: Error occurs while initializing command [/bin/sh, -c, trap '[ -z "$(jobs -p)" ] || kill $(jobs -p); [ -e /tmp/docker-exec-34.pid ] && rm /tmp/docker-exec-34.pid' EXIT; echo $$>/tmp/docker-exec-34.pid; command -v pidof >/dev/null 2>&1 && {
    pidof che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
} || {
    ps -fC che-exec-agent >/dev/null 2>&1 && echo 0 || echo 1
}] in docker container ebbdff02acb6214e845b9902f98799313aeee5b039531f60798910e1c3931ab4: Error response from docker API, status: 409, message: Container ebbdff02acb6214e845b9902f98799313aeee5b039531f60798910e1c3931ab4 is not running

        at org.eclipse.che.api.agent.server.launcher.AbstractAgentLauncher.launch(AbstractAgentLauncher.java:115)
        at org.eclipse.che.api.agent.ExecAgentLauncher.launch(ExecAgentLauncher.java:46)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.launchAgents(WorkspaceRuntimes.java:681)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$MachineAgentsLauncher.started(WorkspaceRuntimes.java:1048)
        at org.eclipse.che.api.environment.server.CheEnvironmentEngine.startEnvironmentQueue(CheEnvironmentEngine.java:800)
        at org.eclipse.che.api.environment.server.CheEnvironmentEngine.start(CheEnvironmentEngine.java:263)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.startEnvironmentAndPublishEvents(WorkspaceRuntimes.java:694)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.access$100(WorkspaceRuntimes.java:103)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartTask.call(WorkspaceRuntimes.java:986)
        at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartTask.call(WorkspaceRuntimes.java:948)
        at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalCallable.call(CopyThreadLocalCallable.java:31)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
MatthewC commented 6 years ago

I managed to fix the error by creating increasing MySQL' s innodb_buffer_pool_size - although I am now getting an error stating: Workspace agent is no longer responding. To fix the problem, verify you have a good network connection and restart the workspace. Additionally, the program is using it's local IP, but when I set it to the public IP I'm getting an error saying:

         conn (browser => ws):    [NOT OK]
         conn (server => ws):     [NOT OK]

Any clue on how to fix this?

ghost commented 6 years ago

@GhostCyborg can you open browser dev console and capture a screenshot? Looks like ports are filtered - make sure the ephemeral port range is not filtered for inbound connections.

MatthewC commented 6 years ago

I'd also like to point something else out, the highlighted IP is the internal IP of the server, however when setting the IP to the external one through che.env - I get the error mentioned about regarding WS. As a side note, I changed the default port to 6868 (as 8080 was already used) through the che.env file. I also made sure all ports are accessible.

gcyborg-errorlog1 gcyborg-errorlog2 gcyborg-errorlog3 gcyborg-errorlog4

ghost commented 6 years ago

@GhostCyborg are you running a multi user Che? I can see that Keycloak initialization failed.

Can you share the command you use to start Che?

MatthewC commented 6 years ago

I'm running the command sudo docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock -v /home/ubuntu/Documents/Docker-che/:/data eclipse/che start

MatthewC commented 6 years ago

So..... I managed to fix it. After setting the CHE_DOCKER_IP_EXTERNAL & CHE_DOCKER_IP to my external IP, and double checking some ports, the workspace seems to have started up correctly. I'm going to try using the "multi user che" now, and see if that also works. I personally don't see any critical errors, although see some STDERR here and there (inside the dev-machine and db console)

ghost commented 6 years ago

@GhostCyborg odds are that it's just stdout is treated as stderr.

I'm closing the issue, feel free to reopen if you have further questions (better open a new one if it's a different problem)

MatthewC commented 6 years ago

Will do! One last quick question, how do I enable SSL?

ghost commented 6 years ago

@GhostCyborg only if you deploy to OpenShift and setup certs there. https isn't supported for Che on Docker

MatthewC commented 6 years ago

And from the looks of it, starting it with the "-e CHE_MULTIUSER=true" doesn't work... I don't get an error when running the command, however, trying to connect through the 5000 (KeyCloak) port, and using the port I set for CHE (6868) doesn't load properly but it does connect.

ghost commented 6 years ago

@GhostCyborg I am not sure I understand what you mean by

I don't get an error when running the command, however, trying to connect through the 5000 (KeyCloak) port, and using the port I set for CHE (6868) doesn't load properly but it does connect.

MatthewC commented 6 years ago

Ok, ok, this might seem a bit confusing, but new error! I fixed the issue by rechecking my variable regarding my public IP (CHE_HOST), once I did this the dashboard sort of loaded, as in it redirected me to KeyClock (Port: 5050), where it then gave me the error: Invalid parameter: redirect_uri. gcyborgerrorlog6

Out of curiosity, when I typed in the URL (i.e example.com:6868/dashboard), and when it redirected me to KeyCloak, it used the IP instead of the domain name, where would I change that?

MatthewC commented 6 years ago

Right, so research previous errors, and found out that I have to log into the administration panel and then set my domain name as a Valid Redirect URL. But my question on how to set the correct URL in KeyCloak still stands.

ghost commented 6 years ago

You need to save redirect urls in Kaycloak admin console. Use the urls you will be using to access Che.

MatthewC commented 6 years ago

Yes, I managed to fix that. although what I was referring to was why when I go to example.com:6868/dashboard, it redirects me to KeyCloak, although it uses the IP (so: example.com:6868/dashboard => [PUBLIC_IP]:5050), how would I set it so it redirects me to the domain name and not IP?

ghost commented 6 years ago

-e CHE_HOST=hostname.com Che CLI will do the rest.

MatthewC commented 6 years ago

Thank you, it now works!