Closed cytown closed 4 years ago
The log of a NEW one should looks like this:
ls: /var/gerrit/review_site/git: No such file or directory
First time initialize gerrit...
[2019-11-12 07:54:10,331] [main] INFO com.google.gerrit.server.config.GerritServerConfigProvider : No /var/gerrit/review_site/etc/gerrit.config; assuming defaults
Generating SSH host key ... rsa... ed25519... ecdsa 256... ecdsa 384... ecdsa 521... done
Initialized /var/gerrit/review_site
Reindexing projects: 100% (2/2)
Reindexed 2 documents in projects index in 0.2s (10.5/s)
/gerrit-entrypoint.sh: ignoring /docker-entrypoint-init.d/*
...
Would you please try the most simple one without ldap and external db like this:
docker run \
-p 8080:8080 \
-p 29418:29418 \
-d openfrontier/gerrit
And let me know the result? At least this one works well in my cluster.
Same result, I swear few days before it works just fine.
Maybe the docker image was corrupt??? I never see this kind of things...
I'll try remove the image and fetch it again.
Same result after remove and fetch the image....
The logs:
root@dockers:/# docker run -p 8080:8080 -d --name ggg openfrontier/gerrit
35471c0c56a097b36c7fe47daaafe4c81d435d36bc3fdc8eb905e0f05a7839e0
root@dockers:/# docker logs -f ggg
ls: /var/gerrit/review_site/git: No such file or directory
First time initialize gerrit...
[2019-11-12 08:08:45,998] [main] INFO com.google.gerrit.server.config.GerritServerConfigProvider : No /var/gerrit/review_site/etc/gerrit.config; assuming defaults
Generating SSH host key ... rsa... ed25519... ecdsa 256... ecdsa 384... ecdsa 521... done
Initialized /var/gerrit/review_site
Exception in thread "main" com.google.inject.CreationException: Unable to create injector, see the following errors:
1) Error in custom provider, java.lang.IllegalArgumentException
at com.google.gerrit.server.cache.CacheModule.bindCache(CacheModule.java:173) (via modules: com.google.gerrit.pgm.util.BatchProgramModule -> com.google.gerrit.server.change.ChangeKindCacheImpl$1)
while locating com.google.common.cache.Cache<com.google.gerrit.server.change.ChangeKindCacheImpl$Key, com.google.gerrit.extensions.client.ChangeKind> annotated with @com.google.inject.name.Named(value=change_kind)
Caused by: java.lang.IllegalArgumentException
......
I try it again in my local environment. It works. It has to be some environment differences between yours and mine. I tested it in a ubuntu 18.04 vm with docker version 18.06.3-ce. Yours?
I am using a debian 10.1 and docker version 19.03.3.
Also, it works fine just day before...
update: the official image gerrit works fine.
Then would you please try openfrontier/gerrit:3.0.2 instead of the latest? The latest one was upgraded to 3.0.4 today.
Seems not this issue, all my jenkins container stop working now, include openfrontier/jenkins and jenkins/jenkins....
Maybe the base image alpine or something goes wrong???
The following are the logs:
INFO: Started w.@33f676f6{Jenkins v2.176.1,/jenkins,file:///var/jenkins_home/war/,AVAILABLE}{/var/jenkins_home/war}
Nov 12, 2019 11:42:59 AM org.eclipse.jetty.server.AbstractConnector doStart
INFO: Started ServerConnector@668bc3d5{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
Nov 12, 2019 11:42:59 AM org.eclipse.jetty.server.Server doStart
INFO: Started @1370ms
Nov 12, 2019 11:42:59 AM winstone.Logger logInternal
INFO: Winstone Servlet Engine v4.0 running: controlPort=disabled
Nov 12, 2019 11:43:01 AM hudson.util.BootFailure publish
SEVERE: Failed to initialize Jenkins
hudson.util.HudsonFailedToLoad: java.lang.IllegalArgumentException
at hudson.WebAppMain$3.run(WebAppMain.java:250)
Caused by: java.lang.IllegalArgumentException
at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1314)
at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1237)
at jenkins.InitReactorRunner.run(InitReactorRunner.java:43)
at jenkins.model.Jenkins.executeReactor(Jenkins.java:1125)
at jenkins.model.Jenkins.<init>(Jenkins.java:932)
at hudson.model.Hudson.<init>(Hudson.java:85)
at hudson.model.Hudson.<init>(Hudson.java:81)
at hudson.WebAppMain$3.run(WebAppMain.java:233)
Nov 12, 2019 11:43:01 AM jenkins.model.Jenkins cleanUp
INFO: Stopping Jenkins
Nov 12, 2019 11:43:01 AM jenkins.model.Jenkins$18 onAttained
INFO: Started termination
Nov 12, 2019 11:43:01 AM jenkins.model.Jenkins$18 onTaskFailed
SEVERE: Failed NioChannelSelector.cleanUp
java.lang.IllegalArgumentException: Unable to inject class jenkins.slaves.NioChannelSelector
at hudson.init.TaskMethodFinder.lookUp(TaskMethodFinder.java:125)
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:105)
at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
at jenkins.model.Jenkins$17.execute(Jenkins.java:3375)
at org.jvnet.hudson.reactor.Reactor$Node.runIfPossible(Reactor.java:139)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:128)
at jenkins.model.Jenkins$17.execute(Jenkins.java:3375)
at org.jvnet.hudson.reactor.Reactor$Node.runIfPossible(Reactor.java:139)
at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:276)
at jenkins.model.Jenkins._cleanUpRunTerminators(Jenkins.java:3372)
at jenkins.model.Jenkins.cleanUp(Jenkins.java:3293)
at hudson.WebAppMain$3.run(WebAppMain.java:254)
Nov 12, 2019 11:43:01 AM jenkins.model.Jenkins$18 onTaskFailed
SEVERE: Failed IOHubProvider.cleanUp
java.lang.IllegalArgumentException: Unable to inject class jenkins.slaves.IOHubProvider
at hudson.init.TaskMethodFinder.lookUp(TaskMethodFinder.java:125)
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:105)
at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
at jenkins.model.Jenkins$17.execute(Jenkins.java:3375)
at org.jvnet.hudson.reactor.Reactor$Node.runIfPossible(Reactor.java:139)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:128)
at jenkins.model.Jenkins$17.execute(Jenkins.java:3375)
at org.jvnet.hudson.reactor.Reactor$Node.runIfPossible(Reactor.java:139)
at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:276)
at jenkins.model.Jenkins._cleanUpRunTerminators(Jenkins.java:3372)
at jenkins.model.Jenkins.cleanUp(Jenkins.java:3293)
at hudson.WebAppMain$3.run(WebAppMain.java:254)
Nov 12, 2019 11:43:01 AM jenkins.model.Jenkins$18 onAttained
INFO: Completed termination
I wonder if all of the alpine based images stop working on your server? What's the Architecture of your server? Can you show me the echo of docker info on your server like below? You can try running nginx:alpine.
Client:
Debug Mode: false
Server:
Containers: 13
Running: 13
Paused: 0
Stopped: 0
Images: 11
Server Version: 19.03.4
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: active
NodeID: 46t7a1ckvv1irgkotfe3fkqji
Is Manager: false
Node Address: 172.20.200.76
Manager Addresses:
172.20.200.75:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
init version: fec3683
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.15.0-47-generic
Operating System: Ubuntu 18.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 11.73GiB
Name: ubuntu76
ID: K46F:QONC:SALP:YQX6:2R4S:UTDT:WYN6:P46A:3ZXM:BG3B:JMPN:KUM6
Docker Root Dir: /opt/data/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Registry Mirrors:
Live Restore Enabled: false
BTW, I tested the latest image on this one, which is docker 19.03.4, again. Nothing wrong....
# docker -H unix:///var/run/docker.sock info
Client:
Debug Mode: false
Server:
Containers: 13
Running: 9
Paused: 0
Stopped: 4
Images: 18
Server Version: 19.03.3
Storage Driver: vfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 5.0.15-1-pve
Operating System: Debian GNU/Linux 10 (buster)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 8GiB
Name: dockers
ID: 3NEC:V2JW:QWAV:PJN6:UH5C:PJXT:HQTU:IRKU:35AI:ACGP:VDAX:3B7Y
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
nginx:alpine has no issue, maybe it's from openjdk:alpine or something like that...
Now the openfrontier/jenkins:latest works fine...
@thinkernel Found the reason, I start the lxc with option"lxc.apparmor.profile: unconfined", then it happened...
Just remove the option and everything works fine.
Thanks and sorry for any trouble.
Happy to know the issue is solved. Thanks for sharing the information.
The start command here: