SonarSource / docker-sonarqube

:whale: SonarQube in Docker
https://hub.docker.com/_/sonarqube/
GNU Lesser General Public License v3.0
1.38k stars 1.02k forks source link

Moving to Postgresql failed #27

Closed afaucogney closed 8 years ago

afaucogney commented 8 years ago

I got the image in Doker-Hub and ran it with Kitematic.

After it started, I updated the required plugins.

Then I wanted to use the Postgresql db provided, by changing the envVar available in Kitematic as the following:

capture d ecran 2016-04-27 a 16 15 00

I also try with jdbc:postgresql://db:5432/sonar from the documentation linked recipe https://github.com/SonarSource/docker-sonarqube/blob/master/recipes.md

And the log is the following:

2016.04.27 14:14:31 INFO  app[o.s.a.AppFileSystem] Cleaning or creating temp directory /opt/sonarqube/temp
2016.04.27 14:14:31 INFO  app[o.s.p.m.JavaProcessLauncher] Launch process[search]: /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Djava.awt.headless=true -Xmx1G -Xms256m -Xss256k -Djava.net.preferIPv4Stack=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Djava.io.tmpdir=/opt/sonarqube/temp -cp ./lib/common/*:./lib/search/* org.sonar.search.SearchServer /tmp/sq-process5449093179564171090properties
2016.04.27 14:14:32 INFO   es[o.s.p.ProcessEntryPoint]  Starting search
2016.04.27 14:14:32 INFO   es[o.s.s.SearchSettings]  Elasticsearch listening on 127.0.0.1:9001
2016.04.27 14:14:32 INFO   es[o.elasticsearch.node]  [sonar-1461766470828] version[1.7.2], pid[15], build[e43676b/2015-09-14T09:49:53Z]
2016.04.27 14:14:32 INFO   es[o.elasticsearch.node]  [sonar-1461766470828] initializing ...
2016.04.27 14:14:32 INFO   es[o.e.plugins]  [sonar-1461766470828] loaded [], sites []
2016.04.27 14:14:32 INFO   es[o.elasticsearch.env]  [sonar-1461766470828] using [1] data paths, mounts [[/opt/sonarqube/data (Users)]], net usable_space [22.2gb], net total_space [232.6gb], types [vboxsf]
2016.04.27 14:14:35 WARN   es[o.e.bootstrap]  JNA not found. native methods will be disabled.
2016.04.27 14:14:38 INFO   es[o.elasticsearch.node]  [sonar-1461766470828] initialized
2016.04.27 14:14:38 INFO   es[o.elasticsearch.node]  [sonar-1461766470828] starting ...
2016.04.27 14:14:38 INFO   es[o.e.transport]  [sonar-1461766470828] bound_address {inet[/127.0.0.1:9001]}, publish_address {inet[/127.0.0.1:9001]}
2016.04.27 14:14:38 INFO   es[o.e.discovery]  [sonar-1461766470828] sonarqube/WPNyZ44KQG6Hlsgsb561uw
2016.04.27 14:14:41 INFO   es[o.e.cluster.service]  [sonar-1461766470828] new_master [sonar-1461766470828][WPNyZ44KQG6Hlsgsb561uw][a9b41b55809e][inet[/127.0.0.1:9001]]{rack_id=sonar-1461766470828}, reason: zen-disco-join (elected_as_master)
2016.04.27 14:14:41 INFO   es[o.elasticsearch.node]  [sonar-1461766470828] started
2016.04.27 14:14:41 INFO   es[o.e.gateway]  [sonar-1461766470828] recovered [6] indices into cluster_state
2016.04.27 14:14:50 INFO  app[o.s.p.m.Monitor] Process[search] is up
2016.04.27 14:14:50 INFO  app[o.s.p.m.JavaProcessLauncher] Launch process[web]: /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djruby.management.enabled=false -Djruby.compile.invokedynamic=false -Xmx768m -Xms256m -XX:MaxPermSize=160m -XX:+HeapDumpOnOutOfMemoryError -Djava.net.preferIPv4Stack=true -Djava.security.egd=file:/dev/./urandom -Djava.io.tmpdir=/opt/sonarqube/temp -cp ./lib/common/*:./lib/server/*:/opt/sonarqube/lib/jdbc/postgresql/postgresql-9.3-1102-jdbc41.jar org.sonar.server.app.WebServer /tmp/sq-process321600078800162041properties
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=160m; support was removed in 8.0
2016.04.27 14:14:51 INFO  web[o.s.p.ProcessEntryPoint] Starting web
2016.04.27 14:14:51 INFO  web[o.s.s.a.TomcatContexts] Webapp directory: /opt/sonarqube/web
2016.04.27 14:14:52 INFO  web[o.a.c.h.Http11NioProtocol] Initializing ProtocolHandler ["http-nio-0.0.0.0-9000"]
2016.04.27 14:14:52 INFO  web[o.a.t.u.n.NioSelectorPool] Using a shared selector for servlet write/read
2016.04.27 14:14:53 INFO  web[o.s.s.p.ServerImpl] SonarQube Server / 5.4 / 7b02df9be3cd9448699b5857586e1c6e2b28c007
2016.04.27 14:14:53 INFO  web[o.sonar.db.Database] Create JDBC data source for jdbc:postgresql://localhost/sonar
2016.04.27 14:14:53 ERROR web[o.a.c.c.C.[.[.[/]] Exception sending context initialized event to listener instance of class org.sonar.server.platform.PlatformServletContextListener
java.lang.IllegalStateException: Can not connect to database. Please check connectivity and settings (see the properties prefixed by 'sonar.jdbc.').
at org.sonar.db.DefaultDatabase.checkConnection(DefaultDatabase.java:104) ~[sonar-db-5.4.jar:na]
at org.sonar.db.DefaultDatabase.start(DefaultDatabase.java:71) ~[sonar-db-5.4.jar:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_45-internal]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_45-internal]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_45-internal]
at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_45-internal]
at org.picocontainer.lifecycle.ReflectionLifecycleStrategy.invokeMethod(ReflectionLifecycleStrategy.java:110) ~[picocontainer-2.15.jar:na]
at org.picocontainer.lifecycle.ReflectionLifecycleStrategy.start(ReflectionLifecycleStrategy.java:89) ~[picocontainer-2.15.jar:na]
at org.sonar.core.platform.ComponentContainer$1.start(ComponentContainer.java:312) ~[sonar-core-5.4.jar:na]
at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.start(AbstractInjectionFactory.java:84) ~[picocontainer-2.15.jar:na]
at org.picocontainer.behaviors.AbstractBehavior.start(AbstractBehavior.java:169) ~[picocontainer-2.15.jar:na]
at org.picocontainer.behaviors.Stored$RealComponentLifecycle.start(Stored.java:132) ~[picocontainer-2.15.jar:na]
at org.picocontainer.behaviors.Stored.start(Stored.java:110) ~[picocontainer-2.15.jar:na]
at org.picocontainer.DefaultPicoContainer.potentiallyStartAdapter(DefaultPicoContainer.java:1016) ~[picocontainer-2.15.jar:na]
at org.picocontainer.DefaultPicoContainer.startAdapters(DefaultPicoContainer.java:1009) ~[picocontainer-2.15.jar:na]
at org.picocontainer.DefaultPicoContainer.start(DefaultPicoContainer.java:767) ~[picocontainer-2.15.jar:na]
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:134) ~[sonar-core-5.4.jar:na]
at org.sonar.server.platform.platformlevel.PlatformLevel.start(PlatformLevel.java:84) ~[sonar-server-5.4.jar:na]
at org.sonar.server.platform.Platform.start(Platform.java:214) ~[sonar-server-5.4.jar:na]
at org.sonar.server.platform.Platform.startLevel1Container(Platform.java:173) ~[sonar-server-5.4.jar:na]
at org.sonar.server.platform.Platform.init(Platform.java:90) ~[sonar-server-5.4.jar:na]
at org.sonar.server.platform.PlatformServletContextListener.contextInitialized(PlatformServletContextListener.java:43) ~[sonar-server-5.4.jar:na]
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4812) [tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5255) [tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) [tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1408) [tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1398) [tomcat-embed-core-8.0.30.jar:8.0.30]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_45-internal]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_45-internal]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_45-internal]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45-internal]
Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.)
at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1549) ~[commons-dbcp-1.4.jar:1.4]
at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1388) ~[commons-dbcp-1.4.jar:1.4]
at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044) ~[commons-dbcp-1.4.jar:1.4]
at org.sonar.db.profiling.NullConnectionInterceptor.getConnection(NullConnectionInterceptor.java:31) ~[sonar-db-5.4.jar:na]
at org.sonar.db.profiling.ProfiledDataSource.getConnection(ProfiledDataSource.java:323) ~[sonar-db-5.4.jar:na]
at org.sonar.db.DefaultDatabase.checkConnection(DefaultDatabase.java:102) ~[sonar-db-5.4.jar:na]
... 30 common frames omitted
Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:215) ~[postgresql-9.3-1102-jdbc41.jar:na]
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:64) ~[postgresql-9.3-1102-jdbc41.jar:na]
at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:144) ~[postgresql-9.3-1102-jdbc41.jar:na]
at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:29) ~[postgresql-9.3-1102-jdbc41.jar:na]
at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:21) ~[postgresql-9.3-1102-jdbc41.jar:na]
at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:31) ~[postgresql-9.3-1102-jdbc41.jar:na]
at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) ~[postgresql-9.3-1102-jdbc41.jar:na]
at org.postgresql.Driver.makeConnection(Driver.java:410) ~[postgresql-9.3-1102-jdbc41.jar:na]
at org.postgresql.Driver.connect(Driver.java:280) ~[postgresql-9.3-1102-jdbc41.jar:na]
at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38) ~[commons-dbcp-1.4.jar:1.4]
at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582) ~[commons-dbcp-1.4.jar:1.4]
at org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1556) ~[commons-dbcp-1.4.jar:1.4]
at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1545) ~[commons-dbcp-1.4.jar:1.4]
... 35 common frames omitted
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_45-internal]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345) ~[na:1.8.0_45-internal]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_45-internal]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_45-internal]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_45-internal]
at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_45-internal]
at org.postgresql.core.PGStream.<init>(PGStream.java:61) ~[postgresql-9.3-1102-jdbc41.jar:na]
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:109) ~[postgresql-9.3-1102-jdbc41.jar:na]
... 47 common frames omitted
2016.04.27 14:14:53 ERROR web[o.a.c.c.StandardContext] One or more listeners failed to start. Full details will be found in the appropriate container log file
2016.04.27 14:14:53 ERROR web[o.a.c.c.StandardContext] Context [] startup failed due to previous errors
2016.04.27 14:14:54 WARN  web[o.a.c.l.WebappClassLoaderBase] The web application [ROOT] appears to have started a thread named [Timer-0] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.util.TimerThread.mainLoop(Timer.java:552)
 java.util.TimerThread.run(Timer.java:505)
2016.04.27 14:14:54 INFO  web[o.a.c.h.Http11NioProtocol] Starting ProtocolHandler ["http-nio-0.0.0.0-9000"]
2016.04.27 14:14:54 INFO  web[o.s.s.a.TomcatAccessLog] Web server is started
2016.04.27 14:14:54 INFO  web[o.s.s.a.EmbeddedTomcat] HTTP connector enabled on port 9000
2016.04.27 14:14:54 WARN  web[o.s.p.ProcessEntryPoint] Fail to start web
java.lang.IllegalStateException: Webapp did not start
at org.sonar.server.app.EmbeddedTomcat.isReady(EmbeddedTomcat.java:84) ~[sonar-server-5.4.jar:na]
at org.sonar.server.app.WebServer.isReady(WebServer.java:47) [sonar-server-5.4.jar:na]
at org.sonar.process.ProcessEntryPoint.launch(ProcessEntryPoint.java:84) ~[sonar-process-5.4.jar:na]
at org.sonar.server.app.WebServer.main(WebServer.java:68) [sonar-server-5.4.jar:na]
2016.04.27 14:14:54 INFO  web[o.a.c.h.Http11NioProtocol] Pausing ProtocolHandler ["http-nio-0.0.0.0-9000"]
2016.04.27 14:14:55 INFO  web[o.a.c.h.Http11NioProtocol] Stopping ProtocolHandler ["http-nio-0.0.0.0-9000"]
2016.04.27 14:14:55 INFO  web[o.a.c.h.Http11NioProtocol] Destroying ProtocolHandler ["http-nio-0.0.0.0-9000"]
2016.04.27 14:14:55 INFO  web[o.s.s.a.TomcatAccessLog] Web server is stopped
2016.04.27 14:14:55 INFO  app[o.s.p.m.Monitor] Process[search] is stopping
2016.04.27 14:14:55 INFO   es[o.s.p.StopWatcher]  Stopping process
2016.04.27 14:14:55 INFO   es[o.elasticsearch.node]  [sonar-1461766470828] stopping ...
2016.04.27 14:14:57 INFO   es[o.elasticsearch.node]  [sonar-1461766470828] stopped
2016.04.27 14:14:57 INFO   es[o.elasticsearch.node]  [sonar-1461766470828] closing ...
2016.04.27 14:14:57 INFO   es[o.elasticsearch.node]  [sonar-1461766470828] closed
2016.04.27 14:14:57 INFO  app[o.s.p.m.Monitor] Process[search] is stopped
Godin commented 8 years ago

@afaucogney How and where you start PostgreSQL? Note that recipe is about docker-compose , where db is name of a linked container which has PostgreSQL. Also note that in Kitematic with boot2docker localhost refers to VM with Docker.

afaucogney commented 8 years ago

@Godin I'm not so advance in Docker stuff. So I just follow the doc, and expect (from what I understood) that a Postgresql db was on the container. That I just need to change envVar to link Sonar to PostGre ! Did I miss understood ?

https://hub.docker.com/_/sonarqube/

Database configuration By default, the image will use an embedded H2 database that is not suited for production.

The production database is configured with these variables: SONARQUBE_JDBC_USERNAME, SONARQUBE_JDBC_PASSWORD and SONARQUBE_JDBC_URL.

$ docker run -d --name sonarqube \ -p 9000:9000 -p 9092:9092 \ -e SONARQUBE_JDBC_USERNAME=sonar \ -e SONARQUBE_JDBC_PASSWORD=sonar \ -e SONARQUBE_JDBC_URL=jdbc:postgresql://localhost/sonar \ sonarqube More recipes can be found here.

Godin commented 8 years ago

@afaucogney yes, you misunderstood - PostgreSQL is not part of SonarQube image, otherwise it would violate principle of "separation" of containers. Cited example will work, if PostgreSQL is available on localhost, e.g. by starting container with PostgreSQL (or non-container) in addition to container with SonarQube.

And indeed - documentation can be improved to make this obvious. It was written with assumption that reader already has a basic knowledges about containers.

afaucogney commented 8 years ago

Ok, yes it is confusion, especialy because of the command docker run ... sonarqube.....jdbc:postgresql://localhost/sonar

I realy feel that the db is on the container.

So if I well understood, I have to run a side Postgreslq container, that will work with the sonarqube container

Godin commented 8 years ago

@afaucogney yes

afaucogney commented 8 years ago

ok @Godin, this is what I did, and I progressed but still failed !

here is the config from SonarQube:

capture d ecran 2016-04-27 a 21 32 21

here is the config for PostGreSql capture d ecran 2016-04-27 a 21 32 34

here is the log fro Sonar

capture d ecran 2016-04-27 a 21 32 43

her is the log for Postgresql

capture d ecran 2016-04-27 a 21 32 52

Godin commented 8 years ago

@afaucogney I don't see failure on screenshot of SQ startup.

Command line equivalent of usage of https://docs.docker.com/compose/ with https://github.com/SonarSource/docker-sonarqube/blob/master/recipes.md :

docker run --name db -e POSTGRES_USER=sonar -e POSTGRES_PASSWORD=sonar -it postgres
docker run --name sq --link db -e SONARQUBE_JDBC_URL=jdbc:postgresql://db:5432/sonar -p 9000:9000 -it sonarqube
afaucogney commented 8 years ago

Sorry for being incompetent, but please do not make any assumption I will do something that is evident for you.

1/ In the first configuration I put in this thread with 4 screenshots, the SQ SONARQUBE_JDBC_URL envVar is jdbc:postgresql://192.168.99.100:32100/sonar : is that correct. This is the IP + port that is available in the other container (MAC IP PORT).

2/ I try to run you CLI command, it create 2 containers. Without any extra configuration, it fails to run. Do I need to do something else, or it is intended to be the turnkey procedure ?

Godin commented 8 years ago

@afaucogney once again - what means

it fails to run

? command not found, you can't open web interface, exception in log such as "connection refused" or some other, containers unexpectedly stop, something else ?

I'm asking because first message in this thread was about the case, when PostgreSQL wasn't started by you. And I don't see any evidence of "failure" afterwards - the last shown screenshot of SQ log looks absolutely clean (there is no exception, no errors, connection to DB looks successful). So please explain.

neilernst commented 8 years ago

Docker is tricky on Macs (for now).

I ran the Docker-Compose script successfully on my mac, using docker-machine (not Kitematic). I had two suggestions. One, on the Mac the "address" of the SonarQube install is not "localhost" but rather the IP of the virtual machine. For docker-machine this means running docker-machine env <machine_name> - for my case, I access SonarQube by going to http://192.168.99.100:9000 (not localhost, which I think is where it lives on Linux). Not sure how you determine this with kitematic but your console output suggests the SonarQube JDBC url is also the same as mine, so try that with port 9000?

The other thing I had to do in my compose approach was to change permissions on the "run.sh" script in the Sonarqube image directory. For some reason it wasn't allowing the sonar user to execute it.

Godin commented 8 years ago

@neilernst thank you for the assistance here.

Not sure that attempt to put all this information into documentation worth an efforts, this will also require maintenance afterwards. There are too many possible ways to use containers - on Linux, not on Linux, thru docker-machine, thru docker-compose, directly or thru kitematic, remotely, whatever else - you can name it, and new ways are coming. But PRs for https://github.com/SonarSource/docker-sonarqube/blob/master/recipes.md and https://github.com/docker-library/docs/tree/master/sonarqube are welcomed.

Permissions for file "run.sh" - do you remember what was the error and which changes you did?

neilernst commented 8 years ago

Don't recall the specific error - I think in Docker Compose the logs mentioned "could not execute entry script" or something, so I did chmod 755 run.sh, and it worked. There was some mention of this in the Docker documentation.

I agree this is too much for the wiki but I appreciate the compose script. I'll try to send PRs if I find something others could use.

Godin commented 8 years ago

Closing this ticket - no responses from author.

mahadazad commented 8 years ago

@afaucogney actually you need to docker exec into the postgres container and create a database named sonar. But make sure you have already mapped the postgres volumes otherwise the database will be gone when the container exit.

docker exec -it [postgres_container_id] /bin/bash createdb -U [postgres_user] sonar