Failed to listen to incoming slave connection

[jgillich]

Running the latest tag on CentOS 7 & Docker 0.11.1, Jenkins fails to bind the port:

Failed to listen to incoming slave connection

java.net.BindException: Failed to listen on port 50000 because it's already in use.
    at hudson.TcpSlaveAgentListener.<init>(TcpSlaveAgentListener.java:75)
    at jenkins.model.Jenkins.<init>(Jenkins.java:825)
    at hudson.model.Hudson.<init>(Hudson.java:82)
    at hudson.model.Hudson.<init>(Hudson.java:78)
    at hudson.WebAppMain$3.run(WebAppMain.java:222)
Caused by: java.net.BindException: Address already in use
    at sun.nio.ch.Net.bind0(Native Method)
    at sun.nio.ch.Net.bind(Net.java:444)
    at sun.nio.ch.Net.bind(Net.java:436)
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
    at hudson.TcpSlaveAgentListener.<init>(TcpSlaveAgentListener.java:73)
    ... 4 more

The port isn't used on the host.

[jgillich]

Jenkins works with aespinosa/jenkins, no idea what is going on here.

[carlossg]

Same thing here. It is due to the init.groovy script, but if I remove it and use the script console to run the same thing, then it works. Maybe it is a timing issue? init.groovy is ran too early?

[michaelneale]

can you share your docker command used to launch this?

Also - please use a current version of docker 0.11 is far to old to support.

[vikraman]

I can reproduce with docker 1.2.0 on ubuntu. docker run -p 8080:8080 jenkins

[michaelneale]

OK thanks - taking a look. Any particular version of ubuntu? that other image doesn't set the slave port... (if you don't set the slave port - then jnlp slaves won't connect - as the port is allocated dynamically - so likely people aren't using it with slaves - only simple cases?)

[michaelneale]

@carlossg if you run it via the groovy console - it won't actually be listening on that port... it just sets a value which means nothing though? as listening port is set on startup? Or were you able to connect jnlp slaves?

[vikraman]

This is on ubuntu 14.04 server. I'm not using any slaves. On the console, I can see that the slave listener is starting, so probably something is trying to start it again.

Running from: /usr/share/jenkins/jenkins.war
webroot: EnvVars.masterEnvVars.get("JENKINS_HOME")
Sep 26, 2014 7:16:28 AM winstone.Logger logInternal
INFO: Beginning extraction from war file
Sep 26, 2014 7:16:28 AM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: jetty-8.y.z-SNAPSHOT
Sep 26, 2014 7:16:30 AM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: NO JSP Support for , did not find org.apache.jasper.servlet.JspServlet
Jenkins home directory: /var/jenkins_home found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
Sep 26, 2014 7:16:30 AM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: Started SelectChannelConnector@0.0.0.0:8080
Sep 26, 2014 7:16:30 AM winstone.Logger logInternal
INFO: Winstone Servlet Engine v2.0 running: controlPort=disabled
Sep 26, 2014 7:16:30 AM jenkins.InitReactorRunner$1 onAttained
INFO: Started initialization
Sep 26, 2014 7:16:36 AM jenkins.InitReactorRunner$1 onAttained
INFO: Listed all plugins
Sep 26, 2014 7:16:36 AM jenkins.InitReactorRunner$1 onAttained
INFO: Prepared all plugins
Sep 26, 2014 7:16:36 AM jenkins.InitReactorRunner$1 onAttained
INFO: Started all plugins
Sep 26, 2014 7:16:36 AM jenkins.InitReactorRunner$1 onAttained
INFO: Augmented all extensions
Sep 26, 2014 7:16:36 AM jenkins.InitReactorRunner$1 onAttained
INFO: Loaded all jobs
Sep 26, 2014 7:16:36 AM jenkins.util.groovy.GroovyHookScript execute
INFO: Executing bundled script: file:/var/jenkins_home/war/WEB-INF/init.groovy
--> setting agent port for jnlp
Sep 26, 2014 7:16:36 AM hudson.TcpSlaveAgentListener <init>
INFO: JNLP slave agent listener started on TCP port 50000
Sep 26, 2014 7:16:38 AM org.jenkinsci.main.modules.sshd.SSHD start
INFO: Started SSHD at port 43338
Sep 26, 2014 7:16:38 AM jenkins.InitReactorRunner$1 onAttained
INFO: Completed initialization
Sep 26, 2014 7:16:38 AM hudson.WebAppMain$3 run
INFO: Jenkins is fully up and running
Effective SlaveRestarter on master: null

I can connect to it using netcat if I forward port 50000.

Sep 26, 2014 7:16:47 AM hudson.TcpSlaveAgentListener$ConnectionHandler run
INFO: Accepted connection #1 from /172.27.3.42:42018

[michaelneale]

@vikraman so you don't see that error?

[carlossg]

@michaelneale the error doesn't show in the console but in the web interface at /administrativeMonitor/hudson.model.Hudson.tcpBind/

The port 50000 is correctly opened If I run the groovy script in the Jenkins web console the port is changed. Verified with netstat in the instance

[michaelneale]

@carlossg ok interesting... so few questions:

1) when you use init.groovy - does it actually listen on 50000 - and start - and its just that something is trying again, later? 2) (assuming 1 is kaput for you) - do you think we should just ditch the init.groovy? if it changes what it listens on, I see no need to hard code it ahead of time - as long as people know to set it at some point? (good practice? your preference?)

[carlossg]

1) yes It all works fine, other than the annoying message in the web interface

When you click on "see the log for more details"

[carlossg]

It does work despite the log error, I tested with

JENKINS=192.168.59.103
docker run -p 8080:8080 -p 50000:50000 jenkins
# created the slave by hand in the web ui
docker run -t -i dgageot/java8 /bin/sh -c "curl -O http://$JENKINS:8080/jnlpJars/slave.jar && java -jar slave.jar -jnlpUrl http://$JENKINS:8080/computer/myslave/slave-agent.jnlp"

Slave is connected

[michaelneale]

OK good news - have a work around - I defer the setting of that port so that it doesn't try back later on. Not sure of the race condition that causes it - but this will do for now.

[michaelneale]

Will be in next LTS release of this image.

[xbeta]

I still have this issue on the 1.565.1 (currently the latest LTS). Can we please not close any ticket if we can still reproduce this issue?

[michaelneale]

If you build it from the docker file it will be ok.. It can take days for the official build team to approve.new version.

It's a bit like submitting to the app store On Saturday, September 27, 2014, Sam Xiao notifications@github.com wrote:

I still have this issue on the 1.565.1 (currently the latest LTS). Can we please not close any ticket if we can still reproduce this issue?

— Reply to this email directly or view it on GitHub https://github.com/cloudbees/jenkins-ci.org-docker/issues/6#issuecomment-57027638 .

Michael D Neale home: www.michaelneale.net blog: michaelneale.blogspot.com

[Ohadbasan]

I still hit that problem. wasn't it fixed?

thanks

[michaelneale]

Yes it should be using the init groovy. @ndeloof is the init.groovy still in use that sets the fixed slave port?

[ndeloof]

yes it is https://github.com/cloudbees/jenkins-ci.org-docker/blob/master/Dockerfile#L22

[holmesb]

Still having this 'port 50000 already in use' problem in current LTS Jenkins release, running on Kubernetes. Restarting the Jenkins container within the pod seems a valid workaround. Prob still timing issues during startup.

[nitish-predera]

I was having the same issue after restoring Jenkins backup to my Jenkins setup. It was deployed using Helm charts on a Kubernetes cluster.

Manage Jenkins > Configure Global Security > Agents

Disable the setting "TCP port for inbound agents" and Apply, then undo the radio to Fixed: 50000 option and save.

Worked for me.