scala / scala-jenkins-infra

A Chef cookbook that manages Scala's CI infrastructure.
https://scala-ci.typesafe.com
Apache License 2.0
14 stars 17 forks source link

Unable to run jenkins ecs-slave node #213

Closed rajasekaran07 closed 7 years ago

rajasekaran07 commented 7 years ago

Hi Team, I am trying to run jenkins job with ECS Slave but it is showing your slave is offline. Steps to reproduce. Created ECS default cluster for slave. Created IAM role to give access to that cluster. Created jenkins master with IAM role to communicate with default ECS cluster. Configured ECS setup in jenkins configuration page. Fixed the TCP listen port for JNLP slaves of the Jenkins master (e.g. 5000) Started test job to print echo "hello world" it was not successful. My ECS-logs: [ec2-user@ip-172-31-44-102 ~]$ sudo tail -f /var/log/ecs/ecs-agent.log.2017-01-2 4-05 2017-01-24T05:28:03Z [INFO] Registering Instance with ECS 2017-01-24T05:28:04Z [INFO] Registered! module="api client" 2017-01-24T05:28:04Z [INFO] Registration completed successfully. I am running as 'arn:aws:ecs:us-east-1:274091617646:container-instance/096140a7-b534-446b-a2a3- 85e196cf697c' in cluster 'default' 2017-01-24T05:28:04Z [INFO] Saving state! module="statemanager" 2017-01-24T05:28:04Z [INFO] Beginning Polling for updates 2017-01-24T05:28:04Z [INFO] Event stream DeregisterContainerInstance start liste ning... 2017-01-24T05:28:04Z [INFO] Initializing stats engine 2017-01-24T05:28:04Z [INFO] Creating poll dialer, host: ecs-a-3.us-east-1.amazon aws.com 2017-01-24T05:28:04Z [INFO] Creating poll dialer, host: ecs-t-3.us-east-1.amazon aws.com 2017-01-24T05:28:14Z [INFO] Saving state! module="statemanager" 2017-01-24T05:39:04Z [INFO] Creating poll dialer, host: ecs-t-3.us-east-1.amazonaws.com 2017-01-24T05:43:28Z [INFO] Creating poll dialer, host: ecs-a-3.us-east-1.amazonaws.com 2017-01-24T05:50:04Z [INFO] Creating poll dialer, host: ecs-t-3.us-east-1.amazonaws.com my Jenkins logs: Jan 24, 2017 5:49:43 AM hudson.ExpressionFactory2$JexlExpression evaluate WARNING: Caught exception evaluating: instance.descriptor in /computer/ecs-cloud-1e37b6d04da/configure. Reason: java.lang.reflect.InvocationTargetException java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.commons.jexl.util.PropertyExecutor.execute(PropertyExecutor.java:125) at org.apache.commons.jexl.util.introspection.UberspectImpl$VelGetterImpl.invoke(UberspectImpl.java:314) at org.apache.commons.jexl.parser.ASTArrayAccess.evaluateExpr(ASTArrayAccess.java:185) at org.apache.commons.jexl.parser.ASTIdentifier.execute(ASTIdentifier.java:75) at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83) at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57) at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51) at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80) at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:74) at org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:99) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:99) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:99) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:95) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53) at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.java:95) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:362) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.MetaClass$3.doDispatch(MetaClass.java:197) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649) at org.kohsuke.stapler.Stapler.service(Stapler.java:238) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:135) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:126) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:86) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.AssertionError: class com.cloudbees.jenkins.plugins.amazonecs.ECSSlave is missing its descriptor at jenkins.model.Jenkins.getDescriptorOrDie(Jenkins.java:1533) at hudson.model.Slave.getDescriptor(Slave.java:459) Kindly someone suggest how to fix this issue.

Regards, Raja

rajasekaran07 commented 7 years ago

Hi Team, My task is started now but immediately that task will be stopped. I dont know the reason can you just give some suggestion to fix this issue. Jenkins Logs: Jan 24, 2017 6:12:22 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResources INFO: Resources found in instance arn:aws:ecs:us-east-1:274091617646:container-instance/096140a7-b534-446b-a2a3-85e196cf697c: [ {Name: CPU,Type: INTEGER,DoubleValue: 0.0,LongValue: 0,IntegerValue: 1024,StringSetValue: []} , {Name: MEMORY,Type: INTEGER,DoubleValue: 0.0,LongValue: 0,IntegerValue: 995,StringSetValue: []} , {Name: PORTS,Type: STRINGSET,DoubleValue: 0.0,LongValue: 0,IntegerValue: 0,StringSetValue: [22, 2376, 2375, 51678, 51679]} , {Name: PORTS_UDP,Type: STRINGSET,DoubleValue: 0.0,LongValue: 0,IntegerValue: 0,StringSetValue: []} ] Jan 24, 2017 6:12:22 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResources INFO: Instance arn:aws:ecs:us-east-1:274091617646:container-instance/096140a7-b534-446b-a2a3-85e196cf697c has 995mb of free memory. 512mb are required Jan 24, 2017 6:12:22 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResources INFO: Instance arn:aws:ecs:us-east-1:274091617646:container-instance/096140a7-b534-446b-a2a3-85e196cf697c has 1,024 units of free cpu. 20 units are required Jan 24, 2017 6:12:22 AM com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback call INFO: Created Slave: ecs-cloud-2b525d006ba Jan 24, 2017 6:12:22 AM com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback call INFO: Slave ecs-cloud-2b525d006ba - Slave Task Started : arn:aws:ecs:us-east-1:274091617646:task/dd540fc4-8fa8-4261-8e02-4df1f620ab10 ECS-Logs: aws.com 2017-01-24T05:28:04Z [INFO] Creating poll dialer, host: ecs-t-3.us-east-1.amazon aws.com 2017-01-24T05:28:14Z [INFO] Saving state! module="statemanager" 2017-01-24T05:39:04Z [INFO] Creating poll dialer, host: ecs-t-3.us-east-1.amazonaws.com 2017-01-24T05:43:28Z [INFO] Creating poll dialer, host: ecs-a-3.us-east-1.amazonaws.com 2017-01-24T05:50:04Z [INFO] Creating poll dialer, host: ecs-t-3.us-east-1.amazonaws.com 2017-01-24T05:58:04Z [INFO] No eligible images for deletion for this cleanup cycle 2017-01-24T05:58:04Z [INFO] End of eligible images for deletion Slave-container logs: {"log":"Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behavior\n","stream":"stdout","time":"2017-01-24T06:12:51.468360646Z"} {"log":"Jan 24, 2017 6:12:52 AM hudson.remoting.jnlp.Main createEngine\n","stream":"stderr","time":"2017-01-24T06:12:52.078777831Z"} {"log":"INFO: Setting up slave: ecs-cloud-2b525d006ba\n","stream":"stderr","time":"2017-01-24T06:12:52.078797271Z"} {"log":"Jan 24, 2017 6:12:52 AM hudson.remoting.jnlp.Main$CuiListener \u003cinit\u003e\n","stream":"stderr","time":"2017-01-24T06:12:52.081422273Z"} {"log":"INFO: Jenkins agent is running in headless mode.\n","stream":"stderr","time":"2017-01-24T06:12:52.081434661Z"} {"log":"Jan 24, 2017 6:12:52 AM hudson.remoting.jnlp.Main$CuiListener status\n","stream":"stderr","time":"2017-01-24T06:12:52.118307707Z"} {"log":"INFO: Locating server among [http://52.53.170.100:8080/]\n","stream":"stderr","time":"2017-01-24T06:12:52.118328189Z"} {"log":"Jan 24, 2017 6:13:22 AM hudson.remoting.jnlp.Main$CuiListener error\n","stream":"stderr","time":"2017-01-24T06:13:22.169854689Z"} {"log":"SEVERE: Failed to connect to http://52.53.170.100:8080/tcpSlaveAgentListener/: connect timed out\n","stream":"stderr","time":"2017-01-24T06:13:22.169876089Z"} {"log":"java.io.IOException: Failed to connect to http://52.53.170.100:8080/tcpSlaveAgentListener/: connect timed out\n","stream":"stderr","time":"2017-01-24T06:13:22.169880582Z"} {"log":"\u0009at hudson.remoting.Engine.run(Engine.java:237)\n","stream":"stderr","time":"2017-01-24T06:13:22.169884416Z"} {"log":"Caused by: java.net.SocketTimeoutException: connect timed out\n","stream":"stderr","time":"2017-01-24T06:13:22.169887947Z"} {"log":"\u0009at java.net.PlainSocketImpl.socketConnect(Native Method)\n","stream":"stderr","time":"2017-01-24T06:13:22.169891176Z"} {"log":"\u0009at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)\n","stream":"stderr","time":"2017-01-24T06:13:22.169894606Z"} {"log":"\u0009at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)\n","stream":"stderr","time":"2017-01-24T06:13:22.169897771Z"} {"log":"\u0009at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)\n","stream":"stderr","time":"2017-01-24T06:13:22.169901101Z"} {"log":"\u0009at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)\n","stream":"stderr","time":"2017-01-24T06:13:22.169904281Z"} {"log":"\u0009at java.net.Socket.connect(Socket.java:589)\n","stream":"stderr","time":"2017-01-24T06:13:22.169907244Z"} {"log":"\u0009at sun.net.NetworkClient.doConnect(NetworkClient.java:175)\n","stream":"stderr","time":"2017-01-24T06:13:22.169910155Z"} {"log":"\u0009at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)\n","stream":"stderr","time":"2017-01-24T06:13:22.169913048Z"} {"log":"\u0009at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)\n","stream":"stderr","time":"2017-01-24T06:13:22.169915963Z"} {"log":"\u0009at sun.net.www.http.HttpClient.\u003cinit\u003e(HttpClient.java:211)\n","stream":"stderr","time":"2017-01-24T06:13:22.169918851Z"} {"log":"\u0009at sun.net.www.http.HttpClient.New(HttpClient.java:308)\n","stream":"stderr","time":"2017-01-24T06:13:22.169922118Z"} {"log":"\u0009at sun.net.www.http.HttpClient.New(HttpClient.java:326)\n","stream":"stderr","time":"2017-01-24T06:13:22.169925021Z"} {"log":"\u0009at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)\n","stream":"stderr","time":"2017-01-24T06:13:22.169927943Z"} {"log":"\u0009at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)\n","stream":"stderr","time":"2017-01-24T06:13:22.169930926Z"} {"log":"\u0009at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)\n","stream":"stderr","time":"2017-01-24T06:13:22.169933959Z"} {"log":"\u0009at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)\n","stream":"stderr","time":"2017-01-24T06:13:22.169936928Z"} {"log":"\u0009at hudson.remoting.Engine.run(Engine.java:234)\n","stream":"stderr","time":"2017-01-24T06:13:22.16993996Z"} {"log":"\n","stream":"stderr","time":"2017-01-24T06:13:22.169949275Z"} Regards, Raja

SethTisue commented 7 years ago

this has nothing to do with this repository