Closed rsmartins78 closed 6 years ago
Hi @rsmartins78
It's a bit hard to tell what the problem could be from the diagram. Is the grails.serverURL set to the LB FQDN or CNAME to the LB? Is SSL terminated at the LB? Do you have the LB configured to send the correct headers? http://rundeck.org/docs/administration/configuring-ssl.html#using-an-ssl-terminated-proxy
Is the grails.serverURL set to the LB FQDN or CNAME to the LB? Since the access URL remains the same (via IP) I have not changed the file.
Is SSL terminated at the LB? No, just a simple LB created in Rancher where I set respective ports and redirects
Do you have the LB configured to send the correct headers? No, I'll try with these parameters and let you know soon.
Sounds good! Also, see if there's anything interesting in the rundeck log when trying to login 😃
The parameter -Drundeck.jetty.connector.forwarded=true already is set, but without results.
My service.log
INFO ExecutionUtilService: Execution successful: 482 in project Test
INFO ExecutionService: updated scheduled Execution
WARNING: HTTPS is not enabled, specify -Drundeck.ssl.config=/var/lib/rundeck/server/config/ssl.properties to enable.
2018-03-26 16:54:28.344:INFO:oejs.Server:main: jetty-9.0.7.v20131107
2018-03-26 16:54:33.347:INFO:oejw.StandardDescriptorProcessor:main: NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
2018-03-26 16:54:36.488:INFO:/:main: Initializing Spring root WebApplicationContext
WARNING: HTTPS is not enabled, specify -Drundeck.ssl.config=/var/lib/rundeck/server/config/ssl.properties to enable.
2018-03-26 16:56:41.853:INFO:oejs.Server:main: jetty-9.0.7.v20131107
2018-03-26 16:56:48.578:INFO:oejw.StandardDescriptorProcessor:main: NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
2018-03-26 16:56:51.541:INFO:/:main: Initializing Spring root WebApplicationContext
2018-03-26 16:57:19.576:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-183: Login attempts: 3, Hits: 0, Ratio: 0%.
2018-03-26 16:57:19.577:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-183: Cache Eviction for rsmartins.
2018-03-26 16:57:19.579:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-183: Attempting authentication: CN=RAFAEL DA SILVA MARTINS,CN=Users,DC=indproj,DC=com,DC=br
INFO ProjectManagerService: Loading project definition for Homologacao...
INFO ProjectManagerService: Loaded project Homologacao in 19ms
INFO ProjectManagerService: Loading project definition for Manutencao-Rundeck...
INFO ProjectManagerService: Loaded project Manutencao-Rundeck in 9ms
INFO ProjectManagerService: Loading project definition for Test...
INFO ProjectManagerService: Loaded project Test in 4ms
INFO BootStrap: Starting Rundeck 2.10.8-1 (2018-03-23) ...
INFO BootStrap: using rdeck.base config property: /var/lib/rundeck
INFO BootStrap: loaded configuration: /etc/rundeck/framework.properties
WARN ProjectManagerService: Discovered filesystem project Homologacao, was previously imported, skipping.
WARN ProjectManagerService: Discovered filesystem project Test, was previously imported, skipping.
WARN ProjectManagerService: Discovered filesystem project Manutencao-Rundeck, was previously imported, skipping.
INFO BootStrap: RSS feeds disabled
INFO BootStrap: Preauthentication is disabled
INFO BootStrap: Rundeck is ACTIVE: executions can be run.
WARN BootStrap: The JVM default encoding is not UTF-8: US-ASCII, you may not see output as expected for multibyte locales. Specify -Dfile.encoding=UTF-8 in the JVM options.
INFO BootStrap: Rundeck startup finished in 2557ms
2018-03-26 16:57:51.163:INFO:/:main: Initializing Spring FrameworkServlet 'grails'
2018-03-26 16:57:51.296:INFO:oejsh.ContextHandler:main: Started o.e.j.w.WebAppContext@f5b6e78{/,file:/var/lib/rundeck/exp/webapp/,AVAILABLE}{/var/lib/rundeck/exp/webapp}
2018-03-26 16:57:51.420:INFO:oejs.ServerConnector:main: Started ServerConnector@106ac5f4{HTTP/1.1}{0.0.0.0:4440}
INFO ProjectManagerService: Loading project definition for Manutencao-Rundeck...
INFO ProjectManagerService: Loaded project Manutencao-Rundeck in 384ms
INFO ScheduledExecutionService: scheduling new job in project Manutencao-Rundeck 3408ecd9-e04f-474b-8089-95c4b064eadd: 1:Auto-Commit
INFO ScheduledExecutionService: scheduled job 3408ecd9-e04f-474b-8089-95c4b064eadd. next run: Tue Mar 27 06:10:00 BRT 2018
INFO ScheduledExecutionService: rescheduled job in project Manutencao-Rundeck: 3408ecd9-e04f-474b-8089-95c4b064eadd
INFO ScheduledExecutionService: scheduling new job in project Manutencao-Rundeck 5e177bad-17bf-44b3-a324-059638f37c32: 2:Backup
INFO ScheduledExecutionService: scheduled job 5e177bad-17bf-44b3-a324-059638f37c32. next run: Tue Mar 27 06:00:00 BRT 2018
INFO ScheduledExecutionService: rescheduled job in project Manutencao-Rundeck: 5e177bad-17bf-44b3-a324-059638f37c32
INFO ProjectManagerService: Loading project definition for Test...
INFO ProjectManagerService: Loaded project Test in 35ms
INFO ScheduledExecutionService: scheduling new job in project Test 9ddf8d94-0b89-4aa8-8434-04eebf8be001: 21:Date
INFO ScheduledExecutionService: scheduled job 9ddf8d94-0b89-4aa8-8434-04eebf8be001. next run: Tue Mar 27 10:00:00 BRT 2018
INFO ScheduledExecutionService: rescheduled job in project Test: 9ddf8d94-0b89-4aa8-8434-04eebf8be001
INFO ScheduledExecutionService: scheduling new job in project Test 75e70953-204b-45a8-ac6c-538c249ec80e: 22:DF -H
INFO ScheduledExecutionService: scheduled job 75e70953-204b-45a8-ac6c-538c249ec80e. next run: Mon Mar 26 17:00:00 BRT 2018
INFO ScheduledExecutionService: rescheduled job in project Test: 75e70953-204b-45a8-ac6c-538c249ec80e
2018-03-26 16:58:09.434:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-19: Login attempts: 1, Hits: 0, Ratio: 0%.
2018-03-26 16:58:09.452:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-19: Attempting authentication: CN=MYLOGIN,CN=Users,DC=mydomain,DC=com
2018-03-26 16:58:12.084:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-183: Login attempts: 4, Hits: 0, Ratio: 0%.
After each attempt to login, the last line repeats
I realized that occurs erros in scheduled job executions too
rundeck.services.ExecutionServiceException: Job "DF -H" {{Job 75e70953-204b-45a8-ac6c-538c249ec80e}} is currently being executed {{Execution 483}}
at rundeck.services.ExecutionService.createExecution(ExecutionService.groovy:2236)
at rundeck.quartzjobs.ExecutionJob.initialize(ExecutionJob.groovy:321)
at rundeck.quartzjobs.ExecutionJob.execute_internal(ExecutionJob.groovy:114)
at rundeck.quartzjobs.ExecutionJob$_execute_closure1.doCall(ExecutionJob.groovy:93)
at com.codahale.metrics.Timer.time(Timer.java:99)
at rundeck.quartzjobs.ExecutionJob.execute(ExecutionJob.groovy:92)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
ERROR JobRunShell: Job Test:DF -H:group1/subgroup1.22:DF -H threw an unhandled Exception:
rundeck.services.ExecutionServiceException: Job "DF -H" {{Job 75e70953-204b-45a8-ac6c-538c249ec80e}} is currently being executed {{Execution 483}}
at rundeck.services.ExecutionService.createExecution(ExecutionService.groovy:2236)
at rundeck.quartzjobs.ExecutionJob.initialize(ExecutionJob.groovy:321)
at rundeck.quartzjobs.ExecutionJob.execute_internal(ExecutionJob.groovy:114)
at rundeck.quartzjobs.ExecutionJob$_execute_closure1.doCall(ExecutionJob.groovy:93)
at com.codahale.metrics.Timer.time(Timer.java:99)
at rundeck.quartzjobs.ExecutionJob.execute(ExecutionJob.groovy:92)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
Hi @rsmartins78
Hmm, maybe an issue with the JAAS configs? com.dtolabs.rundeck.jetty.jaas.JettyCachingLdapLoginModule and org.eclipse.jetty.jaas.spi.PropertyFileLoginModule have debug settings, can you try enabling debug, restarting Rundeck and see if the debug statements provide a clue?
The service.log doesn't change
2018-04-02 09:58:50.801:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-14: Login attempts: 1, Hits: 0, Ratio: 0%.
2018-04-02 09:58:50.819:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-14: Attempting authentication: CN=RAFAEL DA SILVA MARTINS,CN=Users,DC=indproj,DC=com,DC=br
2018-04-02 09:58:56.645:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-18: Login attempts: 2, Hits: 0, Ratio: 0%.
2018-04-02 09:58:59.006:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-19: Login attempts: 2, Hits: 0, Ratio: 0%.
2018-04-02 09:59:02.010:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-18: Login attempts: 3, Hits: 1, Ratio: 33.33%.
2018-04-02 09:59:17.055:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-19: Login attempts: 3, Hits: 1, Ratio: 33.33%.
2018-04-02 09:59:20.068:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-18: Login attempts: 4, Hits: 2, Ratio: 50%.
2018-04-02 09:59:21.565:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-19: Login attempts: 4, Hits: 2, Ratio: 50%.
2018-04-02 09:59:23.449:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-18: Login attempts: 5, Hits: 3, Ratio: 60%.
2018-04-02 09:59:24.021:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-19: Login attempts: 5, Hits: 3, Ratio: 60%.
2018-04-02 09:59:24.445:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-18: Login attempts: 6, Hits: 4, Ratio: 66.67%.
2018-04-02 09:59:24.814:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-19: Login attempts: 6, Hits: 4, Ratio: 66.67%.
2018-04-02 09:59:25.183:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-18: Login attempts: 7, Hits: 5, Ratio: 71.43%.
2018-04-02 09:59:25.914:INFO:cdrjj.JettyCachingLdapLoginModule:qtp189568618-14: Login attempts: 7, Hits: 5, Ratio: 71.43%.
it's as if each click on the login button, tries to log into each container and does not do any of them.
Hi @rsmartins78
Are you able to login as as the admin user or another local rundeck account in realm.properties?
I've tried with admin account with no success.
Have you tried only running one rundeck container? It seems to be an issue with session persistence. Let's see if one container and one backend works at least 😛
In my default scenario, it's just one container each, everything works perfectly, I was just testing the container scalability part, but I think it's a problem with rundeck and not with this image. I will check other cases in depth.
Thank you for your willingness to help me.
NP! Sorry I couldn't help. I'm pretty sure it's just a matter of configuring the rundeck instances so they can maintain persistence in an HA setup. Maybe the main Rundeck project will be able to help. Good luck! 😄
If I get any good results, put it here.
Again, thank you for your commitment.
Much appreciated, @rsmartins78 😄
Hello, I have the following problem, but I do not know who to turn to. I use Rancher with Cattle to orchestrate my Docker containers, but when I scale my service to 2 containers or more, Rundeck is stay in loop on the login screen.