aws / aws-codedeploy-agent

Host Agent for AWS CodeDeploy
https://aws.amazon.com/codedeploy
Apache License 2.0
328 stars 188 forks source link

CodeDeployAgent service shows its status running, but logs shows error #334

Closed AadiMehta closed 1 year ago

AadiMehta commented 1 year ago

Error in the logs:

2022-09-28 00:32:01 INFO  [codedeploy-agent(49989)]: master 49989: been told to replace child 92509
2022-09-28 00:32:01 INFO  [codedeploy-agent(49989)]: master 49989: not enough child processes running - missing at least 1 - respawning
2022-09-28 00:32:06 INFO  [codedeploy-agent(49989)]: master 49989: Spawned child 1/1
2022-09-28 00:32:07 ERROR [codedeploy-agent(92510)]: Plugin codedeploy could not be loaded: cannot load such file -- webrick/httputils.
2022-09-28 00:32:07 ERROR [codedeploy-agent(92510)]: booting child: error during start or run: LoadError - cannot load such file -- webrick/httputils - <internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:148:in `require'
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:148:in `require'
/opt/codedeploy-agent/vendor/gems/aws-sdk-s3-1.81.0/lib/aws-sdk-s3/legacy_signer.rb:7:in `<top (required)>'
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:96:in `require'
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:96:in `require'
/opt/codedeploy-agent/vendor/gems/aws-sdk-s3-1.81.0/lib/aws-sdk-s3/customizations.rb:10:in `<top (required)>'
/opt/codedeploy-agent/vendor/gems/aws-sdk-s3-1.81.0/lib/aws-sdk-s3.rb:39:in `require_relative'
/opt/codedeploy-agent/vendor/gems/aws-sdk-s3-1.81.0/lib/aws-sdk-s3.rb:39:in `<top (required)>'
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:160:in `require'
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:160:in `rescue in require'
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:149:in `require'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:4:in `<top (required)>'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/register_plugin.rb:2:in `require_relative'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/register_plugin.rb:2:in `<top (required)>'
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:148:in `require'
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:148:in `require'
/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:17:in `block in load_plugins'
/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:13:in `each'
/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:13:in `load_plugins'
/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:29:in `prepare_run'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:64:in `block in prepare_run_with_error_handling'
/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:78:in `with_error_handling'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:63:in `prepare_run_with_error_handling'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:20:in `start'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:206:in `block in spawn_child'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:204:in `fork'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:204:in `spawn_child'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:283:in `block (2 levels) in replace_terminated_children'
/opt/codedeploy-agent/vendor/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'
ubuntu@ip-172-31-8-217:~$ sudo service codedeploy-agent status
● codedeploy-agent.service - LSB: AWS CodeDeploy Host Agent
     Loaded: loaded (/etc/init.d/codedeploy-agent; generated)
     Active: active (running) since Wed 2022-09-28 00:20:22 UTC; 1min 56s ago
       Docs: man:systemd-sysv-generator(8)
    Process: 471 ExecStart=/etc/init.d/codedeploy-agent start (code=exited, status=0/SUCCESS)
      Tasks: 4 (limit: 4635)
     Memory: 104.4M
        CPU: 2.485s
     CGroup: /system.slice/codedeploy-agent.service
             ├─707 "codedeploy-agent: master 707" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""
             └─709 "codedeploy-agent: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller of master 707"

Also I have tried restarting the codedeploy agent but nothing worked. I even upgraded codedeploy agent and even ruby, but nothing seems to help.

NEEDED to RESTART the instance to resolve this.

philstrong commented 1 year ago

What version of the agent we're/are you using? What Linux distro?

AadiMehta commented 1 year ago

OS : Ubuntu 20.04 Version: 1.3.2

t0shiii commented 1 year ago

Please provide your Agent version and Ruby runtime version. Docs Please make sure you are using the latest Agent version with the latest corresponding Ruby version. Docs Please turn on verbose logging to see debug level messages. Docs Please provide the corresponding agent log. Docs

mwjones-aws commented 1 year ago

@AadiMehta any update?

eino-makitalo commented 1 year ago

same here

t0shiii commented 1 year ago

@eino-makitalo if you are seeing the same issue, can you provide the following info: OS Version: Agent Version: Ruby Version: Relevant Log Snippets:

mwjones-aws commented 1 year ago

Please reopen if you still encounter the problem with CodeDeploy Agent 1.6.0