feverLu
commented
7 years ago This error message doesn't seem to be ruby sdk version not supported. Do you have a DeploymentId and region info that I can investigate?
Closed Jinkxed closed 1 year ago
feverLu
commented
7 years ago This error message doesn't seem to be ruby sdk version not supported. Do you have a DeploymentId and region info that I can investigate?
Jinkxed
commented
7 years ago Sure had lots of them but can't see past the last 100.
Here is one of the latest ones though: DeploymentID: d-TWVJDS5RO Region: us-east-1
Was a single instance of the group that just fails with no messages because the agent was unresponsive.
Jinkxed
commented
7 years ago Here is another from the same region single instance.
DeploymentID: d-V276NI8RO Region: us-east-1
Jinkxed
commented
7 years ago Please let me know if I can provide any more data or help on this one. I'd love any ideas on how to resolve.
mattlo
commented
6 years ago I also had this happen but it started right after we had to install a gem dependency. OP's fix works.
Jinkxed
commented
6 years ago This happened again this afternoon. Can't really pin point why yet, but doing my above steps resolves it.
I have a sneaky suspicion it has to do with a system bundler gem being installed. Instead of being in shims or rvm gemsets.
mattlo
commented
6 years ago @sc-chad for us, it was something with global dependencies installed on the box. In our narrow scope case, our app's dependency had the same dependent as the CodeDeploy agent, but with different versions. Wiping, installing CodeDeploy first, then our app's works but only by luck (nothing broke in our app's dependency with the mismatched peer dependency).
This CodeDeploy agent really needs to be in a container or bundled application where code is already rolled up.
CoreyKaylor
commented
6 years ago This happened to us last night and seems to correlate with the code-deploy agent getting updated to a newer version. The above workaround seems to have resolved things for us.
xtitter
commented
6 years ago I'm seeing this same error on the master node of AWS EMR clusters (version 5.7 and up). I haven't got the workaround above to help yet.
floodedcodeboy
commented
6 years ago so this bit me in the ass too. Clearly it's still an issue in OFFICIAL_1.0-1.1352_rpm
Seems running gem install aws-sdk worked fine after codedeploy was installed - then after the codedeploy-agent auto update; it broke.
starrago
commented
6 years ago Any solution on this?
2017-12-11 15:43:27 ERROR [codedeploy-agent(16258)]: booting child: error during start or run: NoMethodError - undefined method underscore' for Seahorse::Util:Module - /root/aws-codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:51:incommand'
/root/aws-codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:76:in <class:CommandExecutor>' /root/aws-codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:20:in<module:Plugins>' /root/aws-codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:17:in<top (required)>' /root/aws-codedeploy-agent/lib/instance_agent/plugins/codedeploy/register_plugin.rb:2:inrequire_relative'
/root/aws-codedeploy-agent/lib/instance_agent/plugins/codedeploy/register_plugin.rb:2:in <top (required)>' /usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:120:inrequire'
/usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:120:in require' /root/aws-codedeploy-agent/lib/instance_agent/runner/child.rb:16:inblock in load_plugins'
/root/aws-codedeploy-agent/lib/instance_agent/runner/child.rb:12:in each' /root/aws-codedeploy-agent/lib/instance_agent/runner/child.rb:12:inload_plugins'
/root/aws-codedeploy-agent/lib/instance_agent/runner/child.rb:28:in prepare_run' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:64:inblock in prepare_run_with_error_handling'
/root/aws-codedeploy-agent/lib/instance_agent/runner/child.rb:55:in with_error_handling' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:63:inprepare_run_with_error_handling'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:20:in start' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:206:inblock in spawn_child'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:204:in fork' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:204:inspawn_child'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:283:in block (2 levels) in replace_terminated_children' /var/lib/gems/2.3.0/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:inblock in create_with_logging_context'
2017-12-11 15:43:27 INFO [codedeploy-agent(15390)]: master 15390: Received CHLD - cleaning dead child process
2017-12-11 15:43:27 INFO [codedeploy-agent(15390)]: master 15390: been told to replace child 16258
2017-12-11 15:43:27 INFO [codedeploy-agent(15390)]: master 15390: not enough child processes running - missing at least 1 - respawning
2017-12-11 15:43:32 INFO [codedeploy-agent(15390)]: master 15390: Spawned child 1/1
2017-12-11 15:43:32 ERROR [codedeploy-agent(16262)]: booting child: error during start or run: NoMethodError - undefined method underscore' for Seahorse::Util:Module - /root/aws-codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:51:incommand'
/root/aws-codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:76:in <class:CommandExecutor>' /root/aws-codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:20:in<module:Plugins>' /root/aws-codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:17:in<top (required)>' /root/aws-codedeploy-agent/lib/instance_agent/plugins/codedeploy/register_plugin.rb:2:inrequire_relative'
/root/aws-codedeploy-agent/lib/instance_agent/plugins/codedeploy/register_plugin.rb:2:in <top (required)>' /usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:120:inrequire'
/usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:120:in require' /root/aws-codedeploy-agent/lib/instance_agent/runner/child.rb:16:inblock in load_plugins'
/root/aws-codedeploy-agent/lib/instance_agent/runner/child.rb:12:in each' /root/aws-codedeploy-agent/lib/instance_agent/runner/child.rb:12:inload_plugins'
/root/aws-codedeploy-agent/lib/instance_agent/runner/child.rb:28:in prepare_run' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:64:inblock in prepare_run_with_error_handling'
/root/aws-codedeploy-agent/lib/instance_agent/runner/child.rb:55:in with_error_handling' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:63:inprepare_run_with_error_handling'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:20:in start' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:206:inblock in spawn_child'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:204:in fork' /opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:204:inspawn_child'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:283:in block (2 levels) in replace_terminated_children' /var/lib/gems/2.3.0/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:inblock in create_with_logging_context'
2017-12-11 15:43:32 INFO [codedeploy-agent(15390)]: master 15390: Received CHLD - cleaning dead child process
2017-12-11 15:43:32 INFO [codedeploy-agent(15390)]: master 15390: been told to replace child 16262
2017-12-11 15:43:32 INFO [codedeploy-agent(15390)]: master 15390: not enough child processes running - missing at least 1 - respawning
^C
starrago
commented
6 years ago In our case we followed the following steps and it worked (Ubuntu 16, Ruby app with Puma web server):
systemctl stop codedeploy-agent.service apt-get purge codedeploy-agent rm -rf /opt/codedeploy-agent/ gem uninstall -aIx cd /var/www/html nano Gemfile gem 'aws-sdk', '~> 3.0', '>= 3.0.1' gem 'aws-sdk-v1' apt install ruby-bundler bundle install --path=. cd /home/ubuntu wget https://aws-codedeploy-us-east-1.s3.amazonaws.com/latest/install chmod +x ./install ./install auto
rebooted and everything worked fine.
floodedcodeboy
commented
6 years ago @starrago let us know if it still works after the agent auto updates itself ..
starrago
commented
6 years ago @floodedcodeboy
It's been working so far.
cat /opt/codedeploy-agent/.version
agent_version: OFFICIAL_1.0-1.1352_deb
Jinkxed
commented
6 years ago I had one quit working today (not sure if there are more yet), uninstalled and reinstalled fresh. The agent failed to start as bundle wasn't ran.
Had to manually go into /opt/codedeploy-agent and run bundle install and reboot the server.
We use the rpm based version on Amazon Linux: releases/codedeploy-agent-1.0-1.1352.noarch.rpm
deopard
commented
6 years ago Same problem here with Amazon Linux version: agent_version: OFFICIAL_1.0-1.1352_rpm system default ruby version: 2.3.3, 2.4.3 (tried with both versions)
It works fine right after following the instructions, but when deploying the AMI to a ASG the error occurs again.
roy211
commented
6 years ago Hello Same problem. But it is working now having applied the following changes:
gem uninstall aws-sdk-3.0.1
gem uninstall aws-sdk-core
restart codedeployagent.
note: update gem with sudo gem update --system if required??
this AWS EMR clusters
Jinkxed
commented
6 years ago As of this morning:
Package: codedeploy-agent-1.0-1.1448.noarch OS: Amazon Linux
/etc/init.d/codedeploy-agent status
You are trying to install in deployment mode after changing
your Gemfile. Run `bundle install` elsewhere and add the
updated Gemfile.lock to version control.
If this is a development machine, remove the /opt/codedeploy-agent/Gemfile freeze
by running `bundle install --no-deployment`.
The gemspecs for path gems changedNothing has changed with this image in a couple months, we do however do yum updates automatically.
Uninstalling the package and reinstalling didn't fix.
Had to go to /opt/codedeploy-agent and run:
bundle install --no-deployment
/etc/init.d/codedeploy-agent restartWhich then gives me:
2018-02-28 17:26:34 ERROR [codedeploy-agent(29901)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller: Cannot reach InstanceService: Aws::CodeDeployCommand::Errors::AccessDeniedException -
2018-02-28 17:26:34 ERROR [codedeploy-agent(29901)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller: Cannot reach InstanceService: Aws::CodeDeployCommand::Errors::AccessDeniedException -Until I restart the server.
Jinkxed
commented
6 years ago Looks like on a fresh install of 1.0-1.1448 it's missing:
/etc/init.d/codedeploy-agent status
Bundler could not find compatible versions for gem "concurrent-ruby":
In Gemfile:
aws_codedeploy_agent was resolved to 0.1, which depends on
concurrent-ruby (~> 1.0.5)
Could not find gem 'concurrent-ruby (~> 1.0.5)', which is required by gem 'aws_codedeploy_agent', in any of the sources.bundle install --no-deployment
Fetching gem metadata from http://rubygems.org/..........
Resolving dependencies...
Using archive-tar-minitar 0.5.2
Using aws-sigv4 1.0.2
Using jmespath 1.3.1
Fetching aws-sdk-core 2.11.6 (was 2.10.98)
Installing aws-sdk-core 2.11.6 (was 2.10.98)
Fetching concurrent-ruby 1.0.5
Installing concurrent-ruby 1.0.5
Using docopt 0.5.0
Using gli 2.17.1
Using json_pure 1.8.6
Using little-plugger 1.1.4
Fetching multi_json 1.13.1 (was 1.12.2)
Installing multi_json 1.13.1 (was 1.12.2)
Using logging 1.8.2
Using rubyzip 1.1.7
Using simple_pid 0.2.1
Using aws_codedeploy_agent 0.1 from source at `.`
Using bundler 1.16.0
Using codedeploy-commands 1.0.0 from source at `vendor/gems/codedeploy-commands-1.0.0`
Using process_manager 0.0.13 from source at `vendor/gems/process_manager-0.0.13`
Bundle complete! 3 Gemfile dependencies, 17 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.Starting the agent gives access denied permissions ( this really sucks btw because we have to reboot the server here ).
v1jayr
commented
6 years ago Can you provide the following information:
asaf-erlich
commented
6 years ago Can you also list the ruby version you're using the all the gems you have installed? People that are installing using the default approach do get that gem installed on their box, but its possible that your gem path is not looking there. I'd like to get all the information so we can reproduce and fix your problem.
Thanks, -Asaf
rohkat-aws
commented
6 years ago @sc-chad can you please provide the above mentioned details . So that we can help solve the issue . Or should i just close this.
Jinkxed
commented
6 years ago Sorry this dropped off my radar.
Initial state: Agent version, Ruby version:
ruby -v
ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
Agent version was the one prior to: 1.0-1.1448How did you install the latest release? with auto installer or manually installed the rpm?
Forked version of: https://forge.puppet.com/walkamongus/codedeploy
# This class is called from codedeploy for install. Can ignore the debian portions we use Amazon Linux.
#
class codedeploy::install {
case $::osfamily {
'RedHat', 'Amazon': {
exec { 'codedeploy_safety_check':
command => '/usr/bin/yum remove codedeploy-agent -y && /bin/rm -rf /tmp/install',
creates => '/opt/codedeploy-agent/.version'
}
exec { 'download_codedeploy_installer':
command => '/usr/bin/wget --quiet --timestamping https://aws-codedeploy-us-east-1.s3.amazonaws.com/latest/install',
cwd => '/tmp',
creates => '/tmp/install'
}
file { '/tmp/install':
ensure => present,
owner => 'root',
group => 'root',
mode => '0740',
subscribe => Exec['download_codedeploy_installer'],
notify => Exec['install_codedeploy_agent'],
}
exec { 'install_codedeploy_agent':
command => '/tmp/install auto',
cwd => '/tmp',
creates => '/opt/codedeploy-agent/.version'
}
}
'Debian': {
if ! defined(Package['awscli']) {
package { 'awscli':
ensure => present,
}
}
if ! defined(Package['ruby2.0']) {
package { 'ruby2.0':
ensure => present,
}
}
exec { 'download_codedeploy_installer':
command => '/usr/bin/wget --quiet --timestamping https://s3.amazonaws.com/aws-codedeploy-us-east-1/latest/install',
cwd => '/tmp',
creates => '/tmp/install'
}
file { '/tmp/install':
ensure => present,
owner => 'root',
group => 'root',
mode => '0740',
subscribe => Exec['download_codedeploy_installer'],
notify => Exec['install_codedeploy_agent'],
}
exec { 'install_codedeploy_agent':
command => '/tmp/install auto',
cwd => '/tmp',
refreshonly => true,
}
}
default: {
fail("${::operatingsystem} not supported")
}
}
}Was the older version of agent still installed while you updated to latest version?
Yes
You mentioned "status" command returned the error. Did the install step complete successfully?
Yes the install looks like it works correctly, but gets access denied messages until I reboot the server.
We use iam roles for permissions, so I'm assuming the service isn't able to assume the role until it's been rebooted? Just a guess here.
marcmillien
commented
6 years ago I solved this issue by modifying the init script this way: Replace:
nohup $BIN start >/dev/null </dev/null 2>&1By:
GEM_HOME=/opt/codedeploy-agent/vendor GEM_PATH=/opt/codedeploy-agent/vendor nohup $BIN start >/dev/null </dev/null 2>&1This will force ruby to use gem dependencies from the codedeploy directory and so will not conflict with system installed gem.
rohkat-aws
commented
6 years ago @sc-chad @marcmillien seems like the issue is being caused due to the version mismatch between aws-ruby sdk being used by the agent and the system. @marcmillien i like the idea of GEM_HOME, will look into it further . Thanks for the response @sc-chad
rohkat-aws
commented
6 years ago @sc-chad @marcmillien does the issue happen only when we install the sdk and then auto update of the agent happens. Because i havenot been able to reproduce it. If you guys can share the steps that would help me reproduce it, i can try putting a fix for this.
whtevn
commented
6 years ago does the issue happen only when we install the sdk and then auto update of the agent happens.
That does seem to be the case. It's very difficult to reproduce. If I spin up a new machine, deploys will work for an amount of time, but that amount of time is unclear. They will eventually stop, and uninstalling gems / restarting the codedeploy agent does not seem to fix the situation.
It really seems like the issue is that the gems are interfering with one another. I can send you my list of gems, if that helps? The list is included below.
Please let me know if there is anything else I can provide to get this resolved. I see that this has been an issue for a while, but it has only started to affect us recently. We can't seem to correlate an exact action with this issue, but I can say that we recently updated our app's aws sdk to 3.0. However, uninstalling that gem and restarting the codedeploy agent does not seem to correct the issue.
# gem list --local
*** LOCAL GEMS ***
actioncable (5.1.5)
actionmailer (5.1.5)
actionpack (5.1.5)
actionview (5.1.5)
activejob (5.1.5)
activemodel (5.1.5)
activerecord (5.1.5)
activesupport (5.1.5)
addressable (2.5.2)
arel (8.0.0)
aws-eventstream (1.0.0)
aws-partitions (1.91.0)
aws-sdk (3.0.1, 2.11.15)
aws-sdk-acm (1.7.0)
aws-sdk-acmpca (1.0.0)
aws-sdk-alexaforbusiness (1.4.0)
aws-sdk-apigateway (1.10.0)
aws-sdk-applicationautoscaling (1.8.0)
aws-sdk-applicationdiscoveryservice (1.1.0)
aws-sdk-appstream (1.9.0)
aws-sdk-appsync (1.2.0)
aws-sdk-athena (1.0.1)
aws-sdk-autoscaling (1.5.0)
aws-sdk-autoscalingplans (1.2.0)
aws-sdk-batch (1.4.0)
aws-sdk-budgets (1.7.0)
aws-sdk-cloud9 (1.1.0)
aws-sdk-clouddirectory (1.3.0)
aws-sdk-cloudformation (1.5.0)
aws-sdk-cloudfront (1.2.0)
aws-sdk-cloudhsm (1.3.0)
aws-sdk-cloudhsmv2 (1.1.0)
aws-sdk-cloudsearch (1.0.1)
aws-sdk-cloudsearchdomain (1.0.1)
aws-sdk-cloudtrail (1.0.1)
aws-sdk-cloudwatch (1.5.0)
aws-sdk-cloudwatchevents (1.3.0)
aws-sdk-cloudwatchlogs (1.3.0)
aws-sdk-codebuild (1.9.0)
aws-sdk-codecommit (1.3.0)
aws-sdk-codedeploy (1.5.0)
aws-sdk-codepipeline (1.3.0)
aws-sdk-codestar (1.1.0)
aws-sdk-cognitoidentity (1.0.1)
aws-sdk-cognitoidentityprovider (1.4.0)
aws-sdk-cognitosync (1.0.1)
aws-sdk-comprehend (1.0.0)
aws-sdk-configservice (1.11.0)
aws-sdk-connect (1.0.0)
aws-sdk-core (3.21.2, 2.11.15)
aws-sdk-costandusagereportservice (1.0.1)
aws-sdk-costexplorer (1.4.0)
aws-sdk-databasemigrationservice (1.5.0)
aws-sdk-datapipeline (1.0.1)
aws-sdk-dax (1.0.1)
aws-sdk-devicefarm (1.6.0)
aws-sdk-directconnect (1.1.0)
aws-sdk-directoryservice (1.2.0)
aws-sdk-dynamodb (1.6.0)
aws-sdk-dynamodbstreams (1.0.1)
aws-sdk-ec2 (1.35.0)
aws-sdk-ecr (1.3.0)
aws-sdk-ecs (1.14.0)
aws-sdk-efs (1.0.1)
aws-sdk-eks (1.0.0)
aws-sdk-elasticache (1.3.0)
aws-sdk-elasticbeanstalk (1.6.0)
aws-sdk-elasticloadbalancing (1.2.0)
aws-sdk-elasticloadbalancingv2 (1.10.0)
aws-sdk-elasticsearchservice (1.5.0)
aws-sdk-elastictranscoder (1.0.1)
aws-sdk-emr (1.1.0)
aws-sdk-firehose (1.3.0)
aws-sdk-fms (1.0.0)
aws-sdk-gamelift (1.4.0)
aws-sdk-glacier (1.7.0)
aws-sdk-glue (1.8.0)
aws-sdk-greengrass (1.3.0)
aws-sdk-guardduty (1.4.0)
aws-sdk-health (1.0.1)
aws-sdk-iam (1.4.0)
aws-sdk-importexport (1.0.1)
aws-sdk-inspector (1.4.0)
aws-sdk-iot (1.8.0)
aws-sdk-iot1clickdevicesservice (1.0.0)
aws-sdk-iot1clickprojects (1.0.0)
aws-sdk-iotanalytics (1.0.0)
aws-sdk-iotdataplane (1.0.1)
aws-sdk-iotjobsdataplane (1.0.0)
aws-sdk-kinesis (1.2.0)
aws-sdk-kinesisanalytics (1.2.0)
aws-sdk-kinesisvideo (1.0.0)
aws-sdk-kinesisvideoarchivedmedia (1.0.0)
aws-sdk-kinesisvideomedia (1.0.0)
aws-sdk-kms (1.5.0)
aws-sdk-lambda (1.5.0)
aws-sdk-lambdapreview (1.0.1)
aws-sdk-lex (1.3.0)
aws-sdk-lexmodelbuildingservice (1.5.0)
aws-sdk-lightsail (1.4.0)
aws-sdk-machinelearning (1.0.1)
aws-sdk-marketplacecommerceanalytics (1.0.1)
aws-sdk-marketplaceentitlementservice (1.0.1)
aws-sdk-marketplacemetering (1.0.1)
aws-sdk-mediaconvert (1.2.0)
aws-sdk-medialive (1.6.0)
aws-sdk-mediapackage (1.1.0)
aws-sdk-mediastore (1.1.0)
aws-sdk-mediastoredata (1.1.0)
aws-sdk-mediatailor (1.1.0)
aws-sdk-migrationhub (1.2.0)
aws-sdk-mobile (1.0.0)
aws-sdk-mq (1.0.0)
aws-sdk-mturk (1.3.0)
aws-sdk-neptune (1.0.0)
aws-sdk-opsworks (1.2.0)
aws-sdk-opsworkscm (1.2.0)
aws-sdk-organizations (1.9.0)
aws-sdk-pi (1.0.0)
aws-sdk-pinpoint (1.4.0)
aws-sdk-polly (1.5.0)
aws-sdk-pricing (1.0.0)
aws-sdk-rds (1.21.0)
aws-sdk-redshift (1.3.0)
aws-sdk-rekognition (1.3.0)
aws-sdk-resourcegroups (1.0.0)
aws-sdk-resourcegroupstaggingapi (1.0.1)
aws-sdk-resources (3.20.0, 2.11.15)
aws-sdk-route53 (1.9.0)
aws-sdk-route53domains (1.2.0)
aws-sdk-s3 (1.13.0)
aws-sdk-sagemaker (1.9.0)
aws-sdk-sagemakerruntime (1.0.0)
aws-sdk-secretsmanager (1.7.0)
aws-sdk-serverlessapplicationrepository (1.2.0)
aws-sdk-servicecatalog (1.5.0)
aws-sdk-servicediscovery (1.2.0)
aws-sdk-ses (1.7.0)
aws-sdk-shield (1.2.0)
aws-sdk-simpledb (1.0.1)
aws-sdk-sms (1.0.1)
aws-sdk-snowball (1.2.0)
aws-sdk-sns (1.2.0)
aws-sdk-sqs (1.3.0)
aws-sdk-ssm (1.13.0)
aws-sdk-states (1.2.0)
aws-sdk-storagegateway (1.4.0)
aws-sdk-support (1.0.1)
aws-sdk-swf (1.0.1)
aws-sdk-transcribeservice (1.1.0)
aws-sdk-translate (1.1.0)
aws-sdk-waf (1.4.0)
aws-sdk-wafregional (1.4.0)
aws-sdk-workdocs (1.1.0)
aws-sdk-workmail (1.1.0)
aws-sdk-workspaces (1.2.0)
aws-sdk-xray (1.2.0)
aws-sigv2 (1.0.1)
aws-sigv4 (1.0.2)
bcrypt (3.1.11)
bigdecimal (1.2.8)
builder (3.2.3)
bundler (1.14.4)
bundler-unload (1.0.2)
business_time (0.9.3)
certifi (2016.09.26)
climate_control (0.2.0)
cocaine (0.5.8)
color-generator (0.0.4)
concurrent-ruby (1.0.5)
crass (1.0.3)
devise (4.4.2)
did_you_mean (1.0.0)
domain_name (0.5.20170404)
erubi (1.7.1)
executable-hooks (1.3.2)
faraday (0.10.1)
faraday-http-cache (1.3.1)
gem-wrappers (1.2.7)
globalid (0.4.1)
http-cookie (1.0.3)
i18n (0.9.5)
infusionsoft (1.2.1)
io-console (0.4.5)
jmespath (1.4.0, 1.3.1)
json (2.0.0, 1.8.3)
jwt (2.1.0)
little-plugger (1.1.4)
logging (2.2.2)
loofah (2.2.0)
mail (2.7.0)
mail_extract (0.1.4)
method_source (0.9.0)
mime-types (3.1)
mime-types-data (3.2016.0521)
mimemagic (0.3.2)
mini_mime (1.0.0)
mini_portile2 (2.3.0)
minitest (5.11.3, 5.8.3)
monkey-business (1.0.4)
multi_json (1.13.1)
multipart-post (2.0.0)
mustermann (1.0.2)
net-telnet (0.1.1)
netrc (0.11.0)
nio4r (2.2.0)
nokogiri (1.8.2)
orghunter (0.2.0)
orm_adapter (0.5.0)
paperclip (5.0.0)
pg (1.0.0)
power_assert (1.1.2, 0.2.6)
psych (2.0.17)
public_suffix (3.0.2)
rack (2.0.4)
rack-cors (1.0.2)
rack-protection (2.0.1)
rack-test (0.8.3)
rails (5.1.5)
rails-dom-testing (2.0.3)
rails-html-sanitizer (1.0.3)
railties (5.1.5)
rake (12.3.0, 10.4.2)
rdoc (4.2.1)
redcarpet (3.4.0)
redis (3.3.5)
responders (2.4.0)
rest-client (2.0.2)
rollbar (2.15.5)
ruby_http_client (3.3.0)
rubygems-bundler (1.4.4)
rvm (1.11.3.9)
sendgrid-ruby (5.2.0)
shoryuken (3.2.3)
silencer (1.0.1)
sinatra (2.0.1)
sprockets (3.7.1)
sprockets-rails (3.2.1)
stripe (1.58.0)
test-unit (3.1.9, 3.1.5)
thor (0.20.0)
thread_safe (0.3.6)
tilt (2.0.8)
tzinfo (1.2.5)
unf (0.1.4)
unf_ext (0.0.7.5)
validate_url (1.0.2)
warden (1.2.7)
websocket-driver (0.6.5)
websocket-extensions (0.1.3)
will_paginate (3.1.6)it has come to my attention that aws-sdk-core is another item on our list of gems that has a second installed version. I was under the impression that uninstalling the second aws-sdk version would also uninstall the sub-gems, but a quick experiment shows that is not the case. It will be some time before this makes a deploy fail, I don't know how to induce it beyond just waiting until it stops working, but I have a strong feeling that the conflict for us comes down to that aws-sdk-core gem
rohkat-aws
commented
6 years ago so things you can try @whtevn Not install aws-sdk , if you only need it for the agent .
Second if you have to install, then install the sdk first and then the agent
benlamonica
commented
6 years ago I've implemented the change suggested by @marcmillien in PR #175
benlamonica
commented
6 years ago @rohkat-aws can we get this fix merged? It affects installing codedeploy on EMR clusters as well.
rohkat-aws
commented
6 years ago @benlamonica will do it . But The changes will be rolled out in a new release of the agent .
benlamonica
commented
5 years ago @rohkat-aws will you please merge this so that I can stop hacking my start scripts? It's such a small change.
tlacroix
commented
4 years ago We're experiencing the same problem. For us, the cause that we installed the latest AWS SDK for another Ruby script. It appears that the latest CodeDeploy agent is not compatible with the latest Ruby AWS SDK.
So basically, steps to reproduce:
gem install aws-sdk2017-09-19 14:47:56 ERROR [codedeploy-agent(4396)]: booting child: error during start or run: NoMethodError - undefined method `underscore' for Seahorse::Util:Module - /opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:45:in `command'codedeploy-agent: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller of master XXXXAny chance this can be fixed? The original ticket was opened more than 2 years ago...
t0shiii
commented
1 year ago The agent source code on GitHub is for development purposes only. Please use the official AWS CodeDeploy Agent installation instructions to install CodeDeploy Agent on your hosts. Here is a link to the documentation
This should solve your problem.
OS: Amazon Linux Version: OFFICIAL_1.0-1.1231_rpm Issue: Agent will be unresponsive and unable to receive deployments. Reboot / restarting the agent does not fix. Logs:
Solution is to remove all locally installed gems via:
Uninstall the codedeploy agent, remove everything from /opt/codedeploy and reinstall.
We are using the puppet module to install which has been working great.
The one thing that I think that might be happening is our rails apps are installing a local version of the aws-sdk that is older or newer than what the agent supports. But that's just a hunch.
This issue has been randomly popping up during deployments and has been causing a great amount of grief lately. To the point we are looking to switch to blue / green just to get around it. Unfortunately that deployment strategy has it's own issues to deal with.