aws / aws-codedeploy-agent

Host Agent for AWS CodeDeploy
https://aws.amazon.com/codedeploy
Apache License 2.0
328 stars 188 forks source link

Permission denied @ rb_sysopen #395

Open a-jackson opened 3 months ago

a-jackson commented 3 months ago

We're periodically getting failed deployments with Permission denied @ rb_sysopen. This is on a new instance following an autoscaling action.

Code Deploy agent 1.7.0.92_msi OS: Windows Server 2019

The agent is installed in the AMI and has worked on other instances of the same AMI with the same deployment so it seems to be some transient glitch. In response to this error the instance was automatically terminated and a new one created which worked. Full logs below

CodeDeploy logs

``` 2024-05-28T07:33:46 WARN [codedeploy-agent(1404)]: Received Errno::EACCESS when creating deployment tracking file, retrying creation 2024-05-28T07:33:46 WARN [codedeploy-agent(1404)]: Permission denied @ rb_sysopen - C:\ProgramData/Amazon/CodeDeploy/ongoing-deployment/d-05MLR7346 2024-05-28T07:33:47 INFO [codedeploy-agent(1404)]: Version file found in C:/ProgramData/Amazon/CodeDeploy/.version with agent version OFFICIAL_1.7.0.92_msi. 2024-05-28T07:33:47 WARN [codedeploy-agent(1404)]: Received Errno::EACCESS when creating deployment tracking file, retrying creation 2024-05-28T07:33:47 WARN [codedeploy-agent(1404)]: Permission denied @ rb_sysopen - C:\ProgramData/Amazon/CodeDeploy/ongoing-deployment/d-05MLR7346 2024-05-28T07:33:49 WARN [codedeploy-agent(1404)]: Received Errno::EACCESS when creating deployment tracking file, retrying creation 2024-05-28T07:33:49 WARN [codedeploy-agent(1404)]: Permission denied @ rb_sysopen - C:\ProgramData/Amazon/CodeDeploy/ongoing-deployment/d-05MLR7346 2024-05-28T07:33:54 WARN [codedeploy-agent(1404)]: Received Errno::EACCESS when creating deployment tracking file, retrying creation 2024-05-28T07:33:54 WARN [codedeploy-agent(1404)]: Permission denied @ rb_sysopen - C:\ProgramData/Amazon/CodeDeploy/ongoing-deployment/d-05MLR7346 2024-05-28T07:33:54 ERROR [codedeploy-agent(1404)]: Exhausted retries on creating tracking file, rethrowing exception 2024-05-28T07:33:54 INFO [codedeploy-agent(1404)]: Version file found in C:/ProgramData/Amazon/CodeDeploy/.version with agent version OFFICIAL_1.7.0.92_msi. 2024-05-28T07:33:54 INFO [codedeploy-agent(1404)]: [Aws::CodeDeployCommand::Client 200 0.134703 0 retries] put_host_command_complete(command_status:"Failed",diagnostics:{format:"JSON",payload:"{\"error_code\":5,\"script_name\":\"\",\"message\":\"Permission denied @ rb_sysopen - C:\\\\ProgramData/Amazon/CodeDeploy/ongoing-deployment/d-05MLR7346\",\"log\":\"\"}"},host_command_identifier:"redacted") 2024-05-28T07:33:54 ERROR [codedeploy-agent(1404)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller: Error during perform: Errno::EACCES - Permission denied @ rb_sysopen - C:\ProgramData/Amazon/CodeDeploy/ongoing-deployment/d-05MLR7346 - C:/ProgramData/Amazon/CodeDeploy/bin/ocr4E59.tmp/src/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/deployment_command_tracker.rb:22:in `write' C:/ProgramData/Amazon/CodeDeploy/bin/ocr4E59.tmp/src/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/deployment_command_tracker.rb:22:in `create_ongoing_deployment_tracking_file' C:/ProgramData/Amazon/CodeDeploy/bin/ocr4E59.tmp/src/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_poller.rb:135:in `process_command' C:/ProgramData/Amazon/CodeDeploy/bin/ocr4E59.tmp/src/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_poller.rb:119:in `acknowledge_and_process_command' C:/ProgramData/Amazon/CodeDeploy/bin/ocr4E59.tmp/src/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_poller.rb:99:in `block in perform' C:/ProgramData/Amazon/CodeDeploy/bin/ocr4E59.tmp/gemhome/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:363:in `run_task' C:/ProgramData/Amazon/CodeDeploy/bin/ocr4E59.tmp/gemhome/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:352:in `block (3 levels) in create_worker' C:/ProgramData/Amazon/CodeDeploy/bin/ocr4E59.tmp/gemhome/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:335:in `loop' C:/ProgramData/Amazon/CodeDeploy/bin/ocr4E59.tmp/gemhome/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:335:in `block (2 levels) in create_worker' C:/ProgramData/Amazon/CodeDeploy/bin/ocr4E59.tmp/gemhome/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:334:in `catch' C:/ProgramData/Amazon/CodeDeploy/bin/ocr4E59.tmp/gemhome/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:334:in `block in create_worker' **C**:/ProgramData/Amazon/CodeDeploy/bin/ocr4E59.tmp/gemhome/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context' ```