search

sous-chefs / aws

Development repository for the aws cookbook
https://supermarket.chef.io/cookbooks/aws
Apache License 2.0
610 stars 554 forks source link

"unexpected value at params[:expires_in]" error when using aws_s3_file resource #413

Closed NelsonTang19 closed 2 years ago

NelsonTang19 commented 4 years ago

Cookbook version 8.2.0

Chef-client version 16.1.0

Platform Details Ubuntu 18.04.4 LTS 5.3.0-1030-aws

Scenario: Trying to download a S3 file using aws_s3_file

Steps to Reproduce: In test cookbook, add dependency to aws cookbook

name              'test-aws-cookbook'
depends 'aws', '~> 8.0'

Add aws_s3_file resource to default recipe (bucket and remote_path values redacted):

aws_s3_file "#{Chef::Config[:file_cache_path]}/test" do
  remote_path "test"
  bucket 'test-bucket'
end

Expected Result: test file in test-bucket is downloaded

Actual Result:

This particular output was with cinc client (a fork of chef). (Bucket and file names redacted)

Recipe: test-aws-cookbook::default
  * aws_s3_file[/root/.cinc/local-mode-cache/cache/test] action create

    ================================================================================
    Error executing action `create` on resource 'aws_s3_file[/root/.cinc/local-mode-cache/cache/test]'
    ================================================================================

    ArgumentError
    -------------
    unexpected value at params[:expires_in]

    Cookbook Trace:
    ---------------
    /root/.cinc/local-mode-cache/cache/cookbooks/aws/resources/s3_file.rb:138:in `s3_url'
    /root/.cinc/local-mode-cache/cache/cookbooks/aws/resources/s3_file.rb:157:in `block in do_s3_file'
    /root/.cinc/local-mode-cache/cache/cookbooks/aws/resources/s3_file.rb:155:in `do_s3_file'
    /root/.cinc/local-mode-cache/cache/cookbooks/aws/resources/s3_file.rb:40:in `block in class_from_file'

    Resource Declaration:
    ---------------------
    # In /root/.cinc/local-mode-cache/cache/cookbooks/test-aws-cookbook/recipes/default.rb

     10: aws_s3_file "#{Chef::Config[:file_cache_path]}/test" do
     11:   remote_path "test"
     12:   bucket 'test-bucket'
     13: end
     14: 

    Compiled Resource:
    ------------------
    # Declared in /root/.cinc/local-mode-cache/cache/cookbooks/test-aws-cookbook/recipes/default.rb:10:in `from_file'

    aws_s3_file("/root/.cinc/local-mode-cache/cache/test") do
      action [:create]
      default_guard_interpreter :default
      declared_type :aws_s3_file
      cookbook_name "test-aws-cookbook"
      recipe_name "default"
      remote_path "test"
      bucket "test-bucket"
      region "eu-west-1"
      path "/root/.cinc/local-mode-cache/cache/test"
    end

    System Info:
    ------------
    chef_version=16.1.0
    platform=ubuntu
    platform_version=18.04
    ruby=ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux]
    program_name=/opt/cinc/bin/cinc-client
    executable=/opt/cinc/bin/cinc-client

Running handlers:
[2020-07-23T09:10:43+00:00] ERROR: Running exception handlers
Running handlers complete
[2020-07-23T09:10:43+00:00] ERROR: Exception handlers complete
Cinc Client failed. 0 resources updated in 05 seconds
[2020-07-23T09:10:43+00:00] FATAL: Stacktrace dumped to /root/.cinc/local-mode-cache/cache/cinc-stacktrace.out
[2020-07-23T09:10:43+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2020-07-23T09:10:43+00:00] FATAL: ArgumentError: aws_s3_file[/root/.cinc/local-mode-cache/cache/test] (test-aws-cookbook::default line 10) had an error: ArgumentError: unexpected value at params[:expires_in]

Stacktrace output at /root/.cinc/local-mode-cache/cache/cinc-stacktrace.out:

ArgumentError: aws_s3_file[/root/.cinc/local-mode-cache/cache/test] (test-aws-cookbook::default line 10) had an error: ArgumentError: unexpected value at params[:expires_in]
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.104.2/lib/aws-sdk-core/param_validator.rb:35:in `validate!'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.104.2/lib/aws-sdk-core/param_validator.rb:15:in `validate!'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.104.2/lib/aws-sdk-core/plugins/param_validator.rb:25:in `call'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.104.2/lib/seahorse/client/plugins/raise_response_errors.rb:16:in `call'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/aws-sdk-s3-1.38.0/lib/aws-sdk-s3/plugins/sse_cpk.rb:22:in `call'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/aws-sdk-s3-1.38.0/lib/aws-sdk-s3/plugins/dualstack.rb:26:in `call'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/aws-sdk-s3-1.38.0/lib/aws-sdk-s3/plugins/accelerate.rb:35:in `call'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.104.2/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:22:in `call'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.104.2/lib/aws-sdk-core/plugins/idempotency_token.rb:19:in `call'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.104.2/lib/aws-sdk-core/plugins/param_converter.rb:26:in `call'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.104.2/lib/seahorse/client/plugins/request_callback.rb:71:in `call'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.104.2/lib/aws-sdk-core/plugins/response_paging.rb:12:in `call'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.104.2/lib/seahorse/client/plugins/response_target.rb:24:in `call'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.104.2/lib/seahorse/client/request.rb:72:in `send_request'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/aws-sdk-s3-1.38.0/lib/aws-sdk-s3/presigner.rb:57:in `presigned_url'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/aws-sdk-s3-1.38.0/lib/aws-sdk-s3/customizations/object.rb:189:in `presigned_url'
/root/.cinc/local-mode-cache/cache/cookbooks/aws/resources/s3_file.rb:138:in `s3_url'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/resource.rb:1311:in `method_missing'
/root/.cinc/local-mode-cache/cache/cookbooks/aws/resources/s3_file.rb:157:in `block in do_s3_file'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/resource_builder.rb:67:in `instance_eval'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/resource_builder.rb:67:in `build'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/dsl/declare_resource.rb:318:in `build_resource'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/dsl/declare_resource.rb:273:in `declare_resource'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/dsl/resources.rb:36:in `remote_file'
/root/.cinc/local-mode-cache/cache/cookbooks/aws/resources/s3_file.rb:155:in `do_s3_file'
/root/.cinc/local-mode-cache/cache/cookbooks/aws/resources/s3_file.rb:40:in `block in class_from_file'
(eval):2:in `block in action_create'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/provider.rb:279:in `instance_eval'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/provider.rb:279:in `compile_and_converge_action'
(eval):2:in `action_create'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/provider.rb:220:in `run_action'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/resource.rb:591:in `run_action'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/runner.rb:74:in `run_action'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/runner.rb:108:in `block in run_all_actions'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/runner.rb:108:in `each'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/runner.rb:108:in `run_all_actions'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/runner.rb:132:in `block in converge'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/resource_collection/resource_list.rb:96:in `block in execute_each_resource'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/resource_collection/stepable_iterator.rb:115:in `call_iterator_block'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/resource_collection/stepable_iterator.rb:86:in `step'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/resource_collection/resource_list.rb:94:in `execute_each_resource'
/opt/cinc/embedded/lib/ruby/2.7.0/forwardable.rb:235:in `execute_each_resource'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/runner.rb:130:in `converge'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/client.rb:714:in `block in converge'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/client.rb:709:in `catch'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/client.rb:709:in `converge'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/client.rb:733:in `converge_and_save'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/client.rb:288:in `run'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/application.rb:292:in `run_with_graceful_exit_option'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/application.rb:268:in `block in run_chef_client'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/local_mode.rb:42:in `with_server_connectivity'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/application.rb:251:in `run_chef_client'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/application/base.rb:335:in `run_application'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-16.1.0/lib/chef/application.rb:68:in `run'
/opt/cinc/embedded/lib/ruby/gems/2.7.0/gems/chef-bin-16.1.0/bin/cinc-client:25:in `<top (required)>'
/opt/cinc/bin/cinc-client:171:in `load'
/opt/cinc/bin/cinc-client:171:in `<main>'

Upgrading the aws-sdk-s3 gem to version 1.63.0 appeared to address this:

chef_gem 'aws-sdk-s3' do
  version 1.63.0
  action :install
end
majormoses commented 4 years ago

If you would like to open a pull request to update the deps that is always appreciated: https://github.com/chef-cookbooks/aws/blob/master/metadata.rb#L16-L35

NelsonTang19 commented 2 years ago

Appears to be resolved since commit released in v8.3.1: https://github.com/sous-chefs/aws/commit/ae36b0aa053117c10e8e8b3130cd9874b63cff30#diff-eea7c7f5e64dcee356a42e1162f4817fdae0ed799f69aba8c61ff842efdcd857