Fog.mock! not working with Storage provider

[llebleis]

Hello, I'm facing an issue with Fog.mock! not mocking s3 commands. To reproduce the issue:

Gemfile is:

ruby '~> 3.0.4'

gem 'fog-aws'

gem 'mime-types'
gem 'dotenv'

report.rb :

require 'dotenv/load'
require 'fog-aws'

Fog.mock!

aws = Fog::Storage.new(provider: 'aws', aws_access_key_id: ENV['AWS_S3_ACCESS_KEY'],
                            aws_secret_access_key: ENV['AWS_S3_SECRET_ACCESS_KEY'],
                            region: ENV['AWS_S3_REGION'])

bucket_name = ENV['AWS_S3_BUCKET']

bucket = aws.directories.new(key: bucket_name)
path = 'test_bucket/test_file.txt'
file = bucket.files.create({ key: path, body: 'test_body', public: true })
content = bucket.files.get(path)

if content
  puts "File content is #{content.body}"
else
  puts "no content"
end

Output of bundle exec ruby report.rb is:

/vendor/bundle/ruby/3.0.0/gems/fog-aws-3.23.0/lib/fog/aws/requests/storage/shared_mock_methods.rb:31:in `verify_mock_bucket_exists': Expected(200) <=> Actual(404 Not Found) (Excon::Error::NotFound)
    from /Users/loic/Dev/koalect/src/test_fog/vendor/bundle/ruby/3.0.0/gems/fog-aws-3.23.0/lib/fog/aws/requests/storage/put_object.rb:70:in `put_object'
    from /Users/loic/Dev/koalect/src/test_fog/vendor/bundle/ruby/3.0.0/gems/fog-aws-3.23.0/lib/fog/aws/models/storage/file.rb:283:in `save'
    from /Users/loic/Dev/koalect/src/test_fog/vendor/bundle/ruby/3.0.0/gems/fog-core-2.4.0/lib/fog/core/collection.rb:53:in `create'
    from report.rb:14:in `<main>'

(line 14 is file = bucket.files.create({ key: path, body: 'test_body', public: true }))

It runs smooth without the Fog.mock!

Any idea how to fix that ?

Thanks, and thanks for the good job on that gem. Loïc

[geemus]

@llebleis Hello. Looking at the code you shared, I think it comes down to the way you are addressing the bucket. aws.directories.new will create a bucket object but it won't actually persist it (to mocks or otherwise). If you change the new to create or else call save on the bucket before trying to create the file I think you should have better luck.

[llebleis]

Thanks @geemus for the quick answer. When using aws.directories.create it works perfectly with Fog.mock! but no more without. To make it work I added a '/' character at the end of the bucket_name and now I've got warning from Fog:

[fog][WARNING] fog: the specified s3 bucket name(bucket_name/) is not a valid dns name, which will negatively impact performance.  For details see: http://docs.amazonwebservices.com/AmazonS3/latest/dev/BucketRestrictions.html

But I think I can live with these warning.

Thanks then I will close that "non" issue

[geemus]

@llebleis - Ah, yeah in real usage you would probably create the bucket out-of-band and then just refer to it with new here (instead of creating it each time). You could make it work in both by first checking to see if the bucket exists before trying to create it, but that might be more complicated that really want (and it incurs a performance impact to check for the bucket each time).

That warning has to do with whether or not the bucket_name can be used as a subdomain (which something with / cannot because it's not valid in that context). I don't think that the / should be needed in general. Adding it here would create a new bucket in real usage (instead of using the existing bucket, where it would error because it already exists). I'm guessing that isn't really your intention, so I would probably take that back out and delete the new bucket if you don't need it. Then you can either presume the bucket exists in real usage or do the check for it's existence I mentioned above.

Glad to hear that helped, just let me know if you have any further questions.