API Implement cache / move to silverstripe vendor

[tractorcow]

See https://github.com/silverstripe/silverstripe-assets/issues/43 for parent story

Still need to write unit tests, but this can be peer reviewed in the mean time

[sminnee]

What's the best way to set up an integration test rig for this? Does it just need an S3 bucket?

[tractorcow]

I was thinking about using a mock for the s3 client, actually.

[sminnee]

Yeah then it wouldn't be an integration test, it'd be a component test.

With a vanilla S3 bucket / subfolder, would it be feasible to do an integration test of this with the substantial caveat that protected assets were publicly accessible?

[tractorcow]

In order to test this out:

• Setup your s3 bucket as per readme.md. (public / protected folders)

  {
  "Version": "2012-10-17",
  "Id": "AccessForPublicAssetsAndPassthruForProtectedAssets",
  "Statement": [
      {
          "Sid": "AllowAccessToPublicAssets",
          "Effect": "Allow",
          "Principal": "*",
          "Action": "s3:GetObject",
          "Resource": "arn:aws:s3:::<bucket-name>/public/*"
      },
      {
          "Sid": "ServerFullAccess",
          "Effect": "Allow",
          "Principal": {"AWS":"<your-ARN>"},
          "Action": ["s3:GetObject", "s3:DeleteObject", "s3:PutObject"],
          "Resource": "arn:aws:s3:::<bucket-name>/*"
      }
  ]
  }

• Set your .env with:

  AWS_REGION="ap-southeast-2"
  AWS_BUCKET_NAME="silverstripe-solutions-ss4test"
  AWS_ACCESS_KEY_ID="<redacted>"
  AWS_SECRET_ACCESS_KEY="<redacted>"

Then flush, and that should be sufficient to get this installed and working.

When testing this PR it may fail to autoload psr-4 due to the updated namespace and psr-4 root. you will need to set a custom repository and install from composer that way.

{
    "require": {
        "silverstripe/s3": "dev-pulls/1.0/cache-me-riding-dirty"
    },
    "repositories": {
        "open-sausages": {
            "type": "vcs",
            "url": "git@github.com:open-sausages/silverstripe-s3.git"
        }
    }
}

[sminnee]

Edge-case issue: if you don't have write permission on the bucket, the error isn't helpful https://gist.github.com/sminnee/24f5831a0aa87c18b12d5918ea04aad6 - it would be better if it said "The webserver doesn't have permission to write the file to S3" or something in the response.

[sminnee]

OK — I've got this working but there's a bug. Draft assets are using the public asset URLs for their thumbnails, which don't yet exist and so throw "forbidden" errors.

[tractorcow]

Yes, something broken recently. Looking at this bug now.

[tractorcow]

I've traced this to a bug where exists() would use getMetadata(), and rely on potentially cached data which never actually calls the backend. We need a concept for agnostic-exists (will invoke a cache miss).

[tractorcow]

Updated with tests @sminnee

The public url bug is fixed.

[tractorcow]

Tests green. :P

[sminnee]

I've done a local test and it works for me now and the code looks pretty good.