silverstripe / silverstripe-s3

Silverstripe module to store assets in S3 rather than on the local filesystem (SS4/SS5 only)
BSD 3-Clause "New" or "Revised" License
20 stars 25 forks source link

Horrible performance on Draft files. #67

Closed theTigerDuck closed 2 months ago

theTigerDuck commented 3 months ago

Even using jgivoni\Flysystem\Cache\CacheAdapter for every draft file gets at least 20 times called "doesObjectExistV2" plus more for its variants. I got rid of the Hashfolder by doing this:

---
Name: assetsflysystemfix
After:
  - "#assetsflysystem"
---
SilverStripe\Core\Injector\Injector:
  # Define protected resolution strategy
  SilverStripe\Assets\FilenameParsing\FileResolutionStrategy.protected:
    class: SilverStripe\Assets\FilenameParsing\FileIDHelperResolutionStrategy
    properties:
      DefaultFileIDHelper: '%$SilverStripe\Assets\FilenameParsing\NaturalFileIDHelper'
      ResolutionFileIDHelpers:
        - '%$SilverStripe\Assets\FilenameParsing\NaturalFileIDHelper'
      VersionedStage: Stage

otherwise it's even more calls of "doesObjectExistV2" because it fist looks for the file without the hash.

wilr commented 2 months ago

HI @theTigerDuck can you test this PR and see if you get better performance? https://github.com/silverstripe/silverstripe-s3/pull/70

theTigerDuck commented 4 weeks ago

Hi @wilr,

thanks for the grat work. Unfortunately I hadn't time to test this. Now that it's merged I installed it and there are still some issues.

First of all PublicCDNAdapter does not get used. Instead of:

---
Name: silverstripes3-cdn-custom
Only:
  envvarset: AWS_PUBLIC_CDN_PREFIX
After:
  - '#silverstripes3-cdn'
---
SilverStripe\Core\Injector\Injector:
  SilverStripe\S3\Adapter\PublicAdapter:
    class: SilverStripe\S3\Adapter\PublicCDNAdapter
    constructor:
      s3Client: '%$Aws\S3\S3Client'
      bucket: '`AWS_BUCKET_NAME`'
      prefix: '`AWS_PUBLIC_BUCKET_PREFIX`'
      visibility: null
      mimeTypeDetector: null
      cdnPrefix: '`AWS_PUBLIC_CDN_PREFIX`'
      options: []
      cdnAssetsDir: '`AWS_PUBLIC_BUCKET_PREFIX`'

I had to change it to:

---
Name: silverstripes3-cdn-custom
Only:
  envvarset: AWS_PUBLIC_CDN_PREFIX
After:
  - "#assetsflysystem"
  - "#silverstripes3-flysystem"
  - "#silverstripes3-cdn"
---
SilverStripe\Core\Injector\Injector:
  SilverStripe\Assets\Flysystem\PublicAdapter:
    class: SilverStripe\S3\Adapter\PublicCDNAdapter
    constructor:
      s3Client: '%$Aws\S3\S3Client'
      bucket: "`AWS_BUCKET_NAME`"
      prefix: "`AWS_PUBLIC_BUCKET_PREFIX`"
      visibility: null
      mimeTypeDetector: null
      cdnPrefix: "`AWS_PUBLIC_CDN_PREFIX`"
      options: []
      cdnAssetsDir: "`AWS_PUBLIC_BUCKET_PREFIX`"

Whenever I puplish a file. It gives me "File could not be found.". Physically it's there in the public folder. But in the AssetAdmin it's not until I call ?flush=all manually.

Because of privious performance issius I had a file extension, that autopublishes everything on upload. public function onAfterUpload(){ $this->getOwner()->publishSingle(); } this leads to errors un upload because for some reason readFiles is called on the protected folder afer file is allready located in public. It's probably the same reason why I get "File could not be found."

Do you have any suggestions? I would love to here from you.