umbraco-community / UmbracoFileSystemProviders.Azure

:cloud: An Azure Blob Storage IFileSystem provider for Umbraco
94 stars 67 forks source link

Question - expected behaviour with UmbracoFileSystemProviders.Azure + AzureBlobCache + cdn #140

Closed AstuteMediaDev closed 5 years ago

AstuteMediaDev commented 5 years ago

Hi guys please excuse me if this is documented somewhere. I can't find a detailed explanation of how these three component work together and the behaviour we're seeing after set up doesn't seem right.

Environment

UmbracoCms 7.5.14 ImageProcessor 2.7.0.100 ImageProcessor.Web 4.10.0.100 ImageProcessor.Web.Config 2.5.0.100 ImageProcessor.Web.Plugins.AzureBlobCache 1.5.0.100 UmbracoFileSystemProviders.Azure 1.0.3 Azure storage account (Standard/Hot, GRS, StorageV2 (general purpose v2), container access level Blob) Azure cdn - Standard Microsoft (cache query strings true)

Configuration files (see end)

Observed behaviour

Our requirements are basic. We have an international site and need to serve mostly images and a few large mp4 files from cdn.

When we request a page with a full range of assets we see this behaviour:

  1. Images with IP query string return 302 to the cdn. We see caching occur in the 'cache' blob container. OK makes sense.

  2. Images without IP query string return 200. Unsure this is correct. Should these not also 302 to the cdn?

  3. mp4 files are range requested and return 206 but are never cached, they are downloaded on each page load. This can't be correct and defeats the point of the cdn. We previously used a web.config clientCache setting when serving straight from IIS and the mp4 file were cached and loaded from disk on the client.

TIA it's quite confusing for a newcomer.

Configuration files

security.config

<?xml version="1.0" encoding="utf-8"?>
<security>
  <services>
    <!--Disable the LocalFileImageService and enable this one when using virtual paths. -->
    <!--<service name="CloudImageService" type="ImageProcessor.Web.Services.CloudImageService, ImageProcessor.Web">
      <settings>
        <setting key="Container" value=""/>
        <setting key="MaxBytes" value="8194304"/>
        <setting key="Timeout" value="30000"/>
        <setting key="Host" value="http://yourhost.com/"/>
      </settings>
    </service>-->
    <service prefix="media/" name="CloudImageService" type="ImageProcessor.Web.Services.CloudImageService, ImageProcessor.Web">
      <settings>
        <setting key="Container" value="media" />
        <setting key="MaxBytes" value="8194304" />
        <setting key="Timeout" value="30000" />
        <setting key="Host" value="http://xxx.blob.core.windows.net/" />
      </settings>
    </service>
    <service name="LocalFileImageService" type="ImageProcessor.Web.Services.LocalFileImageService, ImageProcessor.Web" />
    <service prefix="remote.axd" name="RemoteImageService" type="ImageProcessor.Web.Services.RemoteImageService, ImageProcessor.Web">
      <settings>
        <setting key="MaxBytes" value="4194304" />
        <setting key="Timeout" value="3000" />
        <setting key="Protocol" value="http" />
      </settings>
      <whitelist>
      </whitelist>
    </service>
  <service name="AzureImageService" type="ImageProcessor.Web.Plugins.AzureBlobCache.AzureImageService, ImageProcessor.Web.Plugins.AzureBlobCache">
      <settings>
        <setting key="StorageAccount" value="DefaultEndpointsProtocol=https;AccountName=xxx;AccountKey=xxx" />
        <setting key="Container" value="cache" />
        <setting key="AccessType" value="Blob" />
      </settings>
    </service>
  </services>
</security>

cache.config

<?xml version="1.0" encoding="utf-8"?>
<caching currentCache="AzureBlobCache">
  <caches>
    <cache trimCache="false" name="DiskCache" type="ImageProcessor.Web.Caching.DiskCache, ImageProcessor.Web" maxDays="365" memoryMaxMinutes="1" browserMaxDays="7">
      <settings>
        <setting key="VirtualCachePath" value="~/app_data/cache" />
      </settings>
    </cache>
    <cache name="AzureBlobCache" type="ImageProcessor.Web.Plugins.AzureBlobCache.AzureBlobCache, ImageProcessor.Web.Plugins.AzureBlobCache" maxDays="365">
      <settings>
        <setting key="CachedStorageAccount" value="DefaultEndpointsProtocol=https;AccountName=xxx;AccountKey=xxx" />
        <setting key="CachedBlobContainer" value="cache" />
        <setting key="UseCachedContainerInUrl" value="true" />
        <setting key="CachedCDNRoot" value="https://xxx.azureedge.net" />
        <setting key="CachedCDNTimeout" value="10000" />
        <setting key="SourceStorageAccount" value="" />
        <setting key="SourceBlobContainer" value="" />
        <setting key="StreamCachedImage" value="false" />
      </settings>
    </cache>
  </caches>
</caching>

filesystemproviders.config

<?xml version="1.0"?>
<FileSystemProviders>

  <!-- Media -->
  <Provider alias="media" type="Our.Umbraco.FileSystemProviders.Azure.AzureBlobFileSystem, Our.Umbraco.FileSystemProviders.Azure">
    <Parameters>
        <add key="containerName" value="media"/>
        <add key="rootUrl" value="http://xxx.blob.core.windows.net/"/>
        <add key="connectionString"
            value="DefaultEndpointsProtocol=https;AccountName=xxx;AccountKey=xxx;BlobEndpoint=https://xxx.blob.core.windows.net/;QueueEndpoint=https://xxx.queue.core.windows.net/;TableEndpoint=https://xxx.table.core.windows.net/;FileEndpoint=https://xxx.file.core.windows.net/;"/>
        <!--
        Optional configuration value determining the maximum number of days to cache items in the browser.
        Defaults to 365 days.
      -->
        <add key="maxDays" value="365"/>
        <!--
        When true this allows the VirtualPathProvider to use the deafult "media" route prefix regardless 
        of the container name.
      -->
        <add key="useDefaultRoute" value="true"/>
        <!--
        When true blob containers will be private instead of public what means that you can't access the original blob file directly from its blob url.
      -->
        <add key="usePrivateContainer" value="false"/>
    </Parameters>
  </Provider>

</FileSystemProviders>
JimBobSquarePants commented 5 years ago
  1. This is expected. You have given IP a command to process and cache.
  2. This is expected. You have not given IP a command so it does not have an image to process nor cache.
  3. This is expected. Mp4s are not images so ImageProcessor will ignore the requests.

This library does not contain a cache, hence why there is no documentation on caching. It implements the IFileSystem interface from Umbraco, nothing more.

AstuteMediaDev commented 5 years ago

FYI for any other confused newcomers see the AzureCDNToolKit by Jeavon:

https://github.com/CrumpledDog/Umbraco-AzureCDNToolkit

It builds on the aforementioned packages and provides extension methods to render cdn urls.