Enhanced configuration

[asiffermann]

Disclaimer: This PR introduces several breaking changes!

• Strongly typed configuration for providers and scopes
• Proper exceptions from StorageFactory (fix #36)
• New Configuration (detailed below, fix #42 and way more!)
• ConnectionStringName parameter mapping: you can use the override of AddStorage that takes a IConfigurationRoot as parameter, and the connection strings will be resolved by name
• ScopedStores
• Initialization method on IStore (fix #38)
• Enhance Access Level for GeekLearning.Storage.FileSystem.Server (fix #47)

New Configuration

{
  "ConnectionStrings": {
    "ConnectionStringFromAppSettings": "DefaultEndpointsProtocol=https;AccountName=<YourAccount>;AccountKey=<YourKey>;EndpointSuffix=core.windows.net"
  },

  "Storage": {

    "Providers": {
      "FirstAzure": {
        "Type": "Azure",
        "ConnectionString": "DefaultEndpointsProtocol=https;AccountName=<YourAccount>;AccountKey=<YourKey>;EndpointSuffix=core.windows.net"
      },
      "AnotherAzure": {
        "Type": "Azure",
        "ConnectionStringName": "ConnectionStringFromAppSettings"
      },
      "FirstFileSystem": {
        "Type": "FileSystem"
      },
      "AnotherFileSystem": {
        "Type": "FileSystem",
        "RootPath": "../FileVault2"
      }
    },

    "Stores": {
      "Store1": {
        "ProviderName": "FirstFileSystem"
      },
      "Store2": {
        "ProviderName": "FirstFileSystem",
        "AccessLevel": "Public",
        "FolderName": "AnotherPath"
      },
      "Store3": {
        "ProviderName": "FirstAzure",
        "AccessLevel": "Private"
      },
      "Store4": {
        "ProviderType": "Azure",
        "ConnectionString": "DefaultEndpointsProtocol=https;AccountName=<YourAccount>;AccountKey=<YourKey>;EndpointSuffix=core.windows.net"
      },
      "Store5": {
        "ProviderName": "AnotherAzure"
      },
      "Store6": {
        "ProviderType": "Azure",
        "ConnectionStringName": "ConnectionStringFromAppSettings"
      }
    },

    "ScopedStores": {
      "ScopedStore1": {
        "ProviderName": "AnotherFileSystem",
        "FolderNameFormat": "AnotherPath-{0}"
      },
      "ScopedStore2": {
        "ProviderName": "AnotherAzure",
        "AccessLevel": "Confidential",
        "FolderNameFormat": "AnotherPath-{0}"
      }
    }
  }
}

Global:

• Provider-level configuration available
• Flattened store-level configuration (aka no more 'Parameters')
• Mandatory keys for a Store:
  • Either ProviderName
  • Or ProviderType and provider's configuration keys
• Container and Path merged into one single configuration key: FolderName. If not specified, the name of the Store is taken.
• AccessLevel: used to define the Azure container public access level, or if the Store may be exposed by the FileSystem server. Available values are:
  • Public: map to BlobContainerPublicAccessType.Container, can list store and read files
  • Confidential: map to BlobContainerPublicAccessType.Blob, can read files
  • Private (default): map to BlobContainerPublicAccessType.Off, no access

ScopeStores:

• FolderNameFormat is mandatory to initialize a Store with params (for example one store per user)
• All other Stores' parameters are available

FileSystem:

• RootPath can now be set from configuration at provider-level

Azure:

• ConnectionString is now defined at provider-level
• It can be replaced with ConnectionStringName if we call an override to AddAzureStorage which take a configuration section to pass the default ConnectionStrings section of the appsettings.json
• At Store-level, if we do not want to reference a named provider, we can either set ConnectionStringName or ConnectionString

[asiffermann]

Here is the configuration we have today:

"Storage": {
  "Stores": {
    "Youpi1": {
      "Provider": "FileSystem",
      "Parameters": {
        "Path": "Youpi1"
      }
    },
    "Youpi2": {
      "Provider": "FileSystem",
      "Parameters": {
        "Path": "AnotherPath",
        "Access": "Public"
      }
    },
    "Youpi3": {
      "Provider": "FileSystem",
      "Parameters": {
        "Path": "Youpi3"
      }
    },
    "Youpi4": {
      "Provider": "Azure",
      "Parameters": {
        "ConnectionString": "",
        "Container": "Youpi4"
      }
    },
    "Youpa": {
      "Provider": "Azure",
      "Parameters": {
        "ConnectionString": "",
        "Container": "Youpa"
      }
    }
  }
}

And here is the configuration I'd like to have:

"Storage": {
  "Providers": {
    "Azure": {
      "ConnectionStrings": {
        "DefaultConnection": "",
        "AnotherConnection": ""
      },
      "DefaultConnectionStringName": "DefaultConnection"
    },
    "FileSystem": {
      "RootPath": ""
    }
  },
  "Stores": {
    "Youpi1": {
      "Provider": "FileSystem"
    },
    "Youpi2": {
      "Provider": "FileSystem",
      "AccessLevel": "Public",
      "FolderName": "AnotherPath"
    },
    "Youpi3": {
      "Provider": "FileSystem",
      "FolderNameFormat": "AnotherPath-{0}"
    },
    "Youpi4": {
      "Provider": "Azure"
    },
    "Youpa": {
      "Provider": "Azure",
      "AccessLevel": "Confidential",
      "FolderNameFormat": "Youpa-{0}",
      "ConnectionStringName": "AnotherConnection"
    }
    "Youpa2": {
      "Provider": "Azure",
      "ConnectionString": ""
    }
  }
}

Global:

• Provider-level configuration available
• Flattened store-level configuration (aka no more 'Parameters')
• Only one key is mandatory for a Store: Provider
• Container and Path merged into one single configuration key: Folder. If not specified, the name of the Store is taken.
• FolderNameFormat can be used instead to initialize a Store with params (for example one store per user)
• AccessLevel: used to define the Azure container public access level, or if the Store may be exposed by the FileSystem server. Available values are:
  • Public: map to BlobContainerPublicAccessType.Container, can list store and read files
  • Confidential: map to BlobContainerPublicAccessType.Blob, can read files
  • Private (default): map to BlobContainerPublicAccessType.Off, no access

FileSystem:

• RootPath can now be set from configuration at provider-level too

Azure:

• ConnectionStrings are defined at provider-level, and we referenced them by their names. Plus, there will be an override to AddAzureStorage which will take a configuration section to pass the default ConnectionStrings section of the appsettings.json if we want to share some with other components.
• DefaultConnectionStringName at provider-level will be applied to every container which does not override the settings
• At Store-level, we can either set ConnectionStringName to reference a connection string put in the global list or a ConnectionString to set a very specific one to this store

What do you think @sandorfr? 😃

[sandorfr]

I like it very much, Just wondering how it would impact IStore<TOptions>

[sandorfr]

Be carefull with you azure sample as container names have limitations on their names. Provisionning and AccessLevel feature is ❤️

[asiffermann]

After talking with @annayafi, I changed a little bit some options, introduced Named Providers, and extract the ScopedStores configurations in another key:

"Storage": {

  "Providers": {
    "FirstAzure": {
      "Type": "Azure",
      "ConnectionString": "DefaultEndpointsProtocol=https;AccountName=default;AccountKey=<key>;EndpointSuffix=core.windows.net"
    },
    "AnotherAzure": {
      "Type": "Azure",
      "ConnectionStringName": "ConnectionStringFromAppSettings"
    },
    "FirstFileSystem": {
      "Type": "FileSystem",
      "RootPath": "C:/First"
    },
    "AnotherFileSystem": {
      "Type": "FileSystem",
      "RootPath": "D:/Another"
    }
  },

  "Stores": {
    "Youpi1": {
      "ProviderName": "FirstFileSystem"
    },
    "Youpi2": {
      "ProviderName": "FirstFileSystem",
      "AccessLevel": "Public",
      "FolderName": "AnotherPath"
    },
    "Youpi4": {
      "ProviderName": "FirstAzure",
      "AccessLevel": "Private"
    },
    "Youpa2": {
      "ProviderType": "Azure",
      "ConnectionString": "DefaultEndpointsProtocol=https;AccountName=excpetionaccount;AccountKey=<key>;EndpointSuffix=core.windows.net"
    }
  },

  "ScopedStores": {
    "Youpi3": {
      "ProviderName": "AnotherFileSystem",
      "FolderNameFormat": "AnotherPath-{0}"
    },
    "Youpa": {
      "ProviderName": "AnotherAzure",
      "AccessLevel": "Confidential",
      "FolderNameFormat": "Youpa-{0}"
    }
  }
}

Global:

• Provider-level configuration available
• Flattened store-level configuration (aka no more 'Parameters')
• Mandatory keys for a Store:
  • Either ProviderName
  • Or ProviderType and provider's configuration keys
• Container and Path merged into one single configuration key: FolderName. If not specified, the name of the Store is taken.
• AccessLevel: used to define the Azure container public access level, or if the Store may be exposed by the FileSystem server. Available values are:
  • Public: map to BlobContainerPublicAccessType.Container, can list store and read files
  • Confidential: map to BlobContainerPublicAccessType.Blob, can read files
  • Private (default): map to BlobContainerPublicAccessType.Off, no access

ScopeStores:

• FolderNameFormat is mandatory to initialize a Store with params (for example one store per user)
• All other Stores' parameters are available

FileSystem:

• RootPath can now be set from configuration at provider-level

Azure:

• ConnectionString is now defined at provider-level
• It can be replaced with ConnectionStringName if we call an override to AddAzureStorage which take a configuration section to pass the default ConnectionStrings section of the appsettings.json
• At Store-level, if we do not want to reference a named provider, we can either set ConnectionStringName or ConnectionString

What do you think @sandorfr and @arnaudauroux? 😃

[asiffermann]

@arnaudauroux and @annayafi are a little bit shy, but they found it cool! 😄

[arnaudauroux]

@asiffermann We are talking about a configuration file enhancement ;) This is not gonna save kittens ! :P

[anna-git]

this conversation is getting too weird for me !

[sandorfr]

Moi je pense que c'est vital pour les chatons !

[asiffermann]

@sandorfr and @arnaudauroux, it would be great if you could review the PR 😃

There is only two things left:

• Fix sample project
• Add a Validate method on Options to throw proper exceptions if there are missing parameters