search

duplicati / duplicati

Store securely encrypted backups in the cloud!
Other
10.95k stars 896 forks source link

Backend support for Azure Data Lake Storage Gen2 #3583

Open Schonhoffer opened 5 years ago

Schonhoffer commented 5 years ago

Description

Would like to add Backend support for Azure Data Lake Storage Gen2.

Pricing for the Archive tier for Gen2 is currently $0.001 per GB instead of $0.002 per GB for block blobs (half price!) https://azure.microsoft.com/en-us/pricing/details/storage/data-lake/

Schonhoffer commented 5 years ago

I am interested in trying to implement a Pull Request for this, but would like some guidance before I try to start.

  1. Should this be a new backend prefix ("AzureGen2://") or use existing "azure://" with some flags?
    1. The concepts are different for Gen 2 (filesystem instead of container, OATH user/pass login), but could maybe shove them into same URL scheme?
    2. Should I make a new Backend folder in the source, or put both backends in one folder
  2. Requires Oauth, any problem using the existing duplicati oauth service to handle this?
Pectojin commented 5 years ago

It may be cleaner to implement separately as a new backend.

Gen2 is currently in preview and only supports LRS, so I can imagine various odd issues arising from mixing settings for both protocol versions in the UI.

A secondary benefit is that I imagine they may want to retire Gen1 at some point. Then we can cleanly deprecate the Gen1 API and force users to re-configure using Gen2.

As for Oauth, ideally use the same server for every backend 👍