search

Azure / azure-kusto-java

Microsoft Azure Kusto Library for Java
MIT License
38 stars 43 forks source link

Use azure storage V10 to get async storage usage #69

Open toshetah opened 5 years ago

toshetah commented 5 years ago

This issue is with high importance. We need to upgrade to storage client to V10 in order to get async operations. The current implementation is not async.

In version 10 the Azure Storage Java SDK was separated into 2 sub-modules:

azure-storage-blob
azure-storage-queue

In order to use these modules, they must be added as dependencies to the ingest/pom.xml file, instead of the currently used azure-storage dependency: (Correct to the date of writing this comment)

<dependency>
    <groupId>com.microsoft.azure</groupId>
    <artifactId>azure-storage-blob</artifactId>
    <version>10.1.0</version>
</dependency>
<dependency>
    <groupId>com.microsoft.azure</groupId>
    <artifactId>azure-storage-queue</artifactId>
    <version>10.0.0-Preview</version>
</dependency>

Note that currently the azure-storage-queue is a preview version

Migrating to version 10 will require a substantial rewrite of any component that interfaces with Azure Storage. Here is a list of the classes in the “ingest” module that are involved with the Azure Storage SDK: (correct to the date of writing this comment)

ohadbitt commented 4 years ago

Should be followed with performance comparison Note that blob storage v8 does upload to blob blocks multithreaded

ohadbitt commented 2 years ago

240