search

Azure / azure-cosmosdb-spark

Apache Spark Connector for Azure Cosmos DB
MIT License
201 stars 120 forks source link

add 'TokenResolver' property for CosmosDBConfig #363

Open vi-verrone opened 4 years ago

vi-verrone commented 4 years ago

Hi all, I noticed that a 'TokenResolver' property is missing from CosmosDBConfig and for this reason is not possible to configure AsyncDocumentClient with this useful feature.

Having the chance to set a tokenResolver in configuration would help to develop long-running applications that follow strict policies about accessing CosmosDB collections (i.e. using a broker for resource tokens).

Is this feature planned to be added in next release of azure-cosmosdb-spark? If not, can I contribute to add this feature?

Thanks & Regards

vi-verrone commented 4 years ago

Hi all, almost 3 weeks passed, can someone clarify me if we have in backlog this feature for the next releases?

Thanks & Regards

revinjchalil commented 4 years ago

Would you provide bit more details on what exactly are you looking for? Just fyi, the change to honor ResourceToken in AsyncConnection has been added with https://github.com/Azure/azure-cosmosdb-spark/pull/365 in version 2.11 and allows a static ResourceToken to work with AsyncConnection. Thanks.

vi-verrone commented 4 years ago

Hi @revinjchalil , thanks for replying, my request is indeed to have the possibility to set dynamic resource token provisioning, by using custom logic (i.e. by calling a middle-tier service for token provisioning due security constraints) through the TokenResolver feature of AsyncDocumentClient. For example when we're running streaming-mode jobs that need to write on Cosmos, once we set statically the Resource Token we can't change it after the job starts, and this would be a problem for streaming jobs that run over 5 hours (a resource token's max lifetime AFAIK) and this would let the job to fail at some point. Having the chance to set a TokenResolver when configuring the CosmosDB Spark connector would help in this scenarios, where using a master key or static resource token is not an option.

revinjchalil commented 4 years ago

Makes sense. Please go ahead to make the changes and send PR. Thanks.

Revin

vi-verrone commented 4 years ago

Hi @revinjchalil, you can find the PR at the following link: https://github.com/Azure/azure-cosmosdb-spark/pull/372

Let me know if something else it's needed.

Thanks & Regards,