PlagueHO / CosmosDB

PowerShell Module for working with Azure Cosmos DB databases, collections, documents, attachments, offers, users, permissions, triggers, stored procedures and user defined functions.
http://dscottraynsford.com
MIT License
154 stars 46 forks source link

Missing cross-partition querying #379

Closed tamusjroyce closed 4 years ago

tamusjroyce commented 4 years ago

x-ms-query-enable-crosspartition: True

When using function New-CosmosDbDocument, I've noticed that I have to specify a partition key. This would be fine except that I am copying data from collection A to collection B. And all I have is the id, not the partition key.

$documentsPerRequest = 20
$continuationToken = $null
$documents = $null

$sourcePartitionKey = "SomePk"

do {
    $responseHeader = $null
    $getCosmosDbDocumentParameters = @{
        Context = $cosmosDbContext
        CollectionId = 'MyNewCollection'
        MaxItemCount = $documentsPerRequest
        ResponseHeader = ([ref] $responseHeader)
    }

    if ($continuationToken) {
        $getCosmosDbDocumentParameters.ContinuationToken = $continuationToken
    }

    # Is there any way this could return the partition key??? like partitionKeyValue: "SomeValue"? (probably a limitation with Cosmos Rest API - haven't delved in yet)
    $documents = Get-CosmosDbDocument @getCosmosDbDocumentParameters

    $documents | Foreach-Object {
        $id = $_.id
        $query = "SELECT * FROM c WHERE c.id = '$id'"
        $document = Get-CosmosDbDocument -Context $sourceCosmosDbContext -CollectionId $sourceContainerName -Query $query
        $document
        $documentObject = $document | ConvertFrom-Json
        $partitonKey = Invoke-Expression "`$documentObject.$sourcePartitionKey"
        # With the partition key, insert data into destination cosmos collection
     }
    $continuationToken = Get-CosmosDbContinuationToken -ResponseHeader $responseHeader
} while (-not [System.String]::IsNullOrEmpty($continuationToken))

I will try to look into a PR in the future. -AllowCrossPartition as a flag or something. But I have some deadlines first.

Thank you for the great project! Was hoping to copy data similar with CosmosClone with better interactions during the copy and easy customization for DevOps.

tamusjroyce commented 4 years ago

Missed that '-QueryEnableCrossPartition $true' was an option. Looking to add it to the wrong place. Thank you!