Azure / azure-documentdb-datamigrationtool

Azure DocumentDB Data Migration Tool
MIT License
387 stars 182 forks source link

Bulk Import - PartitionKey value must be supplied for this operation #47

Closed garfbradaz closed 2 years ago

garfbradaz commented 6 years ago

Issue

When I want to use the Migration tool to import data from a SQL Source to pre-built (i.e. manually setting up via the Portal with a PartitionKey) Collection I get the following error:

Row 1: PartitionKey value must be supplied for this operation.

I'm using the DocumentDB - Bulk Import selections, because I want to use a custom Bulk Insert Stored Procedure.

I though do include a Partition Key when setting up the target information. The funny thing is though, if I delete the existing Collection and re-run the Migration Tool, it creates the Collection BUT without a Partition Key.

FYI I'm using the local CosmosDB Emulator.

Repro Steps

To get Error and no Row inserted

  1. Setup a new Database and Collection in the Local Emulator
  2. Set the PartitionKey up. Example I set mine up to be /division (nb: I changed the collection to Products after I took the image):

image

  1. Within dtui.exe Source Information I reference a query similar to following gist. This query is so I can use the custom stored procedure later on, to embed the biccodes JSON as subdocuments.
  2. Within dtui.exe Target Information, I set the following up. Note I'm including division in the Partition Key field.

image

  1. Within dtui.exe run a new Import. I get the following error:

Row 1 Microsoft.DataTransfer.DocumentDb.Exceptions.FailedToCreateDocumentException: PartitionKey value must be supplied for this operation. at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.DataTransfer.Core.Service.DataTransferAction.d__1.MoveNext() in F:_Bld\16189\6357\Sources_repo\Core\Microsoft.DataTransfer.Core\Service\DataTransferAction.cs:line 79

Don't get an error and the Row is inserted

  1. Delete the collection you created previously in the Local Emulator
  2. Within the dtui.exe re-use the same Target Information and import
  3. You will find the Collection will be created for you automatically but without a Partition Key assigned to the collection. As you can see NO /division column on the right hand side:

image

image

garfbradaz commented 6 years ago

Any thoughts.....anyone?

nahk-ivanov commented 6 years ago

Hi,

As drop-down option suggests - Bulk import only support single partition collections.

Regards, Alex.

garfbradaz commented 6 years ago

@nahk-ivanov : Haha, I didn't see that (or it didn't twig in my tired brain) :)

The reason I wanted to use Bulk Import is I wanted to use the Stored Procedures to handle array of subdocuments within a record. Similar to an issue described here:

https://stackoverflow.com/questions/45981322/how-to-import-documents-that-have-arrays-with-the-cosmos-db-data-migration-tool[](https://stackoverflow.com/questions/45981322/how-to-import-documents-that-have-arrays-with-the-cosmos-db-data-migration-tool)

gertaylor commented 6 years ago

@garfbradaz did you get anywhere with this issue?

garfbradaz commented 6 years ago

@gertaylor : I didn't. Im going to log a separate Issue about my actual issue with array of SubDocuments from SQL-To-Cosmos. Maybe I'm missing something :)

kevinding0218 commented 5 years ago

Wow, I'm having the exact issue with trying to migrate documents which contains the nested collections using the store procedure, @garfbradaz did you find out any solution later?