AzureCosmosDB / data-migration-desktop-tool

MIT License
121 stars 46 forks source link

Cosmos DB MongoDB API Array field export to json file cannot display correctly #126

Open stella22-github opened 4 months ago

stella22-github commented 4 months ago

Hi Team,

May I know it is DMT tool that cannot support Cosmos DB MongoDB API array export to json format? It can be easily reproduced by the example data provided below and the sample code below.

Here is my sample data at Cosmos DB MongoDB API, as tags inside have array, but after I have export data from Cosmos MongoDB to JSON for the part of arrays it cannot display correctly and instead show "System.Collections.Generic.Dictionary`2[System.String,System.Object]", I suspect it related to the deserialize in json, may I know it is array is not supported or it is a bug? Will it be fix? Thank your for the tool!

The DMT version I'm using is 2.1.4 on Windows.

My sample data inside Cosmos DB MongoDB API:

{
    "_id" : ObjectId("661e2ad4ea41a614c092ad24"),
    "id" : "03246",
    "description" : "Babyfood, dessert, custard pudding, vanilla, junior",
    "tags" : [
        {
            "name" : "babyfood",
            "roles" : "abc"
        },
        {
            "name" : "dessert"
        },
        {
            "name" : "custard pudding"
        },
        {
            "name" : "vanilla"
        },
        {
            "name" : "junior"
        }
    ]
}

My migrationsettings.json:

{
  "Source": "mongodb",
  "Sink": "JSON",
  "SourceSettings": {
    "ConnectionString": "mongodb://xxx==@xxx.mongo.cosmos.azure.com:10255/?ssl=true&replicaSet=globaldb&retrywrites=false&maxIdleTimeMS=120000&appName=@xxx@",
    "DatabaseName": "testIntra",
    "Collection":"testArray"
  },
  "SinkSettings":
  {
    "FilePath": "d:\\testArray1.json",
    "Indented": true
  }
}

Here is the result export to JSON, the tags arrays part cannot display properly.
[
  {
    "_id": "661e2ad4ea41a614c092ad24",
    "id": "03246",
    "description": "Babyfood, dessert, custard pudding, vanilla, junior",
    "tags": [
      "System.Collections.Generic.Dictionary`2[System.String,System.Object]",
      "System.Collections.Generic.Dictionary`2[System.String,System.Object]",
      "System.Collections.Generic.Dictionary`2[System.String,System.Object]",
      "System.Collections.Generic.Dictionary`2[System.String,System.Object]",
      "System.Collections.Generic.Dictionary`2[System.String,System.Object]"
    ]
  }
]
fidelcasto commented 2 months ago

Same problem here, we can't use this tool for export since arrays don't seem to be handled by the tool yet. @stella22-github did you find any workaround to this?