Sink: Cosmos Db, Mongo Db

[jnoren1]

First, awesome tool!

Second, I'm trying to migrate from an existing mongo database to azure cosmos db, mongo flavor.

There are a couple of differences, for example instead of partition key paths it has Shard Keys, though under the hood my understanding is that they are functionally the same. and by reading the error messages I was able to figure out that what would be a translation, e.g. "\id" is now /\"$v\"/\"id\"/\"$v\".

The big issue is now that when it tries to upload the data it cannot.

"Microsoft.Azure.Documents.DocumentClientException: Sql api is not supported for this database account"

Which makes sense, since at the top of Cosmos DB plug-in page it explicitly says that it uses the SQL Api to function. (Even though the name of the plugin in the db is "cosmos-nosql".)

So I guess, the question is, without rolling my own plugin, is there a way to get this to work? (I tried using the MongoDb plugin as the sink, but it didn't like the cosmos connection string.)

Have I really found such a unique data transfer scenario that there doesn't exist a tool to do it already? (The Microsoft provided tools fail to function because the target is serverless instead of provisioned.)

{
  "Source": "MongoDB",
  "Sink": "Cosmos-nosql",
  "SourceSettings": {
    "ConnectionString": "mongodb://.................",
    "DatabaseName": "<dbName>",
  },
  "SinkSettings": {  
    "ConnectionString": "AccountEndpoint=https://...................................==",
    "Database":"<dbName>",
    "IncludeMetadataFields": true,
    "IsServerlessAccount": true,
    "RecreateContainer": false,
  },
  "Operations": [
    {
        "SourceSettings": {
            "Collection": "account"
        },
        "SinkSettings": {
            "Container": "account",
            "PartitionKeyPath": "/\"$v\"/\"id\"/\"$v\""
        }
    },
...
]

[bowencode]

For this scenario, you should be using the MongoDB extension on both ends because, as you noted, the Cosmos-NoSQL extension is specific to that API model.

It sounds like you did try that route but had a problem with the connection string so that's probably the place to look. Those connection strings should be listed with the Keys section in the account and usually look something like mongodb://my-mongo-db:BASE64KEY==@my-mongo-db.mongo.cosmos.azure.com:10255/...

[jnoren1]

It was very odd, when I created the database in Azure, i selected to create the RU billed version instead of the more standard model. The RU billed version gave the more standard Azure connection string not a mongodb connection string as you listed. Eventually, I had got frustrated because no tool would support it, and just deleted it and created the more standard billed version of the same database. This version provides only the mongodb connection string, like you showed. The tool works flawlessly with that version. Not sure what is going on with the RU model of the mongodb, or if something went wrong in my specific creation. (Though I think there is a generally lack of something here... because the microsoft provided migration tools also do not work on that model. They say support is forthcoming.)

[crizzs]

I was also met with the same issue. The SQL API is not supported for cosmosdb mongo