Please improve the Azure Cosmos DB Emulator MongoDB API Guidance

[samgladstone]

I'm finding it really hard to understand how to use the emulator with the MongoDB API. The documentation is quite limited and I have not been able to make much progress.

MongoDB wire protocol version 3.2

Enabling the 3.2 MongoDB protocol is documented OK and I'm able to get this working. However, I need the 3.6 version for some extra features that are supported.

MongoDB wire protocol version 3.6

Following the documentation, I was able to start the emulator with:

C:\Program Files\Azure Cosmos DB Emulator> Microsoft.Azure.Cosmos.Emulator.exe /EnableMongoDbEndpoint=3.6

However, port 10250 isn't being used (as per the documentation).

I found this comment (from 2017), which suggests that the MongoDB API is on port 10250. I tried using that without much luck.

I also tried using:

C:\Program Files\Azure Cosmos DB Emulator>Microsoft.Azure.Cosmos.Emulator.exe /EnableMongoDbEndpoint=3.6 /MongoPort=10255

With this, I am able to connect but get the following error when trying to create new documents.

Error=2, 
Details='Microsoft.Azure.Documents.BadRequestException: 
Message: {"Errors":["PartitionKey extracted from document doesn't match the one specified in the header"]}ActivityId: 02493bde-9f24-4a62-a515-fa0fd27f27a2, 
Request URI: /apps/DocDbApp/services/DocDbServer6/partitions/a4cb4952-38c8-11e6-8106-8cdcd42c33be/replicas/1p/, 
RequestStats: 
RequestStartTime: 2020-02-12T20:47:27.8471941Z, 
RequestEndTime: 2020-02-12T20:47:27.8488575Z,  
Number of regions attempted:1
ResponseTime: 2020-02-12T20:47:27.8488575Z, 
StoreResult: 
StorePhysicalAddress: rntbd://127.0.0.1:10253/apps/DocDbApp/services/DocDbServer6/partitions/a4cb4952-38c8-11e6-8106-8cdcd42c33be/replicas/1p/, 
LSN: 13, 
GlobalCommittedLsn: -1, 
PartitionKeyRangeId: 0, 
IsValid: True, 
StatusCode: 400, 
SubStatusCode: 1001, 
RequestCharge: 1.24, 
ItemLSN: -1, 
SessionToken: -1#13, 
UsingLocalLSN: False, 
TransportException: null, 
ResourceType: Document, 
OperationType: Create, SDK: 
Windows/10.0.18362 cosmos-netstandard-sdk/3.3.2   

at Microsoft.Azure.Documents.StoreResult.ToResponse(RequestChargeTracker requestChargeTracker)   
at Microsoft.Azure.Documents.ConsistencyWriter.WritePrivateAsync(DocumentServiceRequest request, TimeoutHelper timeout, Boolean forceRefresh)   
at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetryAsync(Func`1 callbackMethod, Func`3 callShouldRetry, Func`1 inBackoffAlternateCallbackMethod, TimeSpan minBackoffForInBackoffCallback, CancellationToken cancellationToken, Action`1 preRetryCallback)   at Microsoft.Azure.Documents.ShouldRetryResult.ThrowIfDoneTrying(ExceptionDispatchInfo capturedException)   
at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetryAsync(Func`1 callbackMethod, Func`3 callShouldRetry, Func`1 inBackoffAlternateCallbackMethod, TimeSpan minBackoffForInBackoffCallback, CancellationToken cancellationToken, Action`1 preRetryCallback)   at Microsoft.Azure.Documents.ConsistencyWriter.WriteAsync(DocumentServiceRequest entity, TimeoutHelper timeout, Boolean forceRefresh, CancellationToken cancellationToken)  
at Microsoft.Azure.Documents.ReplicatedResourceClient.<>c__DisplayClass26_0.<<InvokeAsync>b__0>d.MoveNext()--- End of stack trace from previous location where exception was thrown ---   
at Microsoft.Azure.Documents.RequestRetryUtility.ProcessRequestAsync[TRequest,IRetriableResponse](Func`1 executeAsync, Func`1 prepareRequest, IRequestRetryPolicy`2 policy, CancellationToken cancellationToken, Func`1 inBackoffAlternateCallbackMethod, Nullable`1 minBackoffForInBackoffCallback)   
at Microsoft.Azure.Documents.ShouldRetryResult.ThrowIfDoneTrying(ExceptionDispatchInfo capturedException)   
at Microsoft.Azure.Documents.RequestRetryUtility.ProcessRequestAsync[TRequest,IRetriableResponse](Func`1 executeAsync, Func`1 prepareRequest, IRequestRetryPolicy`2 policy, CancellationToken cancellationToken, Func`1 inBackoffAlternateCallbackMethod, Nullable`1 minBackoffForInBackoffCallback)  
at Microsoft.Azure.Documents.StoreClient.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken, IRetryPolicy retryPolicy, Func`2 prepareRequestAsyncDelegate)   at Microsoft.Azure.Documents.ServerStoreModel.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken)   
at Microsoft.Azure.Cosmos.Handlers.TransportHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)

However, I don't get this problem when trying the exact same set of commands with an actual Cosmos DB, which works fine. I am unsure where to go from here.

Docker

The documentation on docker hub does not reference the MongoDB API at all. I figured out that you can enable the MongoDB Endpoint by setting the -e AZURE_COSMOS_EMULATOR_MONGODB_ENDPOINT=true. But I cannot figure out how to use protocol version 3.6.

For now, I will develop locally using a mongo server and hope that everything works when it comes to production. However, I would prefer to keep my develop and production environments as similar as possible, so any help would be much appreciated.

---

Document details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 109ed3f8-faf1-92b9-bf50-4be52e9099d4
• Version Independent ID: ebc5326a-46da-6bc1-1f47-01a8e16d11fb
• Content: Develop locally with the Azure Cosmos Emulator
• Content Source: articles/cosmos-db/local-emulator.md
• Service: cosmos-db
• GitHub Login: @markjbrown
• Microsoft Alias: mjbrown

[NavtejSaini-MSFT]

@samgladstone We are checking this and we will get back to you.

[markjbrown]

Hi @samgladstone

For this issue for creating a docker emulator for MongoDB 3.6 please open the Data Explorer in the emulator and submit this feedback there. The team that does this work tracks their work items from the feedback link in the Data Explorer there.

Thanks.

please-close

[Adebeer]

Please add the below to the documentation when using docker image. This wasn't obvious to me as all the port mappings seem to imply that all apis are by default enabled. Also, enabling MongoDB API is undocumented - even in https://hub.docker.com/r/microsoft/azure-cosmosdb-emulator/

When creating docker image, to enable support for other API endpoints, you need to include these parameters (not sure if you can enable multiple) a) For Gremlin: -e AZURE_COSMOS_EMULATOR_GREMLIN_ENDPOINT=true b) For Cassandra -e AZURE_COSMOS_EMULATOR_CASSANDRA_ENDPOINT=true c) For MongoDB -e AZURE_COSMOS_EMULATOR_MONGODB_ENDPOINT=true

[samgladstone]

In case anyone else comes across this issue and to add to the above, the MongoDB 3.6 Endpoint can be enabled with this param:

-e AZURE_COSMOS_EMULATOR_MONGODB_COMPUTE_ENDPOINT=true

A couple of extra things to note on it;

1. The person I spoke to when submitting my feedback said:

   Previously when MongoDB 3.6 was enabled, it will use the 10250 as the default port; this can be particularly confusing since the connection strings points to port 10255. In the latest version of the emulator (2.9.2) we made this setting consistent and the default port is now 10255.

2. You can't enable both the 3.2 & 3.6 endpoints at the same time (I'm not sure about other endpoints)

3. When enabling the 3.6 protocol, you can't use the UI to carry out tasks such as creating containers etc, this needs to be done via commands.

[ondrejpialek]

Sadly it seems that using AZURE_COSMOS_EMULATOR_MONGODB_COMPUTE_ENDPOINT no longer works to get the 3.6 endpoint. By now we have the 4.0 endpoint, so getting that to work would be great. Any ideas?

[eugbyte]

@ondrejpialek i am facing the same problem. The linux documentation does not publish port 10255. port 10255 is part of the mongoDB connection string

the emulator mongo db connection string is specified as mongodb://localhost:C2y6yDjf5...Jw==@localhost:10255/admin?ssl=true

docker run \
    --publish 8081:8081 \
    --publish 10251-10254:10251-10254 \              // <--- port 10255 is missing
    --memory 3g --cpus=2.0 \
    --name=test-linux-emulator \