AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE flag is not respected.

[SantoshPisini]

AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE flag is not respected.

docker run \ --publish 8081:8081 \ --publish 10250-10255:10250-10255 \ --interactive \ --env AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true \ --tty \ mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest

• OS: macOS 14.2.1 (23C71)
• Image: Linux

[SantoshPisini]

Not able to repro! :)

[SantoshPisini]

Reopening this bug, as it's reproducible by Sergei in Windows host machine + Linux docker containers.

[yaskovdev]

Steps to reproduce (on a Windows machine):

1. Run the emulator: docker run --publish 8081:8081 --publish 10250-10255:10250-10255 --interactive --env AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true --tty mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest
2. Download the certificate: curl -k https://127.0.0.1:8081/_explorer/emulator.pem > emulatorcert0.crt
3. Stop the emulator (Ctrl+C)
4. Run the emulator: docker run --publish 8081:8081 --publish 10250-10255:10250-10255 --interactive --env AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true --tty mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest
5. Download the certificate: curl -k https://127.0.0.1:8081/_explorer/emulator.pem > emulatorcert1.crt
6. Compare emulatorcert0.crt and emulatorcert1.crt. Expected: the files are the same. Actual: the files are different.

Also, if you create a database or a container after step 2, it will be deleted after step 3, despite the AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true environment variable.

[changchiyou]

@yaskovdev

To prevent Docker Desktop from automatically generating a new container name each time you rerun docker run(your step 1. and 4.), consider adding --name cosmos_emulator(replace cosmos_emulator with your preferred name) to your command:

  docker run \
  + --name cosmos_emulator \
    --publish 8081:8081 \
    --publish 10250-10255:10250-10255 \
    --interactive \
    --env AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true \
    --tty mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest

Restart the container with docker restart cosmos_emulator or start the container with docker start cosmos_emulator after it has been stopped (docker stop cosmos_emulator executed), and you can find that emulator.pem and databases/containers persist.

[yaskovdev]

Thank you for the suggestion, @changchiyou! I have found a workaround that utilizes the AZURE_COSMOS_EMULATOR_CERTIFICATE env variable and allows to preserve at least the certificate between the container runs:

docker run -p 8081:8081 -p 10250-10255:10250-10255 -e AZURE_COSMOS_EMULATOR_PARTITION_COUNT=2 `
-e AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=127.0.0.1 -it mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest

docker cp 2cc55b6ba8c208efee0037ac059bd606a65fa0d0878da60d74baf30ea3a3e079:/tmp/cosmos/appdata/default.sslcert.pfx c:\CosmosDB.Emulator\default.sslcert.pfx

# Stop the above container before going further.

docker run -p 8081:8081 -p 10250-10255:10250-10255 -e AZURE_COSMOS_EMULATOR_PARTITION_COUNT=2 `
-e AZURE_COSMOS_EMULATOR_CERTIFICATE=/mnt/host/default.sslcert.pfx -e AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=127.0.0.1 `
-d --restart unless-stopped -v c:/CosmosDB.Emulator:/mnt/host mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest

But your approach is simpler and also allows to preserve the databases. Going to try it.

Update: tried the approach with the --name, works as expected, the detabases and the certificate are preserved between container restarts.

[sajeetharan]

Closing this issue, please feel free to reopen if you are facing any issue!

[yaskovdev]

@sajeetharan, please re-open the issue, since the original problem ("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE flag is not respected") has not been addressed. Tried using the steps to reproduce from my above comment, the content of the files is still different.

Just because there is a workaround does not mean that the issue was resolved.

Also, I don't see the re-open button, looks like I have no permissions to re-open issues (only create new ones).

[sajeetharan]

@yaskovdev

[asgarddesigns]

I am encountering this in docker-compose and not having any luck with this workaround

I assume setting container_name should be the same?

[niteshvijay1995]

@sajeetharan, please re-open the issue, since the original problem ("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE flag is not respected") has not been addressed. Tried using the steps to reproduce from my above comment, the content of the files is still different.

Just because there is a workaround does not mean that the issue was resolved.

Also, I don't see the re-open button, looks like I have no permissions to re-open issues (only create new ones).

@yaskovdev Data persistence will only work in 2 scenarios

1. If the same container is restarted
2. Shared volume is used between containers.

To create a shared volume use - docker volume create --name CosmosDBEmulatorVolume1

and start container using the shared volume -

$parameters = @(
    "--publish", "8081:8081"
    "--publish", "10250-10255:10250-10255"
    "--env", "AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=127.0.0.1"
    "--env", "AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true"
    "--volume", "CosmosDBEmulatorVolume1:/tmp"
    "--detach"
)

docker run @parameters --name emulator1 mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest

// Insert some documents

docker stop emulator1

docker run @parameters --name emulator2 mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest

// Inserted documents are preserved