Azure / azure-cosmos-db-emulator-docker

This repo serves as hub for managing issues, gathering feedback, and having discussions regarding the Cosmos DB Emulator Docker.
https://learn.microsoft.com/en-us/azure/cosmos-db/how-to-develop-emulator?tabs=docker-linux%2Ccsharp&pivots=api-nosql
MIT License
164 stars 47 forks source link

Cosmos Emulator crashing when starting up #113

Open zthurston opened 3 weeks ago

zthurston commented 3 weeks ago

Describe the bug When starting the Cosmos Emulator docker container on our self-hosted GitHub Actions runners, we frequently get a crash:

`This program has encountered a fatal error and cannot continue running at Tue Oct 29 17:54:38 2024 This is an evaluation version. There are [69] days left in the evaluation period. 2.14.20.0 (728f9251) Copyright (C) Microsoft Corporation. All rights reserved. Starting Executing: /usr/local/bin/cosmos/handle-crash.sh with parameters handle-crash.sh /usr/local/bin/cosmos/cosmosdb-emulator 19 /usr/local/bin/cosmos /tmp/cosmos/appdata/log/ 9c431465-bb3a-46ae-8369-473131f2c776 24f10e02-b4bf-4366-841d-de09d0368329 The following diagnostic information is available: Reason: Fatal Signal (0x00000001) Signal: SIGABRT - Aborted (6) Stack: SP IP Function


000074f024c1cc20 0000621b965eb7da 000074f024c1dc80 0000621b965eb1af 000074f024c1dee0 0000621b965ea641 000074f024c1df00 000074f02c8d9090 killpg+0x40 000074f024c1ebd0 000074f02c8d900b gsignal+0xcb 000074f024c1ecf0 000074f02c8b8859 abort+0x12b 000074f024c1ee20 0000621b96573fd2 000074f024c1eed0 0000621b96600d74 000074f024c1eef0 0000621b96634e78 000074f024c1efa0 0000621b96634c5a 000074f024c1f000 0000621b9657fe5a 000074f024c1f080 0000621b9657faad 000074f024c1f160 0000621b965fd2a1 Process: 19 - cosmosdb-emulator Thread: 129 (application thread 0x1dc) Instance Id: 9c431465-bb3a-46ae-8369-473131f2c776 Crash Id: 24f10e02-b4bf-4366-841d-de09d0368329 Build stamp: (null) Distribution: Ubuntu 20.04.6 LTS Processors: 4 Total Memory: 16712458240 bytes Timestamp: Tue Oct 29 17:54:38 2024 Last errno: -34938881 Last errno text: Unknown error -34938881 * PAL PANIC CORE DUMP GENERATION FAILED ** Unable to locate handle-crash.sh. Error: File: signals.cpp:483 [Status: 0xC0000034 Object name not found errno = 0x2(2) No such file or directory] * PANIC CORE DUMP GENERATION FAILED ** Attempt to launch handle-crash.sh failed. This program has encountered a fatal error and cannot continue running at Tue Oct 29 17:54:38 2024 The following diagnostic information is available: Reason: Host Extension RTL_ASSERT (0x00000003) Status: STATUS_WAIT_2 (0x00000002) Message: !killTheTarget Stack: SP IP Function


000074f02a9fbdf0 0000621b965eb7da 000074f02a9fce50 0000621b965eb1af 000074f02a9fd0b0 0000621b9654eaa6 000074f02a9fd0e0 0000621b965eecc4 000074f02a9fe310 0000621b965ee389 000074f02a9fe440 000074f02cf34609 start_thread+0xd9 000074f02a9fe500 000074f02c9b5353 clone+0x43 Process: 17 - cosmosdb-emulator Thread: 18 Instance Id: 9c431465-bb3a-46ae-8369-473131f2c776 Crash Id: 24f10e02-b4bf-4366-841d-de09d0368329 Build stamp: (null) Distribution: Ubuntu 20.04.6 LTS Processors: 4 Total Memory: 16712458240 bytes Timestamp: Tue Oct 29 17:54:38 2024 Last errno: 2 Last errno text: No such file or directory Aborted (core dumped)`

To Reproduce Steps to reproduce the behavior: docker pull mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:stable docker run --publish 8081:8081 --publish 10250-10255:10250-10255 --name=test-container-name --env AZURE_COSMOS_EMULATOR_PARTITION_COUNT=1 -d mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:stable

Expected behavior Container to run and start without crashing

zthurston commented 3 weeks ago

Using the latest tag, I haven't yet had it crash. The version output is 2.14.20.0 (dd7750b6) with 146 days left in the evaluation period.

vinydanylo commented 3 weeks ago

I am also facing this issue when starting the windows emulator. So I always have to reinstall after a reboot...

niteshvijay1995 commented 2 weeks ago

Using the latest tag, I haven't yet had it crash. The version output is 2.14.20.0 (dd7750b6) with 146 days left in the evaluation period.

Use latest tag. stable tagged emulator fails to start on some new Linux kernel versions and hence it is recommended to use latest tagged version on GitHub Actions.

niteshvijay1995 commented 2 weeks ago

I am also facing this issue when starting the windows emulator. So I always have to reinstall after a reboot...

@vinydanylo Could you please share CrashDump from emulator crashes? This page might help you - https://learn.microsoft.com/en-us/troubleshoot/azure/cosmos-db/tools-connectors/emulator