search

hcengineering / huly-selfhost

Huly for Self Hosting
Eclipse Public License 2.0
359 stars 43 forks source link

Huly Cosmos DB Mongo Azure #29

Open czujsnn opened 2 weeks ago

czujsnn commented 2 weeks ago

I have created cosmos db for mongodb in Azure, configured compose.yml to use connection string to my mongo, and it errors out at 20% of workspace creation with following error:

huly-selfhost-account-1 | {"err":{"code":2,"codeName":"BadValue","errorResponse":{"code":2,"codeName":"BadValue","errmsg":"Error=2, Details='Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 3318a5c9-d1b5-4893-861d-73dc4c97ff29; Reason: (Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 3318a5c9-d1b5-4893-861d-73dc4c97ff29; Reason: (Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 3318a5c9-d1b5-4893-861d-73dc4c97ff29; Reason: (Message: {\"Errors\":[\"The index path corresponding to the specified order-by item is excluded.\"]}\r\nActivityId: 3318a5c9-d1b5-4893-861d-73dc4c97ff29, Request URI: /apps/ef91e0f5-0303-489d-9342-a9fbe1d65465/services/79a175d4-3762-49db-afc2-5f3e49f16922/partitions/f1d94ce1-3b30-4c2c-b75b-4613f4aec9df/replicas/133694283049277431s/, RequestStats: Microsoft.Azure.Cosmos.Tracing.TraceData.ClientSideRequestStatisticsTraceDatum, SDK: Windows/10.0.20348 cosmos-netstandard-sdk/3.18.0);););","ok":0},"message":"Error=2, Details='Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 3318a5c9-d1b5-4893-861d-73dc4c97ff29; Reason: (Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 3318a5c9-d1b5-4893-861d-73dc4c97ff29; Reason: (Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 3318a5c9-d1b5-4893-861d-73dc4c97ff29; Reason: (Message: {\"Errors\":[\"The index path corresponding to the specified order-by item is excluded.\"]}\r\nActivityId: 3318a5c9-d1b5-4893-861d-73dc4c97ff29, Request URI: /apps/ef91e0f5-0303-489d-9342-a9fbe1d65465/services/79a175d4-3762-49db-afc2-5f3e49f16922/partitions/f1d94ce1-3b30-4c2c-b75b-4613f4aec9df/replicas/133694283049277431s/, RequestStats: Microsoft.Azure.Cosmos.Tracing.TraceData.ClientSideRequestStatisticsTraceDatum, SDK: Windows/10.0.20348 cosmos-netstandard-sdk/3.18.0);););","ok":0,"stack":"MongoServerError: Error=2, Details='Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 3318a5c9-d1b5-4893-861d-73dc4c97ff29; Reason: (Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 3318a5c9-d1b5-4893-861d-73dc4c97ff29; Reason: (Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 3318a5c9-d1b5-4893-861d-73dc4c97ff29; Reason: (Message: {\"Errors\":[\"The index path corresponding to the specified order-by item is excluded.\"]}\r\nActivityId: 3318a5c9-d1b5-4893-861d-73dc4c97ff29, Request URI: /apps/ef91e0f5-0303-489d-9342-a9fbe1d65465/services/79a175d4-3762-49db-afc2-5f3e49f16922/partitions/f1d94ce1-3b30-4c2c-b75b-4613f4aec9df/replicas/133694283049277431s/, RequestStats: Microsoft.Azure.Cosmos.Tracing.TraceData.ClientSideRequestStatisticsTraceDatum, SDK: Windows/10.0.20348 cosmos-netstandard-sdk/3.18.0);););\n at e.sendCommand (/usr/src/app/bundle.js:286:31053)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async e.command (/usr/src/app/bundle.js:286:31679)\n at async e.command (/usr/src/app/bundle.js:286:60533)\n at async NFe.execute (/usr/src/app/bundle.js:269:51678)\n at async yGa (/usr/src/app/bundle.js:268:63474)\n at async e._initialize (/usr/src/app/bundle.js:269:54283)\n at async e.cursorInit (/usr/src/app/bundle.js:269:44836)\n at async e.fetchBatch (/usr/src/app/bundle.js:269:45224)\n at async e.next (/usr/src/app/bundle.js:269:41952)"},"level":"error","message":"error","status":{"code":"platform:status:InternalServerError","params":{},"severity":"ERROR"},"timestamp":"2024-08-29T20:56:09.615Z"}

huly-selfhost-account-1 | {"level":"error","message":"Unhandled Rejection at:","promise":{},"reason":{"code":2,"codeName":"BadValue","errorResponse":{"code":2,"codeName":"BadValue","errmsg":"Error=2, Details='Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 15207215-34e6-4e09-841f-7b7d38d553c2; Reason: (Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 15207215-34e6-4e09-841f-7b7d38d553c2; Reason: (Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 15207215-34e6-4e09-841f-7b7d38d553c2; Reason: (Message: {\"Errors\":[\"The order by query does not have a corresponding composite index that it can be served from.\"]}\r\nActivityId: 15207215-34e6-4e09-841f-7b7d38d553c2, Request URI: /apps/7cbf22ca-c22e-4dab-a37e-b4a9279a0f42/services/e2e0f9bd-39a7-4ce4-b5c3-354b7cae6c96/partitions/1f2823b1-5be8-40a5-a5ee-ef93979990b8/replicas/133694028058343296s/, RequestStats: Microsoft.Azure.Cosmos.Tracing.TraceData.ClientSideRequestStatisticsTraceDatum, SDK: Windows/10.0.20348 cosmos-netstandard-sdk/3.18.0);););","ok":0},"message":"Error=2, Details='Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 15207215-34e6-4e09-841f-7b7d38d553c2; Reason: (Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 15207215-34e6-4e09-841f-7b7d38d553c2; Reason: (Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 15207215-34e6-4e09-841f-7b7d38d553c2; Reason: (Message: {\"Errors\":[\"The order by query does not have a corresponding composite index that it can be served from.\"]}\r\nActivityId: 15207215-34e6-4e09-841f-7b7d38d553c2, Request URI: /apps/7cbf22ca-c22e-4dab-a37e-b4a9279a0f42/services/e2e0f9bd-39a7-4ce4-b5c3-354b7cae6c96/partitions/1f2823b1-5be8-40a5-a5ee-ef93979990b8/replicas/133694028058343296s/, RequestStats: Microsoft.Azure.Cosmos.Tracing.TraceData.ClientSideRequestStatisticsTraceDatum, SDK: Windows/10.0.20348 cosmos-netstandard-sdk/3.18.0);););","ok":0,"stack":"MongoServerError: Error=2, Details='Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 15207215-34e6-4e09-841f-7b7d38d553c2; Reason: (Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 15207215-34e6-4e09-841f-7b7d38d553c2; Reason: (Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 15207215-34e6-4e09-841f-7b7d38d553c2; Reason: (Message: {\"Errors\":[\"The order by query does not have a corresponding composite index that it can be served from.\"]}\r\nActivityId: 15207215-34e6-4e09-841f-7b7d38d553c2, Request URI: /apps/7cbf22ca-c22e-4dab-a37e-b4a9279a0f42/services/e2e0f9bd-39a7-4ce4-b5c3-354b7cae6c96/partitions/1f2823b1-5be8-40a5-a5ee-ef93979990b8/replicas/133694028058343296s/, RequestStats: Microsoft.Azure.Cosmos.Tracing.TraceData.ClientSideRequestStatisticsTraceDatum, SDK: Windows/10.0.20348 cosmos-netstandard-sdk/3.18.0);););\n at e.sendCommand (/usr/src/app/bundle.js:286:31053)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async e.command (/usr/src/app/bundle.js:286:31679)\n at async e.command (/usr/src/app/bundle.js:286:60533)\n at async NFe.execute (/usr/src/app/bundle.js:269:51678)\n at async yGa (/usr/src/app/bundle.js:268:63474)\n at async e._initialize (/usr/src/app/bundle.js:269:54283)\n at async e.cursorInit (/usr/src/app/bundle.js:269:44836)\n at async e.fetchBatch (/usr/src/app/bundle.js:269:45224)\n at async e.next (/usr/src/app/bundle.js:269:41952)"},"timestamp":"2024-08-29T21:01:25.733Z"}

"The order by query does not have a corresponding composite index that it can be served from." + "The index path corresponding to the specified order-by item is excluded."

I have tried using mongo cloud, and it works "out of the box". Is this error caused by some limitations of mongo API implementation in cosmosDb?

aonnikov commented 1 week ago

Hi @czujsnn , thanks for reporting this. We have never tried to use Cosmos DB for Huly deployment, hence never experienced and expected facing this issue. Based on the documentation and my understanding of the error message, Cosmos DB requires a composite index to exist, when ORDER BY is used. Assuming that ordering can be specified dynamically in Huly, it does not seem to be possible, because we would require a composite index for each combination of fields.

czujsnn commented 1 week ago

hi @aonnikov Thanks for the input! After a lot of trail and error, I agree, as Cosmos db in Azure isn't "true" mongo as in for example atlas.

To anyone having this error - I managed to make it work, using M0/M2/M5 tier from https://cloud.mongodb.com

There is a gotcha though - those tiers support database name lenght of max 38 Bytes. Huly by default creates longer db names, so I had to manually edit a bit of code in Huly Account pod service, and truncate it down to 38 chars. Then after rebuilding project, and docker image my huly self service successfully connects to managed mongo instance, instead of running in docker container.