Yooooomi / your_spotify

Self hosted Spotify tracking dashboard
GNU General Public License v3.0
2.77k stars 110 forks source link

MongoServerError: PlanExecutor error during aggregation :: caused by :: Exceeded memory limit for $group, but didn't allow external sort. #272

Open samip5 opened 1 year ago

samip5 commented 1 year ago

Describe the bug

MongoDB ran out of memory limit for $group and allowDiskUse is not passed.

Expected behavior

I expected it to work like normal

Additional context

[info]  Checking database...
/app/node_modules/mongodb/lib/cmap/connection.js:202
                callback(new error_1.MongoServerError(document));
                         ^

MongoServerError: PlanExecutor error during aggregation :: caused by :: Exceeded memory limit for $group, but didn't allow external sort. Pass allowDiskUse:true to opt in.
    at Connection.onMessage (/app/node_modules/mongodb/lib/cmap/connection.js:202:26)
    at MessageStream.<anonymous> (/app/node_modules/mongodb/lib/cmap/connection.js:61:60)
    at MessageStream.emit (node:events:513:28)
    at processIncomingData (/app/node_modules/mongodb/lib/cmap/message_stream.js:124:16)
    at MessageStream._write (/app/node_modules/mongodb/lib/cmap/message_stream.js:33:9)
    at writeOrBuffer (node:internal/streams/writable:391:12)
    at _write (node:internal/streams/writable:332:10)
    at MessageStream.Writable.write (node:internal/streams/writable:336:10)
    at Socket.ondata (node:internal/streams/readable:754:22)
    at Socket.emit (node:events:513:28) {
  ok: 0,
  code: 292,
  codeName: 'QueryExceededMemoryLimitNoDiskUseAllowed',
  [Symbol(errorLabels)]: Set(0) {}
}
samip5 commented 1 year ago

Hi @Yooooomi, this prevents the whole thing from properly starting up, and I believe it's version 1.7.0.

Yooooomi commented 1 year ago

Damn. I'll check which calls are using a group at the start of the app. I don't remember adding new requests to 1.7.0

samip5 commented 1 year ago

Damn. I'll check which calls are using a group at the start of the app. I don't remember adding new requests to 1.7.0

I'm not even sure if it has to do with 1.7.0 or if it's been there for a while but it just happens to run into it now.

samip5 commented 11 months ago

@Yooooomi Any ideas on fixes to this?

Yooooomi commented 11 months ago

Hello! Sorry I've been on vacation I had no time to look at this. My first thoughts would be to check the check functions at startup to see if there are any group used there.

Yooooomi commented 9 months ago

Hey, I was looking at this issue but I feel like I already solved it in 1.7.1, could you confirm?