No new songs being added after database connection error

[FMeinicke]

Describe the bug

I was wondering why I didn't see any data over the past week or so. The last day that had data was 14.07.2023. Looking into the Spotify Developer Dashboard confirmed that there weren't any API calls after that date. Inspecting the container showed the following error message:

2023-07-14T06:59:05.505608763Z [error]  MongoNetworkTimeoutError: connection 48 to 192.168.144.3:27017 timed out
2023-07-14T06:59:05.778582470Z     at Timeout._onTimeout (/app/node_modules/mongodb/lib/cmap/connection.js:125:32)
2023-07-14T06:59:05.778704495Z     at listOnTimeout (node:internal/timers:559:17)
2023-07-14T06:59:05.778751209Z     at processTimers (node:internal/timers:502:7) {
2023-07-14T06:59:05.778794643Z   connectionGeneration: 1,
2023-07-14T06:59:05.778836874Z   [Symbol(errorLabels)]: Set(1) { 'ResetPool' },
2023-07-14T06:59:05.778882004Z   [Symbol(beforeHandshake)]: true
2023-07-14T06:59:05.778921778Z }

Therefore it seems to me that after this network error happened something in the server stopped working.

Simply restarting the container fixed the issue and I'm seeing the data of the current day at least (seems like I have to import the data for the days since the error up to today to get my complete history).

Expected behavior

Graceful recovery in case of such errors, i.e. restarting the polling of the Spotify API.

Screenshots

Spotify Dashboard:

your_spotify timeline:

(Looking at the monthly timeline it seems I had this issue earlier this month as well but didn't notice it. I think I restarted my docker host about 2 weeks ago which would fix the issue.)

[Yooooomi]

Hello, this is something that should already be working. I stored the last timestamp I requested the data from Spotify. What can happen is that we cannot request more than few days ago, leading to missing data as you have. I use the after parameter in this call.

[FMeinicke]

Hmm, interesting. I definitely used Spotify the last couple of days but it only shows the songs I listened to today (this does include songs from this morning which was before I restarted the container, so some past songs have indeed been retrieved).

Do you think that the database connection error has something to do with the server not doing any requests anymore and would you say that this error can be recovered?

Thanks for the fast reply, BTW :)