dotnet / nuget-trends

Check out NuGet packages adoption and what's trending on NuGet.
https://nugettrends.com
MIT License
150 stars 28 forks source link

Investigate exceptions happening with the Scheduler #167

Open joaopgrassi opened 3 years ago

joaopgrassi commented 3 years ago

While trying to find out what happened on #165, I noticed that the postgres container logs a log of these messages:

2021-08-21 01:07:21.819 UTC [14052] ERROR:  canceling statement due to user request
2021-08-21 01:07:21.819 UTC [14052] STATEMENT:  SELECT package_id FROM pending_packages_daily_downloads

Also we get exceptions like these on Sentry:

Npgsql.NpgsqlException: Exception while reading from stream
  ?, in async ValueTask<IBackendMessage> NpgsqlConnector.ReadMessage(bool async, DataRowLoadingMode dataRowLoadingMode, bool readingNotifications)+ReadMessageLong(?)
  ?, in async Task<bool> NpgsqlDataReader.NextResult(bool async, bool isConsuming, CancellationToken cancellationToken)
  ?, in async ValueTask<NpgsqlDataReader> NpgsqlCommand.ExecuteReader(CommandBehavior behavior, bool async, CancellationToken cancellationToken) x 2
  File "/home/runner/work/nuget-trends/nuget-trends/src/NuGetTrends.Scheduler/DailyDownloadPackageIdPublisher.cs", line 74, col 21, in async Task DailyDownloadPackageIdPublisher.Import(IJobCancellationToken token)
  File "/home/runner/work/nuget-trends/nuget-trends/src/NuGetTrends.Scheduler/DailyDownloadPackageIdPublisher.cs", line 105, col 21, in async Task DailyDownloadPackageIdPublisher.Import(IJobCancellationToken token) x 2
  ?, in void TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

A brief read indicates that it might be related to connection polling, or maybe some cancellation being triggered by something. https://github.com/npgsql/npgsql/issues/2711 https://github.com/npgsql/npgsql/issues/3052