sportradar / UnifiedOddsSdkNetCore

UnifiedFeed SDK is a client library that enables easier integration with the Betradar XML feeds. SDK exposes XML feed service interface in a more user-friendly way and isolates the client from having to do XML feed parsing, proper connection handling, error recovery, event queuing, data caching and dispatching.
https://sportradar.github.io/UnifiedOddsSdkNetCore/
Other
17 stars 14 forks source link

Lock thread in high concurrency when calling FetchMissingSummary #4

Closed Raulfc88 closed 4 years ago

Raulfc88 commented 4 years ago

Hi,

In the feed we send all the messages to akka actors to avoid block the main thread. And in each actor we process the message and get all information from the SDK that we need, fixture, competitors, etc.

This is an example of how we build our DTO:

internal async Task<ProviderFixtureDto> BuildProviderDtoAsync(IMatch match, EventType eventType = EventType.SNAPSHOT)
 {
            var fixtureTask = match.GetFixtureAsync();
            var homeCompetitorTask = match.GetHomeCompetitorAsync();
            var awayCompetitorTask = match.GetAwayCompetitorAsync();
            var tournamentTask = match.GetTournamentAsync();
            var seasonTask = match.GetSeasonAsync();
            var statusTask = match.GetStatusAsync();
            var sportIdTask = match.GetSportIdAsync();
            var bookStatusTask = match.GetBookingStatusAsync();

            await Task.WhenAll(fixtureTask, homeCompetitorTask, awayCompetitorTask,
                                tournamentTask, statusTask, sportIdTask,
                                bookStatusTask, seasonTask);

            var competitors = _mapper.Map<List<ProviderParticipantDto>>(new List<ITeamCompetitor>() { homeCompetitorTask.Result, awayCompetitorTask.Result });
            var tournamentData = await GetTournamentDataAsync(tournamentTask.Result, seasonTask.Result);

            return await BuildProviderFixtureDto(match, fixtureTask.Result, sportIdTask.Result, eventType, competitors, tournamentData, statusTask.Result, bookStatusTask.Result);
}

So when we get like 50 or more matches in a short time, we started to see a high CPU consumption. And it when from 1-3% to 20-30% and then it stays there forever.

So we started to do some CPU profiling and we got to this result: image_2020_07_07T15_37_03_931Z

As you can see, we are getting a deadlock in that thread that is constantly consuming CPU.

It can be a good solution to allow to send a cancelationToken so we can cancel from our side.

Thanks.

PD: I attach a project that reproduce de issue. If you check it, you will see that the SemaphorePool.Acquire is stuck in the line: https://github.com/sportradar/UnifiedOddsSdkNetCore/blob/f3fe255e0e0dec410729d5a09af9ffc73690fe9b/src/Sportradar.OddsFeed.SDK/Common/Internal/SemaphorePool.cs#L166 SportRadarSemaphorePool.zip

ghost commented 4 years ago

Hi,

Unfortunately, due to internal SDK implementation, all async methods should be immediately awaited, just like in the examples.

internal async Task<ProviderFixtureDto> BuildProviderDtoAsync(IMatch match, EventType eventType = EventType.SNAPSHOT)
 {
            var fixture = await match.GetFixtureAsync();
            var homeCompetitor = await match.GetHomeCompetitorAsync();
            var awayCompetitor = await match.GetAwayCompetitorAsync();
            var tournament = await match.GetTournamentAsync();
            var season = await match.GetSeasonAsync();
            var status = await match.GetStatusAsync();
            var sportId = await match.GetSportIdAsync();
            var bookStatus = await match.GetBookingStatusAsync();

            var competitors = _mapper.Map<List<ProviderParticipantDto>>(new List<ITeamCompetitor>() { homeCompetitor, awayCompetitor });
            var tournamentData = await GetTournamentDataAsync(tournament, season);

            return await BuildProviderFixtureDto(match, fixture, sportId, eventType, competitors, tournamentData, status, bookStatus);
}
Raulfc88 commented 4 years ago

Hi,

Thanks for replying so fast!! Yeah we were testing also that in our test and we see that awaiting fixed all problems.

Thanks for confirming.

I will close the issue then.

venjirai commented 3 years ago

Hello, is it still relevant to await every async task? I can't find any awaits in the example projects. Our problem is, that we sometimes very rarely get a deadlock in the following piece of code:

` var bookingStatusTask = bMatch.GetBookingStatusAsync(); var statusTask = bMatch.GetStatusAsync(); var scheduledTimeTask = bMatch.GetScheduledTimeAsync(); var homeTeamTask = bMatch.GetHomeCompetitorAsync(); var awayTeamTask = bMatch.GetAwayCompetitorAsync();

        if (!Task.WaitAll(new Task[] { bookingStatusTask, statusTask, scheduledTimeTask, homeTeamTask, awayTeamTask}, 3000))
        {
            Console.WriteLine($"GetMatch tasks timeout, match id: {bMatch.Id}");
            return null;
        }

`

The timeout doesn't trigger here. Thanks

dhrovat commented 3 years ago

Hi,

I have found where the problem is and implemented fix. Will be available in next release.

Tnx.