Converting a large list to a connection.

[MartinDawson]

Hey,

So I used to do this to create my connection:

Connection<LiveStreamPayload>()
    .Name("liveStreams")
    .Resolve(c =>
    {
        var liveStreams = liveStreamService.GetAudios();
        var splitLiveStreams = ConnectionUtils.ToConection(liveStreams);

        return splitLiveStreams;
    });

However the code internally calls .ToList() which means that for every page it is returning all of my 25k records from the database and it would be incredibly slow obviously.

So instead of doing this I now do:

Connection<LiveStreamPayload>()
    .Name("liveStreams")
    .Resolve(c =>
    {
        var offset = ConnectionUtils.OffsetOrDefault(c.After, 0);
        var liveStreams = liveStreamService.GetAudios();
        var newLiveStreams = liveStreams.Take(c.First.Value + (offset + 1));
        var count = liveStreamService.GetCount();
        var splitLiveStreams = ConnectionUtils.ToConnection(newLiveStreams, c, 0, count);

        return splitLiveStreams;
    });

public IQueryable<LiveStream> GetAudios() { return _repository.GetAll(); }

So this doesn't load the entire 25k records anymore, only the data I need.

Is this the correct way to do this or am I doing something wrong? It was very hard to figure this out and I feel it may be the wrong way.

I'm using Entity Framework core.

Thanks.

[furier]

There should be a ConnectionUtils.ToConection method for IQueryable, applying the paging arguments before passing it to EntityFrameworkCore to prevent putting potentially thousands or millions of records into memory.

[myty]

There should be a ConnectionUtils.ToConection method for IQueryable, applying the paging arguments before passing it to EntityFrameworkCore to prevent putting potentially thousands or millions of records into memory.

I'm working on this very thing. It's for IEnumerable but should theoretically work for IQueryable. I'm thinking this project could use an EFCore sample project to really highlight its usefulness. The EnumerableSliceMetrics class still needs a little more cleanup and some Benchmarks to see what the performance implications are between EnumerableSliceMetrics and ArraySliceMetrics.

I may have gotten a little carried away with updating the Star Wars sample project, but it reads so nice!

I suppose I could open a PR at this point and clean up a little more, but check it out: https://github.com/graphql-dotnet/relay/compare/master...myty:feature/enumerable-connections