scottcowan / dfe-edubase2-soapapi-client

DFE Edubase2 Client
GNU General Public License v3.0
0 stars 0 forks source link

Getting all the establishments takes too long #1

Open scottcowan opened 7 years ago

scottcowan commented 7 years ago

Page size is fixed to 25 items and the requests take 30 mins to get all the responses.

We could do something around parallel requests to speed this up.

scottcowan commented 7 years ago

This could work although we'd want to limit the number of requests

            var tasks = new List<Task<FindEstablishmentsResponse>>();
            for (var i = 1; i < response.PageCount; i++)
            {
                filter.Page = i;
                tasks.Add(RequestEstablishments(filter));
            }

            Task.WaitAll(tasks.Cast<Task>().ToArray());

            foreach (var task in tasks)
            {
                list.AddRange(task.Result.Establishments);
            }