Remove lazy loading Scheduler and add proper Pagination

[Rieven]

We need to figure out if the scheduler is available or not before we do a lazy loading call. A lazy loading call would end up being executed in the template context where we have a harder time handling any errors. Since doing a health check before hand is an extra request, we should not do so, it just makes everything more expensive, and might still end up erroring out when we do an actual data request.

I think we should not make this (or any other restfull api request) lazy loading. If these requests are needed, they should be done in the model (or view), and we should handle any errors and exceptions in that scope too. Our targeted restfull api's dont offer anything that resembles lazy-loading anyway, so there's no gain in having these calls be lazy or deferred to when the template is rendering.

_Originally posted by @underdarknl in https://github.com/minvws/nl-kat-coordination/pull/1957#discussion_r1415199955_

[Rieven]

Scheduler API had already built-in limits and offset that can be used for Pagination

• Count is already implemented which can be used for total tasks
• A limit of 1 will fetch only 1 task for that page.
• next attribute is already defined which we can use to get the next page, which will move the offset automatically to get the next part of the list.

We can let the view handle the pagination which uses Django pagination. See paginate_queryset to get more info: https://ccbv.co.uk/projects/Django/5.0/django.views.generic.list/ListView/

• Or create our own using the next and previous of Schuduler attrs. I will go for Django pagination as it handles more things for the view.

[underdarknl]

We dont need to know if the scheduler is available. We just need to handle Exceptions gracefully if they happen. We also dont need to know how many pages there are before doing the first page-call, Which is atm where django trips up, as it tries to 'slice' a page from the set, and in doing so needs to know the length of the set.

[Rieven]

We dont need to know if the scheduler is available. We just need to handle Exceptions gracefully if they happen. We also dont need to know how many pages there are before doing the first page-call, Which is atm where django trips up, as it tries to 'slice' a page from the set, and in doing so needs to know the length of the set.

I think in this case we can create our own view to handle pages. There is a part where we need to know how many objects there is in total to calculate how many pages there is. In the frontend you will see all the page numbers, this is based on the total objects. The counting happens now like this:

We need to make an easier way to query total rows for tasks. With this we can also know how many splits there are based on the max items per page. This does not mean that we already are querying those pages. We prepare them to show in frontend. Bases on the get request it will fetch only the part necessary, which included the limit and offset. The offset will move when you move up the page numbers.

So the process should be:

• Fetch total objects(tasks) with a simple query. This is also helpful in the frontend "Show .... objects of ...."
• Calculate how many pages there are based on the max items per page.
• Create the get urls built-in for each page. This will then check and query only for the part requested.(limit and offset will be different for each page)