Implement Pagination to Improve Performance in azure-db

[k-allagbe]

Currently, our search functionality experiences significant latency, likely due to inefficient handling of large result sets. The issue stems from iterating over a substantial number of results in a single query, which we suspect triggers numerous network requests, thereby slowing down the response time. To address this, we propose implementing pagination in our Azure DB search queries.

Tasks

• [x] Modify Azure DB search queries to include pagination logic.
• [x] Perform testing to ensure pagination works.
• [x] Update documentation

Acceptance Criteria

• Azure DB searches are paginated, reducing the load time for large result sets.
• Search response times must be below 500ms.
• Documentation is updated to reflect changes in search functionality and usage.

[rngadam]

@k-allagbe were the acceptance criteria tested and passed? where are the tests results?

[k-allagbe]

@k-allagbe were the acceptance criteria tested and passed? where are the tests results?

https://github.com/ai-cfia/azure-db/actions/runs/7822094547/job/21340435423?pr=17

The tests are here: https://github.com/ai-cfia/azure-db/blob/main/azure-ai-search/tests/test_integration.py