hwchase17 / langchain-hub

3.26k stars 267 forks source link

Confluence is enforcing paging (with default = 100). How to load all pages from given space? #38

Open iCUE-Solutions opened 1 year ago

iCUE-Solutions commented 1 year ago

While the example limits the retrieval to 50 documents, Confluences has a page limit of 100. How does one retrieve all pages from a larger space.

The loader.paginate_requests = True seems to only work within the loader limit?

loader = ConfluenceLoader( url="https://yoursite.atlassian.com/wiki", username="me", api_key="12345" ) documents = loader.load(space_key="SPACE",limit=50)