[FEAT]: add a bulk pin feature to select multiple documents at once

[xianghappy]

What would you like to see?

"Can you add a bulk pin feature to select multiple documents at once?"

[timothycarambat]

This is being marked as wontfix as this is intentional. Document pinning is very helpful since it does full document comprehension. However, Document pinning is also a horrible idea for a large number of documents as one or all of the following will occur:

• Your model context size is not big enough for all of the full documents, so they get truncated anyway so the chat completion can complete at least - omitting documents anyway.

• Your model context can fit the documents, but you are burning so many tokens forcing so many useless tokens into the context - resulting in longer inference times and higher costs (if applicable)

• This defeats the purpose of RAG, making citations extraneous if you pin everything. This makes knowing what citations were used nearly impossible.

Simply, document pinning is a solution for some use-cases, which is why it is supported since it can be very handy, but something is wrong if you are trying to pin every document - so the UX is to make it deliberate to discourage this