Create llm cache block builder only when the user requests an update.

[vegetableysm]

Describe your problem

Currently, we will fetch and get a block object if user request a cache block(update or query) and create a builder from this block object. Then delete the block object. There exist an extra memory copy process when create a new builder. We can keep the block object to server the query request from the user and create a new builder only when the user requests an update.

[github-actions[bot]]

/cc @vegetableysm, this issus/pr has had no activity for for a long time, could you folks help to review the status ? To suppress further notifications,

• for issues,
  • if it is waiting for further response from the reporter/author, please help to add the label requires-further-info,
  • if you have already started working on it, please add the label work-in-progress to the issue,
  • if this issue requires further designing discussion and not in current plan, or won't be fixed, please add the label requires-further-discussion or wontfix to the issue,
• for pull requests,
  • if you are still working on it and it is not ready for reviewing, please convert this pull request as draft PR,
  • if you have decided to hold this development on, please add the requires-further-discussion label to the pull request. Thanks!

[sighingnow]

Has this issue been fixed or not?

[vegetableysm]

Has this issue been fixed or not? No. We pull block object only when we access the block, but we will rebuild the builder. We can change it to build the builder only for update and serve the query via block in the future.