Latency optimization for vineyard client/server interaction

[sighingnow]

Created from https://github.com/v6d-io/v6d/issues/1873#issuecomment-2071294061

[github-actions[bot]]

/cc @sighingnow, this issus/pr has had no activity for for a long time, could you folks help to review the status ? To suppress further notifications,

• for issues,
  • if it is waiting for further response from the reporter/author, please help to add the label requires-further-info,
  • if you have already started working on it, please add the label work-in-progress to the issue,
  • if this issue requires further designing discussion and not in current plan, or won't be fixed, please add the label requires-further-discussion or wontfix to the issue,
• for pull requests,
  • if you are still working on it and it is not ready for reviewing, please convert this pull request as draft PR,
  • if you have decided to hold this development on, please add the requires-further-discussion label to the pull request. Thanks!