search

yorkie-team / yorkie

Yorkie is a document store for collaborative applications.
https://yorkie.dev
Apache License 2.0
746 stars 133 forks source link

Implement DB Query for GetDocuments API to improve performance #921

Open hackerwins opened 1 week ago

hackerwins commented 1 week ago

What would you like to be added:

Recently, GetDocuments API was added as a bulk version of the GetDocument API to retrieve multiple documents at once. However, the necessary DB Query for the GetDocuments API has not been implemented yet. It is crucial to implement a database query that can retrieve multiple documents at once to improve performance.

https://github.com/yorkie-team/yorkie/blob/28ef053a5af5a53c2af1d0e043240adc62be1701/server/documents/documents.go#L130-L148

Why is this needed:

To enhance performance by efficiently retrieving multiple documents with a single query from the database.

kokodak commented 20 hours ago

May I try this issue?

sejongk commented 20 hours ago

May I try this issue?

@kokodak Of course. If you have any question, feel free to ask.

krapie commented 20 hours ago

@devleejb I recall that you have the context about this issue regarding CodePair. Is there any context or discord discussion related to this issue?

devleejb commented 20 hours ago

@devleejb I recall that you have the context about this issue regarding CodePair. Is there any context or discord discussion related to this issue?

Sure.

Recently, we changed from calling the GetDocument API multiple times to calling the GetDocuments API once in CodePair. Despite this, there has not been a significant improvement in response time.

The reason is that it is implemented to call GetDocumentSummary multiple times while iterating over the given keys.

Once we resolve this issue, we expect the document loading speed in CodePair to be much faster.

kokodak commented 20 hours ago

Is the problem caused by fewer API calls, but not less DB access via single retrieval queries?

If so, can I solve the issue by preparing queries that can get multiple results in a single query, such as bulk operations?

devleejb commented 20 hours ago

@kokodak

Is the problem caused by fewer API calls, but not less DB access via single retrieval queries?

I didn't fully understand what this means.

If so, can I solve the issue by preparing queries that can get multiple results in a single query, such as bulk operations?

I think this method can solve the issue.

kokodak commented 15 hours ago

I didn't fully understand what this means.

What I meant was that although it reduced the number of API calls by combining multiple GetDocument API calls into a single GetDocuments API call, it seems that the actual implementation of the GetDocuments API is still issuing multiple single queries for documents.

I'll try to resolve the issue in the direction mentioned above. Thanks!