Real-Dev-Squad / website-backend

The backend code for all our website-related apps
https://api.realdevsquad.com/
MIT License
53 stars 253 forks source link

[RFC] Full Text Search In Firestore #1409

Open manish591 opened 1 year ago

manish591 commented 1 year ago

Summary

The user search API is not returning correct responses. This is because Firebase queries are case-sensitive. Also, firebase doesn't support full-text search.

This RFC aims at finding possible solutions for full-text search in Firestore.

Possible Workarounds

Listing a few workarounds, that will resolve the problem along with the pros and cons of each.

1. Use a custom Js filter

Fetch all the user's collections. Then based on the query provided by the user filter the required users.

Example

  const filteredUserByQuery = users.filter((user) =>
     user.username.toLowerCase().includes(query.search.toLowerCase())
  );

This functionality will perform a full-text search on the username and finds the users that match the query. But, one of the disadvantages of this functionality is that it increases firebases usage time. Fetching all users for every query and then filtering1 is very expensive.

2. Use a third-party full-text search solution

Firebase also recommends using third-party services for performing full-text searches. https://firebase.google.com/docs/firestore/solutions/search

Other suggestions

1. Migrating all the usernames to lowercase to counter the Firestore case-sensitive filtering and sorting

Using this we will mark all the usernames in lowercase and then we can use Firestore queries to search users based on the query.

Suppose the user model looks like this.

  {
    username: "Manish",
    // other details
  }

According to our current functionality defined in search API, if the query=M, then the results will be an empty array. Because of the case-sensitive sorting and filtering of Firestore.

After Marking user-name to lowercase:

  {
    username: "manish",
    // other details
  }

Now, if we send the query=M. The results will be the users with m in their username.

But, there is a problem with this approach

Suppose the user model looks like this.

  {
    username: "_manish",
    // other details
  }

This time the username contains an underscore in front i.e. _manish. Now, if the query=M and the username we have changed to lowercase still it will not return anything because Firestore doesn't perform a full-text search.

Please share your thoughts about this.

sahsisunny commented 1 year ago

We are already doing this in the tasks API. https://discord.com/channels/673083527624916993/688816539775926272/1140194709508202546