Migrate to typesense as search backend

[famasya]

Overview

As discussed at #290, we can move to typesense to gain instant-search capability. The search experience should be easier and faster for the user, as it provides typo-tolerant and near real-time search results.

Implementation details

Typesense server will hosted by trustmedis and its contents are crawled hourly by wbw-gsheets-crawler. As discussed by @zainfathoni , there are three indices for now:

• wbw-gsheets-database
• wbw-gsheets-faq
• wbw-gsheets-telemedicine

More indices can be added by making MR into crawler repository.

Fortunately, we have typesense-instantsearch-adapter as a wrapper of Algolia's instantsearch react components. No need to write components from scratch.

API KEY information

CLICK ME

HOST : public-api.trustmedis.id
PROTOCOL : https
PORT : 443
API KEY : FByczfHEjsTCihgkkYlq2YbAgUKMoyVP

Code

A dirty implementation to instantiate typesense class should be looked like:

  const searchAdapter = new TypesenseInstantSearchAdapter({
    server: {
      apiKey: API_KEY, // search-only API key
      nodes: [
        {
          host: "public-api.trustmedis.id",
          port: "443",
          protocol: "https",
        },
      ],
    },
    additionalSearchParameters: {
      queryBy: ["pertanyaan" , "jawaban"], // for wbw-gsheets-faq index
    },
  });

Current Tasks

Unit & integration tests

• [x] #376
• [x] #382
• [x] #458
• [x] #413
• [ ] #468
• [x] #383
• [ ] #459

End-to-end tests

• [x] #380
• [x] #381
• [ ] #414

Implementation

• [x] #328
• [ ] #377
• [ ] #378
• [ ] #379

[mazipan]

Just my two cents, the priority should be reordered. Do not try to create a reusable component on the first place. It will come into unnecessary complexity, an early optimization.

Just create for spesific page first, then we can think about centralized later.

[zainfathoni]

Just my two cents, the priority should be reordered. Do not try to create a reusable component in the first place. It will come into unnecessary complexity, an early optimization.

Just create for any specific page first, then we can think about centralized later.

Agreed, Mas @mazipan. I have updated the task list accordingly.

Thank you for creating this epic with beneficial documentation, Mas @famasya! I think we can treat #328 as a POC for this epic. That way, we don't waste any code further just for the sake of proofing the concept. 😁

[famasya]

Well, I've also written a quick implementation of this approach in this commit from this stateResults reference mas @zainfathoni . Hope it helps.

PS: pardon for bad styling, :))

[resir014]

Friendly reminder that this issue need to be expedited due to the reasons mentioned in https://github.com/kawalcovid19/wargabantuwarga.com/issues/197#issuecomment-886702751:

Found more issues with itemsjs: turns out it's using a vulnerable version of lodash:

A quick peek through the bundle analyser shows that it's coming from itemsjs. We missed out on this initially, because turns out the dependency is inlined within itemsjs itself:

cc: @zainfathoni @mazipan @ekamuktia @famasya

[mazipan]

I just don't think we need this kind of Search Library in the front-end 😂

But ya, from our previous decision we will go with the TypeSense. Can we accelerate the progress?

Currently, I don't know, what can I help on this. Since I am not familiar with our search.

So we need to work with many developer in a single PR? Or do we need to split the issue and the PR?

[resir014]

Current progress is in #401 and #540. But unfortunately progress on these has kind of stagnated for a bit.

[ekamuktia]

Sorry, I wasn't feeling well so I coudn't work on it. Just pushed #401 update. Basic feature is done, but I still have some problems on the setting & testing.

First, for the setting as I mentioned here https://github.com/kawalcovid19/wargabantuwarga.com/pull/401#issuecomment-888246450

And for testing, since the data is from index, mock data can't be initialized and passed as props. Tried to find mock for typesense/algolia but still can't find it so I change the test using hardcoded text -> could result in failed test if the data is updated.

[mazipan]

I think we need to add more developer to help @ekamuktia.

@zainfathoni can you help on the testing part once we get a full flow?

I will try to take a look the PR tomorrow, looking if I can help on this 😂

We have a good start from @ekamuktia

[zainfathoni]

Sure, I'll try to unblock the testing part later today. 👌 Thanks, Mbak @ekamuktia for implementing it this far. 😁🙏