Elasticsearch implementation

[shpomp]

Description

Implementing elastic search on a selection of routes:

1. ES_CLOUD_ID and ES_API_KEY are placed in .env, allowing to connect and use the elastic database.
2. In src/server/config/elastic.js, the connection to elastic is established and exported (client). Index name for the index to be created and used is also declared an exported (usersIndex);
3. A scrip is created and placed in src/server/api/lib/utils/syncES.js, which should be run once with npm run es:setup. It creates the index and its mapping in elastic database and copies existing DB users to elastic documents under the created index.
4. Client and usersIndex are imported and used in users.controller. Functions getUsers, getUsersByKeyword, createUser, deleteUser are refactored to be replaced or supplemented with elastic functionality.

Fixes #124

How to test?

Don't. 🙅🏽

Checklist

• [x] I have performed a self-review of my own code
• [x] I have followed the name conventions for CSS Classnames and filenames, Components names and filenames, Style filenames, if you are in doubt check the project README.MD and here https://github.com/HackYourFuture-CPH/curriculum/blob/master/review/review-checklist.md
• [x] I have commented my code, particularly in hard-to-understand areas, if you code was simple enough mark the box anyway
• [x] I have made corresponding changes to the documentation, if you code was simple enough mark the box anyway
• [x] This PR is ready to be merged and not breaking any other functionality

[shpomp]

@orhantoy ☀️ Current challenges:

1. Placing a subfield of keyword in mappings (syncES:28) so that I could sort by keyword does not work, so I also hid the sorting (users.controller:20)
2. syncES is out of sync itself, tries to move from DB to ES first and then create the index. So I have one of them commented out at a time for now (syncES:72)
3. it seems like in syncES, the client keeps hanging. I tried client.close() and it failed terribly - inside or outside .then()
4. fullName is search-as-you-type, but works as a keyword (you could run the project and check http://localhost:3000/create-board, and then click on add members button)

[shpomp]

@orhantoy ☀️ Current challenges:

1. Placing a subfield (...)

⬆️ Worst case, 1. and 3. could be left unsolved and worked around. A

[shpomp]

Thank you very much @orhantoy ❤️ I will work on all of it tomorrow.

[shpomp]

Seems like all listed issues are fixed now and all is working. 👯

I realized I that now the search became case-sensitive, which it was not before., As I understand per my efforts in research - because of 'keyword'. Or maybe because I don't quite get where and how to apply the autocomplete in search?

I read through quite a few documents and threads and tried to change the mapping, but of course it did not work 😅 I tried some variations of something like this:

  await client.indices.create({
    index: index,
    body: {
      settings: {
        analysis: {
          analyzer: {
            case_insensitive_analyzer: {
              type: "custom",
              filter: [
                "lowercase"
              ],
              tokenizer: "keyword"
            }
          }
        }
      },
      mappings: {
        dynamic: 'strict',
        properties: {
          fullName: {
            type: 'keyword',
            analyzer: "case_insensitive_analyzer",
            fields: {
              autocomplete: {
                type: 'search_as_you_type',
              },
            },
          },
          email: {
            type: 'keyword',
          },
        },
      },
    },
  });

The case-insensitivity is not a necessity to have but would be nice to understand and, if possible, to fix.

Also, while looking into this, I started thinking if I should introduce wildcard, or even use it instead of keyword (to allow partial string match), but I am sure it is highly likely I just don't understand mapping very well still. 🙏🏽

[shpomp]

@orhantoy Thank you once again very much! ❤️ I think it is ready and good to go!

[shpomp]

@senner007 @Midnighter Would you like to take a look before the merge? I have added a short description to explain a bit what is going on.

We don't have an elastic account for the project yet (I am using mine, which is on trial, with 2 weeks left). @Midnighter mentioned that Heroku offers an elastic instance, @orhantoy do you think we could/should use that?