elastic / call-for-meetups

17 stars 6 forks source link

Using Elasticsearch to minimise the number of backend services #10

Closed mousikosk closed 2 years ago

mousikosk commented 4 years ago

Description

Our client analyses emails and also provides a dashboard to the relevant departments of their clients to allow monitoring of the aggregate results and filter-based search down to individual emails. They approached us for the development of a new, more responsive, version of the dashboard. This included the use of an underlying document database (instead of their existing relational database) that would perform well in providing search results and aggregations and would scale along with the increasing amount of processed emails. Due to the nature of the data, Elasticsearch seemed like an ideal candidate. We ended up using the Elasticsearch pipeline to ingest the incoming JSON documents (the products of the email analysis) in the desired form. We also decided to use the same cluster to store the dashboards that users create, as well as to log their sessions to provide analytics regarding the usage of the application. In that way, Elasticsearch turned out to be the only service required to power the React.js front-end application, apart from a intermediary API and the existing Keycloak server our client was already using for authentication. This experience lead to the adoption of Elasticsearch for other projects as well.

Speaker Bio

Kostis is a front-end developer at Logicea, based in Athens, Greece. His main focus is developing web and mobile applications with React. He has been working with Elasticsearch for a little more than a year.

jkakavas commented 4 years ago

Hi @mousikosk ! Nice to meet you, and thanks for the submission. We have our next meetup planned for December the 4th and there is a possibility that we have a presentation slot open for that. Otherwise, we definitely have a spot for the first meetup of 2020 (TBD but probably late January).

Will you be available for December 4th ( we'll know about slot openings by the end of next week ) ?

mousikosk commented 4 years ago

Hello @jkakavas and thank you for the hospitality at the previous meetups. December 4th works fine for me. The duration of the presentation can be modified as well if you would rather host more lightning talks this time. Looking forward to hearing from you.

jkakavas commented 4 years ago

I'll get back to you by end of next week. If we can't do it now, I'll get in touch with you right after the December event so that we can schedule it for the next in January. I think it would be worth having this as a full 30-45 mins session instead of cramping it into a lightning talk !

jkakavas commented 4 years ago

@mousikosk Hi there, and apologies for the late reply as I have been traveling for the last 10 days. Will you still be willing to present in our next meetup that is scheduled for the 4th of December ? We have a proper slot, so 45 mins. Let me know what you think

mousikosk commented 4 years ago

Hi @jkakavas. No worries at all. Yes, 4th of December sounds great. I'll start working on a 45' version. Please let me know of anything that you may require from me (i.e. a shorter description for the schedule).

jkakavas commented 4 years ago

Awesome! A short (2,3 sentence) description for the event page would be great sometime this week. Thanks again for volunteering!

mousikosk commented 4 years ago

Abstract: At Logicea we decided to use Elasticsearch to upgrade the analytics dashboard of our client that does email threat analysis for its customers. In this session we will present the development process as well as the progress after six months in production. We will also focus on how Elasticsearch ended up being the only data store for all the needs of this project.

mousikosk commented 4 years ago

Meetup still thinks I'm a spammer so I'm posting the link to the slides here https://www.dropbox.com/s/sjiwspua9sxbr2v/logicea-elastic-greece.pdf

xeraa commented 4 years ago

Meetup has suffered a lot in recent spam waves and they have tightened down the rules a lot now. @jkakavas I think you (or any other organizer) can post it in the group and I guess we can then close this issue. Thanks everyone :)

jkakavas commented 4 years ago

I'll take care of it, thanks again @mousikosk !