Having complex data structures within a relational database can create long, complex queries that hurt SQL. It is common for data to be relational in nature so suited for a SQL database while needing to be searchable in a highly performant nature.
This problem can become more apparent when using microservices and the increase latency involed.
Solution
The common solution to this problem is to mirror the data into an Elasticsearch database that is read-only. This allows for the fast searching and fetching of the data while allowing the rest of the system to use the data as it wants.
This feature would need:
A command to sync the data to Elasticsearch
Ability to have updating the elastic search happen asynchronously. For example, if someone updates an entity in Athena it then gets updated in elasticsearch straight away instead of needing to wait for a timed
The use of elasticsearch aliases to prevent the data from becoming blank during syncs.
Problem
Having complex data structures within a relational database can create long, complex queries that hurt SQL. It is common for data to be relational in nature so suited for a SQL database while needing to be searchable in a highly performant nature.
This problem can become more apparent when using microservices and the increase latency involed.
Solution
The common solution to this problem is to mirror the data into an Elasticsearch database that is read-only. This allows for the fast searching and fetching of the data while allowing the rest of the system to use the data as it wants.
This feature would need:
Example
More information can be found on this problem at https://getparthenon.com/blog/how-to-scale-complex-data