ed-pilots-network / backend

Project to consume the Kafka, process the messages and provide an API to access the data
Apache License 2.0
12 stars 5 forks source link

Merge `commodity` and `commodity_whitelist` Tables to Improve Query Performance #157

Closed pveeckhout closed 3 months ago

pveeckhout commented 3 months ago

Description

In our current database schema, we have two separate tables: commodity and commodity_whitelist. These tables are interconnected and used to create an intermediary view named validated_commodity_view to facilitate our queries. This setup, while functional, has been identified as a bottleneck in our system, contributing to slower query performances due to the overhead associated with maintaining and querying through the intermediary view.

Objective

The primary goal of this ticket is to merge the commodity and commodity_whitelist tables into a single, efficiently structured table. This consolidation aims to eliminate the need for the validated_commodity_view and, as a result, speed up our query processes by reducing complexity and improving data retrieval times.

Expected Benefits