Continuation of https://github.com/pantherdb/pango/issues/40, but now focus on the loader and api changes
As part of our ongoing efforts to enhance data clarity and improve user experience, we need to refactor the way we handle annotations. Instead of presenting them as a flat list, we have to group annotations by genes. This aggregation will not only provide a clearer view of the data but also improve the overall performance by reducing redundant information and make pagination better. Now that evidence is not part of the initial display, this will reduce the api buggage.
Tasks:
[x] UI: more on https://github.com/pantherdb/pango/issues/40 Adjust the user interface to accommodate and display the grouped annotations. Ensure that the user experience remains intuitive and the changes align with our previous discussions
[x] API: Update the API endpoints responsible for serving annotation data. This will involve changes to both the data retrieval methods and the response structure to ensure that the frontend receives data already grouped by genes. We were using complicated queries before as we were aggregation the annotations table, but not it is just getting the rows
[x] ES : Modify the ElasticSearch queries and indices to support the retrieval of annotations grouped by genes. This might require re-indexing or adjusting the way we query for annotations and also will discuss evidence type
[x] LOADER: Update the data loader scripts to preprocess and load the annotations in a manner that supports the new grouped structure. Ensure data integrity and check for any potential data loss during this transformation. @dustine32 idk how to check if this new genes table (collection) has all the annotation terms and slim terms as before.
Extras
While implementing these changes, it's vital to maintain backward compatibility wherever possible, especially for the API.
Thorough testing is required to ensure that no data is lost or misrepresented during this transition especially the evidence types
Group Annotations by Gene
Continuation of https://github.com/pantherdb/pango/issues/40, but now focus on the loader and api changes As part of our ongoing efforts to enhance data clarity and improve user experience, we need to refactor the way we handle annotations. Instead of presenting them as a flat list, we have to group annotations by genes. This aggregation will not only provide a clearer view of the data but also improve the overall performance by reducing redundant information and make pagination better. Now that evidence is not part of the initial display, this will reduce the api buggage.
Tasks:
Extras