Implement 'ResidentActivity' Model in New 'Metrics' App for Efficient Data Aggregation

[brylie]

Overview

Our current Activity data model, while robust, presents challenges in generating charts and reports due to its complex relationships, particularly the ManyToMany relationship between Residents and Activities. To streamline data aggregation and improve performance, we propose creating a new Django application, metrics, featuring a more efficient data model.

Proposed Model: ResidentActivity

• Purpose: Simplify querying and aggregating activity data by flattening the complex relationships.
• Structure:
  • resident: ForeignKey(Resident)
  • activity: ForeignKey(Activity)
  • residency: ForeignKey(Residency) - Linking Resident and Home at the activity time
  • home: ForeignKey(Home) - Derived from residency
  • activity_type: string
  • activity_minutes: int
  • caregiver_role: str

Synchronization with Activity Model

Ensuring data consistency between the Activity and ResidentActivity models is critical. We are considering two approaches:

1. Custom Django Model Manager:

   • Implement Create, Read, Update, Delete (CRUD) operations within a transaction.
   • Automatically handle ResidentActivity entries whenever an Activity record is modified.
   • Rollback changes if any part of the transaction fails.

2. Django Signals:

   • Utilize signals to update ResidentActivity entries upon CRUD operations on Activity records.
   • This approach decouples the logic but still ensures data synchronization.

Task Breakdown

1. Create metrics App: Develop the new Django application within our project.
2. Model Implementation: Define and implement the ResidentActivity model.
3. Data Synchronization Strategy: Choose and implement the most suitable method (Model Manager or Signals) for data synchronization.
4. Testing and Validation: Ensure that the new model accurately reflects changes in the Activity model and that data integrity is maintained.
5. Performance Benchmarking: Assess the impact on performance and the efficiency of data aggregation.

Next Steps

Once the metrics application and ResidentActivity model are established, we will focus on defining specific metrics and developing visualizations to support caregiving in elder care communities.

Conclusion

This initiative aims to enhance our data handling capabilities, making it easier to generate meaningful insights and reports. The success of this implementation will lay the groundwork for advanced analytics in our caregiving project.

[brylie]

We aim to create a flattened Activities table (ResidentActivity) for more efficient data aggregation and visualization in our Django-based caregiving project. This table should synchronize with the main Activity table and related entities. Our current challenge involves deciding on the most effective method to achieve this synchronization.

Proposed Solutions and Their Trade-offs:

1. Django Signals

• Pros:
  • Automatic Sync: Ensures the ResidentActivity table is automatically updated with changes in the Activity table.
  • Decoupling: The logic for updating the flat table is separate from the main application logic, providing a cleaner architecture.
• Cons:
  • Complexity in Debugging: Signals can be challenging to trace, making debugging more complex.
  • Performance Overhead: May introduce additional database operations, potentially impacting performance.

2. Custom Transactions in Model Save Method

• Pros:
  • Immediate Consistency: Guarantees data consistency as both tables are updated in the same transaction.
  • Control: Offers more direct control over data updates.
• Cons:
  • Tight Coupling: Intertwines main data operations with analytics logic, which can be less maintainable.
  • Risk of Long Transactions: Could lead to performance issues due to extended transaction times.

3. Asynchronous Tasks with Celery

• Pros:
  • Asynchronous Processing: Enhances web application performance by handling updates in the background.
  • Scalability: Facilitates horizontal scaling of the data processing workload.
• Cons:
  • Infrastructure Overhead: Requires additional setup for task queues and worker management.
  • Potential Delay: As the process is asynchronous, there might be a delay in reflecting data changes.

[adrienneli104]

I agree, could I be assigned to this issue as well then?

[adrienneli104]

Hey, I finished the metrics app and implemented the stacked bar chart. I manually tested to see if when an activity is created, the resident_activity table is also triggered and the activity is recorded. I wasn't sure how to go about writing tests. I tried using self.client.post to create an activity object. I receive a 200 response but no activity object is created. Could you take a look and help with writing tests?

Also since updating activity isn't currently a feature, I haven't implemented updating resident_activity_table.

https://github.com/adrienneli104/caregiving/tree/aeli_graphs

[brylie]

Great work! Go ahead and open a pull request so I can test the code.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests

[adrienneli104]

Hey I've made a PR and my teammates have made PRs for the other issues. Would be great if you could test it out sometime!