2.3

[diegoabeltran16]

feature/expense-reports

[diegoabeltran16]

Roadmap

The roadmap for implementing the feature/expense-reports in the next cycle focuses on creating a robust and user-friendly reporting system for the Passive Expenses Bot. This roadmap incorporates best practices, project architecture, and details from the provided files. The goal is to ensure the feature is both scalable and efficient. Here’s a detailed plan:

Scope and Objectives

Goal: Develop a comprehensive reporting system that allows users to generate expense reports based on different filters (e.g., date ranges, categories, or types of expenses). User Commands: /generate_report – generate reports based on user-specified criteria. /list_reports – view past generated reports. /delete_report – delete an existing report.

Feature Requirements

Command Inputs: Users should specify filters like date range (start_date, end_date), expense category (e.g., subscription, utility), and report format (CSV, PDF). Output Formats: Allow for multiple formats, including plain text (for quick viewing in Discord), CSV (for download), and PDF (formatted report). Security Considerations: Ensure reports only access and display data belonging to the requesting user. Data Persistence: Store generated reports temporarily for retrieval and management through commands like /list_reports and /delete_report.

Technical ArchitectureDatabase Design:

Add a reports table in the database to keep track of user-generated reports, including metadata (e.g., user_id, creation_date, filters, file_path). Review the current database schema (referencing /mnt/data/src-utils-db.md) and design the migration file (/mnt/data/src-migrations-migrate_database.md) to support this new table structure. Command Handling: Implement new command handlers (/mnt/data/src-commands-) for: /generate_report: To process the user's filters and generate reports. /list_reports: To retrieve and display the list of available reports for the user. /delete_report: To delete a specific report based on user input. Report Generation Utility: Develop a utility function for generating reports. This function will pull data from the database, apply filters, and format the output in various formats (text, CSV, PDF). This should be placed under the utils module (/mnt/data/src-utils-). File Storage: Store generated reports securely, ensuring file paths and permissions are handled appropriately. Consider adding file path validations for security.

Command Flow and Interaction Design

Review the existing command interaction patterns in /mnt/data/6_interaction-flow.md to ensure consistency. The /generate_report command should have a dynamic interaction flow: User inputs /generate_report. Bot responds with options (filters) and formats. User selects or inputs values, and the bot confirms and generates the report. The /list_reports and /delete_report commands should use interactive menus for selection, maintaining a coherent user experience.

Development Plan

Phase 1: Database & Utility Implementation Implement database migration for the reports table. Develop the core utility for report generation, including data extraction and formatting modules. Phase 2: Command Development Implement /generate_report, ensuring it integrates with the new utility and database schema. Implement /list_reports and /delete_report, allowing users to manage reports seamlessly. Phase 3: Testing & Integration Perform unit and integration testing for each command and the report generation utility. Test the command flow in a local Discord environment to ensure smooth interaction and output.

Best Practices and Standards

Modular Design: Keep the report generation logic separate from command handling to ensure reusability and modularity. This follows the modular structure outlined in /mnt/data/9_project-structure.md. Error Handling: Implement robust error handling in the utility and command layers to manage potential issues like invalid dates or database connection failures gracefully. Localization Support: Ensure that messages and errors are compatible with the multilingual support system introduced in feature/multilingual-support-expansion. Refer to /mnt/data/src-utils-lang.md to integrate translated messages into the new commands. Logging: Integrate logging for report generation activities to help with troubleshooting and audit trails. Follow patterns established in /mnt/data/3_best-practices.md for consistent logging.

SWOT Analysis

Strengths: . Enhances user engagement by providing useful insights. . Leverages existing database and command patterns for consistent development. Weaknesses: . Complexity in formatting reports across multiple formats (text, CSV, PDF). . Potential performance concerns if queries and report generation are not optimized. Opportunities: . Opportunity to introduce more analytics features in the future (e.g., monthly summaries, visual graphs). . Can expand the feature to include automatic monthly reports for users. Threats: . Potential scalability challenges as the user base grows; might need optimizations in data handling. . Security concerns regarding file storage and access must be carefully managed.

Testing and QA Process

Develop unit tests for each module, particularly focusing on the report generation utility to verify the accuracy of data aggregation and formatting. Perform integration tests for the commands in a local Discord environment. Ensure edge cases (e.g., invalid date ranges, no data matching filters) are handled gracefully. Conduct a code review to ensure adherence to standards and best practices outlined in /mnt/data/7_algorithms-and-math.md.

Documentation

Update the documentation for new commands and utilities following the existing structure. Ensure clear usage examples are provided. Update architecture and project structure documentation to include the new reports module and the associated files and commands (/mnt/data/4_architecture.md, /mnt/data/5_components.md).

[diegoabeltran16]

Files and folders

To implement the feature/expense-reports while following the KISS (Keep It Simple, Stupid) and YAGNI (You Aren't Gonna Need It) principles, we need to maintain simplicity and avoid unnecessary complexity. We should build only what is necessary to fulfill the core functionality of expense reporting. Here’s how we will structure the new and updated files, ensuring minimalistic and efficient implementation:

Updated Project Structure for feature/expense-reports

1. New Files

a. Commands Folder

src/commands/generate_report.py
Command to handle /generate_report.

• Responsibilities:
  • Parse user input (filters and formats).
  • Call the report generation utility.
  • Provide feedback to the user in Discord (text report in-channel or link to download CSV/PDF).

src/commands/list_reports.py
Command to handle /list_reports.

• Responsibilities:
  • Fetch a list of reports for the user.
  • Display the reports with options to download or delete.

src/commands/delete_report.py
Command to handle /delete_report.

• Responsibilities:
  • Accept user input for selecting which report to delete.
  • Remove the report from the database and file system.

b. Utilities Folder

src/utils/report_generator.py

• Responsibilities:
  • Fetch data from the database based on user filters (date range, category, etc.).
  • Format the data into specified formats (plain text, CSV, PDF) using Python libraries like csv for CSV and reportlab for PDF.
  • Return the generated file path for CSV/PDF or a string for text reports.

src/utils/file_manager.py

• Responsibilities:
  • Save generated report files to the filesystem.
  • Delete files when reports are no longer needed.
  • Maintain file paths securely to prevent unauthorized access.

c. Database Migrations Folder

src/migrations/add_reports_table.py
Database migration script to add the reports table.

• Responsibilities:
  • Create a new table with fields: report_id, user_id, creation_date, filters, and file_path.
  • Include rollback functionality in case the migration fails.

2. Updated Files

a. Commands Folder

src/commands/init.py

• Register the new commands (/generate_report, /list_reports, /delete_report).

b. Utilities Folder

src/utils/db.py

• Add methods for:
  • Inserting new reports (insert_report).
  • Fetching reports based on user input (get_reports_by_user).
  • Deleting reports (delete_report).

src/utils/lang.py

• Add necessary strings for multilingual support related to the report commands (e.g., messages for report generation, errors).

3. New Tests

a. Tests for Commands

tests/test_commands/test_generate_report.py
Test for the /generate_report command, covering:

• Different combinations of filters and formats.
• Validations (e.g., invalid date range, unsupported format).

tests/test_commands/test_list_reports.py
Test for the /list_reports command, ensuring:

• It retrieves and displays reports correctly.
• Handles cases where no reports exist for the user.

tests/test_commands/test_delete_report.py
Test for the /delete_report command:

• Deletion of existing reports.
• Handling of attempts to delete non-existent reports.

b. Tests for Utilities

tests/test_utils/test_report_generator.py
Test cases for generating reports:

• Data aggregation and formatting correctness.
• File creation for CSV and PDF.

tests/test_utils/test_file_manager.py
Test cases for file management, including:

• Proper saving and retrieval of files.
• Security checks for file path management.
• File deletion functionality.

4. Database Update

src/migrations/migrate_database.py

• Update this file to call the new migration (add_reports_table.py) as part of the migration process.

Summary: Adhering to KISS and YAGNI

KISS: We avoid complex abstractions and keep each component focused on its primary responsibility. For instance:

• The report_generator utility is focused solely on fetching and formatting data.
• The file_manager utility handles file storage and deletion without unnecessary complexity.
• Commands are simple handlers that connect user input to utilities.

YAGNI: We build only the necessary features:

• We create only the essential utilities (report_generator, file_manager) needed for report generation and file handling.
• We avoid premature optimization or building features that aren't immediately needed (e.g., advanced analytics or visual graph generation) until proven necessary.

This structure ensures a minimalistic and efficient addition to the existing project while adhering to our good practices.

[diegoabeltran16]

Modular and Progressive Development Plan for Feature Implementation

For a modular and progressive development approach, we want to build the new feature step by step, ensuring each component is functional and tested before moving on to the next. This helps in maintaining code quality, catching issues early, and keeping the development cycle efficient. Here's the best order to work on these files:

1. Database and Utility Foundation

Start with the database and core utilities as these form the foundation of the feature. Without these components, the commands and tests would not function correctly.

a. Database Migration

• File: src/migrations/add_reports_table.py
• Purpose: Establish the necessary database schema for reports.
• Steps:
  • Define the reports table with the required fields (report_id, user_id, creation_date, filters, and file_path).
  • Implement migration and rollback logic.
• Testing:
  • Run the migration locally to ensure the table is created correctly.
  • Verify the rollback process works if needed.

b. Database Utility Update

• File: src/utils/db.py
• Purpose: Extend the database utility to interact with the new reports table.
• Steps:
  • Add functions: insert_report, get_reports_by_user, and delete_report.
  • Ensure each function interacts properly with the database schema.
• Testing:
  • Write basic tests in tests/test_utils/test_database.py to confirm these functions work as expected.

2. Core Utility Development

Once the database foundation is in place, build the core utilities responsible for the feature's functionality.

a. Report Generation Utility

• File: src/utils/report_generator.py
• Purpose: The heart of the reporting feature; it fetches data based on user filters and formats the output.
• Steps:
  • Implement the method to query the database using filters (e.g., date range, categories).
  • Add support for generating plain text, CSV, and PDF reports.
• Testing:
  • Start with simple tests in tests/test_utils/test_report_generator.py for each format to ensure data is correctly aggregated and formatted.

b. File Manager Utility

• File: src/utils/file_manager.py
• Purpose: Manage file storage and deletion for generated reports.
• Steps:
  • Implement functions for saving and retrieving files securely.
  • Implement file deletion based on the file path.
• Testing:
  • Create tests in tests/test_utils/test_file_manager.py to ensure files are stored, retrieved, and deleted correctly.

3. Command Implementation

With the core utilities functional and tested, move on to implementing the commands. This ensures that the commands can correctly leverage the underlying utility functions.

a. Generate Report Command

• File: src/commands/generate_report.py
• Purpose: Handles /generate_report command, tying together user input, report generation, and output delivery.
• Steps:
  • Parse user inputs (filters and format).
  • Call report_generator to produce the report and use file_manager for storage if necessary.
  • Return appropriate messages or links to the user.
• Testing:
  • Write tests in tests/test_commands/test_generate_report.py to validate various inputs and ensure correct report generation and user response.

b. List Reports Command

• File: src/commands/list_reports.py
• Purpose: Manages the /list_reports command, displaying available reports for the user.
• Steps:
  • Query the database for reports belonging to the user.
  • Format and display the list in a user-friendly manner.
• Testing:
  • Write tests in tests/test_commands/test_list_reports.py to verify report listing, including cases where no reports exist.

c. Delete Report Command

• File: src/commands/delete_report.py
• Purpose: Handles the /delete_report command for removing reports.
• Steps:
  • Parse user input to identify which report to delete.
  • Remove the report using both the database (delete_report function) and the file system (file_manager utility).
• Testing:
  • Create tests in tests/test_commands/test_delete_report.py to confirm that reports are correctly deleted from both the database and file system.

4. Localization Update

After building the core functionality, ensure that localization is correctly implemented for multilingual support.

a. Localization File Updates

• Files: src/utils/lang.py and src/lang/en/reports_commands.json
• Purpose: Add the necessary strings for commands and user messages (e.g., success, error messages).
• Steps:
  • Update localization functions to retrieve these new strings.
  • Ensure language files for supported languages include these strings.

5. Command Registration and Documentation

a. Command Registration

• File: src/commands/__init__.py
• Purpose: Register the new commands (/generate_report, /list_reports, /delete_report).
• Steps:
  • Import the new commands and ensure they are correctly instantiated in the bot.
• Testing:
  • Verify that the commands are accessible and functional in the bot’s development environment.

b. Documentation Update

• Files:
  • docs/commands.md
  • docs/database.md
• Purpose: Document the new commands and update the database schema documentation.
• Steps:
  • Add usage examples and update any relevant diagrams or schemas.

6. Comprehensive Testing and Integration

a. Run Full Test Suite

• Ensure all tests for new utilities and commands pass.
• Test for edge cases (e.g., empty report lists, invalid filters) to confirm stability.

b. Integration Tests

• Test the entire flow from generating a report to listing and deleting it, verifying that everything works together smoothly.

[diegoabeltran16]

1 Database Migration and Feature Development Summary

Step 1: Database Migration and Schema Setup

File Created: src/migrations/add_reports_table.py

Purpose: Establish the schema for the reports table, which will store information about reports logged by users.

Table Structure:

• report_id: Auto-incremented primary key for each report.
• user_id: ID of the user who created the report.
• creation_date: Timestamp indicating when the report was created (defaults to the current timestamp).
• filters: A text field to store any filters used when generating the report.
• file_path: The path to the file where the report is stored.

Migration Logic:

• A migration function was implemented to create the table if it doesn't already exist.
• A rollback function was created to drop the table if needed for a rollback scenario.

Testing:

• The migration was run locally to verify that the table was created successfully.
• The rollback function was tested to ensure it could cleanly drop the table when needed.

Step 2: Database Utility Implementation

File Updated: src/utils/db.py

Purpose: Extend the utility functions in this file to support the reports table.

Functions Implemented:

• insert_report(): Inserts a new report into the database and returns its ID.
• get_reports_by_user(): Retrieves all reports created by a specific user.
• delete_report(): Deletes a report based on its ID.

Testing:

• Tests were written to verify that each function interacts properly with the database.
• Each function was called through commands in the bot, confirming their functionality.

Step 3: Implementation of Bot Commands

Files Created/Updated:

• src/commands/log_report.py
• src/commands/get_reports.py
• src/commands/delete_report.py

Commands Implemented:

• log_report: Logs a report based on user input (report name and file path) and stores it in the database.
• get_reports: Fetches and displays all reports created by the user invoking the command.
• delete_report: Deletes a report using its ID provided by the user.

Challenges Faced:

• Adjustments were made to ensure commands passed the correct parameters to utility functions.
• Issues with intent configurations in the Discord Developer Portal were resolved by enabling necessary intents (e.g., message content and members).
• Debugging involved correcting data formats and ensuring all functions returned the expected values.

Testing and Debugging:

• Each command was tested in a Discord environment:
  • The log_report command successfully logged reports and confirmed their entries in the database.
  • The get_reports command was tested and adjusted to properly format and display data retrieved from the database.
  • The delete_report command was tested to ensure that reports could be deleted using the correct ID.
• The tests confirmed that the commands interacted with the database as expected.

Step 4: Configuration and Permissions Setup

Discord Intents Configuration:

• In the code, intents were defined correctly to handle message content, guilds, messages, and members.
• The intents were also configured in the Discord Developer Portal to ensure the bot had the necessary permissions to function properly.

Bot Initialization:

• The bot was initialized using the intents and command prefix configured in the config.yaml file.
• The bot’s token was managed securely, and login issues were handled with appropriate error messages.

Step 5: Verification and Final Testing

Full Feature Test:

• The database migration was tested in its entirety: creating, updating, retrieving, and deleting records.
• The bot commands were verified in a live Discord server, and all functions performed as expected with no major errors.
• Error handling was added for cases where the commands or database interactions might fail, ensuring the bot provides meaningful feedback to users.

Debugging Common Errors:

• Adjustments were made to resolve issues like missing intents, incorrect command parameters, and database connection problems.
• The bot was updated to ensure that commands handled edge cases (e.g., trying to delete a report that doesn't exist) gracefully.

Summary of Achievements in This Step

• Database Schema Established: The reports table schema was correctly set up to store necessary report data.
• Database Utility Functions: Functions were implemented and tested to handle CRUD (Create, Read, Update, Delete) operations for reports.
• Bot Commands Implemented and Tested: Commands for logging, retrieving, and deleting reports were successfully developed, tested, and debugged.
• Bot Configuration Verified: Proper configuration and permissions were set for the bot to interact with Discord and users efficiently.
• Fully Functional Features: The bot now has a stable and functional foundation to handle user reports, ensuring a solid baseline for building additional features.

With these steps completed, the bot's database and utility foundation is fully functional, and the related commands work as expected.