jschalk / jaar

Cross-cultural communication engine built on customizable economic systems. Produces personalized agendas, calendars, finance reports for users.
0 stars 1 forks source link

175 enable documentation process #392

Closed jschalk closed 4 days ago

jschalk commented 4 days ago

Summary by Sourcery

Enable the documentation process by introducing new brick formats and corresponding table classes for group, node, and road bridges. Refactor the staging transformer to support multiple pidgin categories and update tests to cover new scenarios.

New Features:

Enhancements:

Tests:

sourcery-ai[bot] commented 4 days ago

Reviewer's Guide by Sourcery

This pull request implements a significant refactoring of the world module's ETL process, focusing on improving the bridge functionality and file organization. The changes include renaming methods for better clarity, introducing new brick formats, and restructuring test files.

ER diagram for new brick tables

erDiagram
    br00115AbstractTable {
        String face_id PK
        Integer event_id PK
        String acct_id
        String fiscal_id
        String inx_group_id
        String otx_group_id
        String owner_id
    }
    br00116AbstractTable {
        String face_id PK
        Integer event_id PK
        String acct_id
        String fiscal_id
        String inx_node
        String otx_node
        String owner_id
    }
    br00117AbstractTable {
        String face_id PK
        Integer event_id PK
        String acct_id
        String fiscal_id
        String inx_road
        String otx_road
        String owner_id
    }
    br00115AbstractTable ||--o{ br00115HoldTable : ""
    br00116AbstractTable ||--o{ br00116HoldTable : ""
    br00117AbstractTable ||--o{ br00117HoldTable : ""
    br00115AbstractTable ||--o{ br00115StageTable : ""
    br00116AbstractTable ||--o{ br00116StageTable : ""
    br00117AbstractTable ||--o{ br00117StageTable : ""

Class diagram for ZooAggToStagingTransformer refactoring

classDiagram
    class ZooAggToStagingTransformer {
        +zoo_dir: str
        +pidgin_category: str
        +legitmate_events: set[TimeLinePoint]
        +jaar_type: str
        +transform()
        +insert_staging_rows(stage_df: DataFrame, brick_number: str, zoo_brick_path: str, df_columns: list[str])
        +get_sheet_staging_name() str
        +get_otx_obj(x_row) str
        +get_inx_obj(x_row, missing_col: set[str]) str
    }

File-Level Changes

Change Details Files
Refactored the ETL process by splitting and renaming methods
  • Renamed zoo_agg_to_appt_staging to zoo_agg_to_acct_staging
  • Created new ZooAggToStagingTransformer class to handle different staging types
  • Added support for bridge_acct_id and bridge_group_id categories
  • Introduced pidgin_category and jaar_type parameters for better type handling
src/f10_world/world.py
Added new brick format definitions and models
  • Added br00115 for group bridge format
  • Added br00116 for node bridge format
  • Added br00117 for road bridge format
  • Updated brick category references to include new formats
  • Added corresponding hold and stage table definitions
src/f09_brick/brick_models.py
src/f09_brick/brick_config.py
src/f09_brick/brick_formats/brick_format_00115_group_bridge1_v0_0_0.json
src/f09_brick/brick_formats/brick_format_00116_node_bridge1_v0_0_0.json
src/f09_brick/brick_formats/brick_format_00117_road_bridge1_v0_0_0.json
Reorganized and renamed test files for better organization
  • Split test_world_jungle_to_zoo.py into multiple focused test files
  • Created separate test files for different ETL stages
  • Renamed test methods to match new method names
  • Updated test assertions to reflect new functionality
src/f10_world/test/test_world_etl0_jungle_to_zoo.py
src/f10_world/test/test_world_etl1_zoo_otx_appt.py
src/f10_world/test/test_world_etl1_zoo_otx_group.py
src/f10_world/test/test_world_etl1_zoo_otx_node.py
src/f10_world/test/test_world_etl1_zoo_otx_nub.py
src/f10_world/test/test_world_etl1_zoo_otx_road.py
Improved naming consistency in bridge and pidgin modules
  • Renamed node_id related methods to node for clarity
  • Updated parameter names to be more specific (e.g., otx_word to otx_road)
  • Standardized method naming across bridge implementations
src/f08_pidgin/bridge.py
src/f08_pidgin/pidgin.py
src/f08_pidgin/test_pidgin/test_pidginunit_crud_node.py

Tips and commands #### Interacting with Sourcery - **Trigger a new review:** Comment `@sourcery-ai review` on the pull request. - **Continue discussions:** Reply directly to Sourcery's review comments. - **Generate a GitHub issue from a review comment:** Ask Sourcery to create an issue from a review comment by replying to it. - **Generate a pull request title:** Write `@sourcery-ai` anywhere in the pull request title to generate a title at any time. - **Generate a pull request summary:** Write `@sourcery-ai summary` anywhere in the pull request body to generate a PR summary at any time. You can also use this command to specify where the summary should be inserted. #### Customizing Your Experience Access your [dashboard](https://app.sourcery.ai) to: - Enable or disable review features such as the Sourcery-generated pull request summary, the reviewer's guide, and others. - Change the review language. - Add, remove or edit custom review instructions. - Adjust other review settings. #### Getting Help - [Contact our support team](mailto:support@sourcery.ai) for questions or feedback. - Visit our [documentation](https://docs.sourcery.ai) for detailed guides and information. - Keep in touch with the Sourcery team by following us on [X/Twitter](https://x.com/SourceryAI), [LinkedIn](https://www.linkedin.com/company/sourcery-ai/) or [GitHub](https://github.com/sourcery-ai).