InfuseAI / piperider

Code review for data in dbt
https://www.piperider.io/
Apache License 2.0
480 stars 23 forks source link

[Chore] tuning piperider cli startup speed #869

Closed qrtt1 closed 1 year ago

qrtt1 commented 1 year ago

PR checklist

What type of PR is this?

enhancement

What this PR does / why we need it:

make piperider cli starting faster

codecov[bot] commented 1 year ago

Codecov Report

Merging #869 (97e7b05) into main (f128375) will decrease coverage by 1.63%. The diff coverage is 55.55%.

@@            Coverage Diff             @@
##             main     #869      +/-   ##
==========================================
- Coverage   47.88%   46.25%   -1.63%     
==========================================
  Files          71       75       +4     
  Lines       10003    10106     +103     
==========================================
- Hits         4790     4675     -115     
- Misses       5213     5431     +218     
Files Changed Coverage Δ
piperider_cli/cli_utils/compare_with_recipe.py 0.00% <0.00%> (ø)
piperider_cli/cli.py 65.10% <40.00%> (+3.38%) :arrow_up:
piperider_cli/cli_utils/cloud.py 57.89% <57.89%> (ø)
piperider_cli/configuration.py 61.45% <77.77%> (+0.08%) :arrow_up:
piperider_cli/cli_utils/run_cmd.py 80.55% <80.55%> (ø)
piperider_cli/cli_utils/__init__.py 91.30% <91.30%> (ø)
piperider_cli/event/__init__.py 78.35% <100.00%> (ø)

... and 5 files with indirect coverage changes

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

qrtt1 commented 1 year ago

Before

image

After

image

Notes

import dbt cli cost lots of time, need to refactor it. maybe use a decorator pattern or metaclass?

def dbt_select_option_builder():
    try:
        # For dbt-core >= 1.5.0
        from dbt.cli.options import MultiOption
        return click.option('--select', '-s', default=None, type=tuple, help='Specify the dbt nodes to include.',
                            multiple=True,
                            cls=MultiOption)
    except Exception:
        # For dbt-core < 1.5.0
        return click.option('--select', '-s', default=None, help='Specify the dbt nodes to include.',
                            multiple=True)
sentry-io[bot] commented 1 year ago

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

Did you find this useful? React with a 👍 or 👎