search

cagov / caldata-mdsa-caltrans-pems

CalData's MDSA project with Caltrans on Performance Measurement System (PeMS) data
https://cagov.github.io/caldata-mdsa-caltrans-pems/
MIT License
7 stars 0 forks source link

created the macro function to replace all county code with county nam… #443

Closed mmmiah closed 4 weeks ago

mmmiah commented 4 weeks ago

This PR have implemented macro function to replace county code to county name as part of the issue mentioned at #421 for all mart table that is applicable

mmmiah commented 4 weeks ago

@JamesSLogan , could you please check this github CI error. I have lint and fix in dbt where there was no error, but it pops up error in pre-commit. Could you please check the criteria for pre-commit?

L: 56 | P: 1 | ST03 | Query defines CTE "detector_status_by_station_with_metadata" but | does not use it. [structure.unused_cte] == [/home/runner/work/caldata-mdsa-caltrans-pems/caldata-mdsa-caltrans-pems/transform/models/marts/diagnostics/diagnosticsdetector_daily_detail.sql] FAIL L: 8 | P: 1 | ST03 | Query defines CTE "detector_status" but does not use it. | [structure.unused_cte] == [/home/runner/work/caldata-mdsa-caltrans-pems/caldata-mdsa-caltrans-pems/transform/models/marts/diagnostics/diagnosticsdetector_monthly_by_station.sql] PASS == [/home/runner/work/caldata-mdsa-caltrans-pems/caldata-mdsa-caltrans-pems/transform/models/marts/geo/geocurrent_detectors.sql] FAIL L: 5 | P: 1 | ST03 | Query defines CTE "current_detectors" but does not use it. | [structure.unused_cte] == [/home/runner/work/caldata-mdsa-caltrans-pems/caldata-mdsa-caltrans-pems/transform/models/marts/imputation/imputation__detector_imputed_agg_five_minutes.sql] FAIL L: 7 | P: 6 | ST03 | Query defines CTE "imputation_five_mins" but does not use it. | [structure.unused_cte] == [/home/runner/work/caldata-mdsa-caltrans-pems/caldata-mdsa-caltrans-pems/transform/models/marts/imputation/imputationdetector_summary.sql] PASS == [/home/runner/work/caldata-mdsa-caltrans-pems/caldata-mdsa-caltrans-pems/transform/models/marts/performance/performanceagg_district.sql] PASS == [/home/runner/work/caldata-mdsa-caltrans-pems/caldata-mdsa-caltrans-pems/transform/models/marts/performance/performanceagg_yearly_city.sql] PASS == [/home/runner/work/caldata-mdsa-caltrans-pems/caldata-mdsa-caltrans-pems/transform/models/marts/performance/performanceagg_yearly_county.sql] FAIL L: 12 | P: 1 | ST03 | Query defines CTE "spatial_metrics" but does not use it. | [structure.unused_cte] == [/home/runner/work/caldata-mdsa-caltrans-pems/caldata-mdsa-caltrans-pems/transform/models/marts/performance/performanceagg_yearly_freeway_type_direction.sql] PASS == [/home/runner/work/caldata-mdsa-caltrans-pems/caldata-mdsa-caltrans-pems/transform/models/marts/performance/performancestation_aadt_k_value.sql] FAIL L: 1 | P: 6 | ST03 | Query defines CTE "aadt" but does not use it. | [structure.unused_cte] == [/home/runner/work/caldata-mdsa-caltrans-pems/caldata-mdsa-caltrans-pems/transform/models/marts/performance/performance__station_metric_agg_recent_one_week.sql] FAIL L: 1 | P: 6 | ST03 | Query defines CTE "station_meta" but does not use it. | [structure.unused_cte] == [/home/runner/work/caldata-mdsa-caltrans-pems/caldata-mdsa-caltrans-pems/transform/models/marts/performance/performancestation_metrics_agg_daily.sql] FAIL L: 6 | P: 6 | ST03 | Query defines CTE "daily" but does not use it. | [structure.unused_cte] == [/home/runner/work/caldata-mdsa-caltrans-pems/caldata-mdsa-caltrans-pems/transform/models/marts/performance/performance__station_metrics_agg_monthly.sql] FAIL L: 1 | P: 6 | ST03 | Query defines CTE "monthly" but does not use it. | [structure.unused_cte]

JamesSLogan commented 4 weeks ago

@mmmiah It looks like dbt cloud was running a slightly more recent version of sqlfluff. I tested the version they're using locally and it seems to have resolved the issue. There are a few other minor issues that are still causing CI to fail, though.

mmmiah commented 4 weeks ago

@JamesSLogan it seems only failure is in dbt failure now due to bottleneck PR: 8:34:11 46 of 59 ERROR creating sql incremental model dbt_cloud_pr_15_443_performance.int_performance__bottleneck_delay_metrics_agg_daily [ERROR in 4.15s]. I dropped this PR and re-run, still encountering the same issues. Any idea?

JamesSLogan commented 4 weeks ago

Yes! #445 has the fix for that issue - you could implement it in your branch or pull from main once #445 is merged (waiting on CI)