CouncilDataProject / cdp-backend

Data storage utilities and processing pipelines used by CDP instances.
https://councildataproject.org/cdp-backend
Mozilla Public License 2.0
22 stars 26 forks source link

Handle error in event index pipeline when a term is blank #237

Closed conantp closed 1 year ago

conantp commented 1 year ago

Link to Relevant Issue

This pull request resolves an issue I identified and shared on Slack.

Description of Changes

This PR adds an if statement to generate_event_index_pipeline.py. When a sentence contains a "-" character, it winds up being evaluated as a "" value, which then causes an error on line 254:

target_term_index = terms.index(closest_term)

This causes the entire event index operation to fail, and the only option to restore index functionality is to remove the event that caused the issue. This PR allows event index to continue, skipping that term.

codecov[bot] commented 1 year ago

Codecov Report

Merging #237 (04c47b9) into main (703d7f1) will decrease coverage by 0.02%. The diff coverage is 50.00%.

@@            Coverage Diff             @@
##             main     #237      +/-   ##
==========================================
- Coverage   72.12%   72.11%   -0.02%     
==========================================
  Files          50       50              
  Lines        3376     3378       +2     
==========================================
+ Hits         2435     2436       +1     
- Misses        941      942       +1     
Impacted Files Coverage Δ
..._backend/pipeline/generate_event_index_pipeline.py 96.68% <50.00%> (-0.63%) :arrow_down:
evamaxfield commented 1 year ago

Oh woops, can you run just lint / fix the lint error please?

evamaxfield commented 1 year ago

Will fix lint after merge! Thanks!

conantp commented 1 year ago

Sorry I missed this last week! Since it's already merged, I will just remember to lint prior to PR in the future!