department-of-veterans-affairs / va.gov-team

Public resources for building on and in support of VA.gov. Visit complete Knowledge Hub:
https://depo-platform-documentation.scrollhelp.site/index.html
283 stars 204 forks source link

[Rated Disabilities] Update discrepancy checks to be more stringent #77304

Open jerekshoe opened 8 months ago

jerekshoe commented 8 months ago

Summary

Once we have reached a point where the length discrepancy logging is not producing any more results, we should modify the length check to manually check the IDs of all of the rated disabilities returned by EVSS and Lighthouse (after we have filtered out inactive ratings and those with unwanted decision values). This should ensure that we are returning the exact same list of ratings as EVSS did, not just the same number of ratings

Context

if lh_response_length != evss_response_length log_length_discrepancy((lh_response_length - evss_response_length).abs) end



### Tasks
- [ ] Update length checks to check that the IDs of all ratings match instead of checking for length
- [ ] Update the `message_type` log field to be `lh.rated_disabilities.id_discrepancy`
- [ ] Update the `revision` log field to have a value of `1`
- [ ] Update the query DD monitor [here](https://vagov.ddog-gov.com/monitors/172081?view=spans) to check for the new `message_type`

### Acceptance Criteria
- [ ] DD logs are produced when we identify cases where LH and EVSS are not returning the same disability ratings
- [ ] There is a DD monitor that generates alerts whenever a discrepancy has been logged to DD
HeatherWidmont commented 8 months ago

Waiting on LH to update their service to return correct disability ratings

HeatherWidmont commented 8 months ago

@jerekshoe will check in with them today to see if there's an update

HeatherWidmont commented 8 months ago

They said they should have a fix on Thursday

HeatherWidmont commented 8 months ago

@jerekshoe is meeting with LH today to discuss what needs to happen with this. Likely won't be a fix until next week

HeatherWidmont commented 7 months ago

LH said they will have a fix out today, Jerek to verify

HeatherWidmont commented 7 months ago

Jerek will verify tomorrow

jerekshoe commented 7 months ago

LH pushed out their latest updated today. Turned the feature toggle up to 10% to see if it produces any results

pmclaren19 commented 7 months ago

Received some logs saying there are discrepancies. Jerek is going to investigate and possibly make new tickets

pmclaren19 commented 3 months ago

6/27 - They pushed out the hotfix this morning. Jerek is going to validate and if all looks good he will start this ticket

Jerek validated the hotfix and this is ready to be worked again. Moving to product backlog to be reprioritized.