lpciferri
commented
6 years ago This looks awesome!
re: Assignment parameters
- One other parameter from Jed's case assignment tool is CAVC cases. Brianne set the arbitrary number that judges should only have max of 3 CAVC cases at a time. ~I'm asking Jed to confirm where they come from.~ These come from the gen pop pool. We can decide if it makes sense to keep a cap of 3 CAVC cases in our model or not. I don't see the harm in it.
- % hearing cases
- could we see how many cases are in each judges hearing backlog & in docket range to help define this? might be a @cmgiven question. it could be possible to stack judges with their hearing cases and give other judges gen pops.
- I think it'd be great to try out a few different percentages here.
re: AMA appeal preparation
- The Board's policy is to proportionally allocate appeals from all dockets to judges and attorneys (policy still to be defined and approved). This allocation will take into account what you've spec'd out, plus some other AMA related parameters (e.g. wait 90 days for evidence to be submitted) @cmgiven is going to model out a few scenarios of how this could be implemented, attempting to answer these questions:
- What is the steady state, and when is it, for Direct Reviews to be decided in one year?
- How do we get direct reviews to that steady state, when allocating proportionally?
- How do we discount for legacy appeals and by how much? (already have some numbers on this)
- What will be the proportion of dockets at steady state?
- When does supporting this proportionality + 1 year turnaround become unsustainable?
- Right now, Cheri is under the impression that all judges and attorneys will work both legacy and AMA appeals. It makes sense that any judge who has held a legacy hearing will have to continue working legacy appeals, so perhaps all judges will have to work both legacy and AMA. I think we should move forward thinking that all attorneys and judges will have to work both for now.
- Because of Cheri's desire to decrease judges' legacy hearing backlogs, it is possible that she may agree to a proposal that only judges with small hearing backlogs are scheduled for AMA hearings, so their AMA hearing backlogs don't increase at the expense of legacy. TBD.
re: Specialty case teams
- From what I understand, specialty cases do not need to go to judges before they go to attorneys. When specialty cases come up in docket order, could we automatically assign them straight to attorneys?
- I agree with never assigning appeals with specialty case team issues in the general auto-assignment process, reserving them exclusively for the specialty teams
re: User Interface
- Still on board with Option 1!
re: Rollout
- Agree with rolling out in conjunction with judge queue. Could even be after judge release 2 - allowing judges to assign to attorneys - so we can get feedback on finishing an entire list of cases to assign before getting the next batch.
anyakhvost
cmgiven
amprokop
Context
See https://github.com/department-of-veterans-affairs/appeals-design-research/issues/645
Overview
We'll develop a Case Storage auto-assignment model that:
Implementation
Assigning a case from case storage to a judge
VACOLS::CaseDocket.docket_date_of_nth_appeal_in_case_storageAssignment parameters
By law, we need to prioritize AODs, and the board has slated hearing cases as a priority. Currently, AOD decision output is somewhere around 20%, which we see as a healthy figure. However, it appears that judges and case storage staff manually keep case category throughput in balance. If we develop a relatively simple auto-assignment model, but keep certain parameters configurable, we can control the throughput of various case categories centrally and monitor throughput to ensure it meets compliance and production goals.
Proposed configurable parameters:
Provisional auto-assignment logic
Auto-assignment will happen for one judge at a time.
Find and assign (batch size (Max % AOD)) AOD cases to a judge. Find and assign the minimum of (batch size - already assigned cases) and (batch size (Max % hearing)) hearing cases to a judge. Find and assign (batch size - already assigned cases) gen pop cases to the judge.
Example
Batch size is 30, AOD % is 20, Hearing % is 50.
A judge requests a batch of 30 cases.
The judge has 25 hearing cases awaiting decision, there are 500 AOD cases currently on docket, and 7000 genpop cases. The judge requests a batch of cases from case storage.
30 .2 = 6, so 6 AOD cases are assigned to the judge. 30 .5 = 15 is the max # of hearing cases the judge can receive in a batch. The judge has 30 - 6 = 24 cases that can still be assigned, so 15 hearing cases are assigned to the judge. 30 - 6 - 15 = 9, so 9 genpop cases are assigned to the judge.
Monitoring
To validate our model is behaving as we intend and so we can watch for any needed adjustments, we'll need to track the flow of all types of cases to judges separately. However, in the end, the top-line metric we care about is decision output, not case assignment. We'll want to also ensure descision output by case type is surfaced and easily available. We might opt to do this in Datadog at first, but ideally we'd make this easy to view for BVA employees as well, and ensure Tableau or a business metric tool can surface necessary data.
AMA appeal preparation
The business requirements for AMA case storage auto-assignment are not yet clear. It may be the case that separate judge/attorney teams exclusively work AMA appeals.
In this case, we could just create a separate case storage auto-assignment code path with its own configuration variables.
Additionally, we want to make it easy to track case storage assignments. We'll defer a final decision until later, but we'll consider creating a CaseStorageAssignment record for all appeals, so we ensure we're tracking assignments in Caseflow.
Specialty case teams
Business requirements for specialty case teams are not clear yet, but adding specialty case teams may significantly increase the complexity of this effort.
A few options:
These questions probably deserve their own separate spec.
Manual assignment
It's likely that there will be assignment hiccups. When we initially roll out case storage auto-assignment, we should ensure we can manually tweak case assignments as needed. At first, this can just be through a method exposed to the Rails console that a system admin engineer can run. This manual method must be able to support switching cases from judge to judge.
User interface
We'll weigh two different approaches for triggering the assignment of cases from case storage to judges.
Option 1: Self-service button
When a judge loads Caseflow Queue and views the appeals assigned to them, we could present the judge a "Fetch more from Case Storage" button. Upon clicking a button, we'd assign a batch of case storage cases to the judge, and they'd appear in the Queue list view immediately after assignment is complete.
Goal: Judges should understand the methodology behind automatic case allocation
Goal: Cases should not be allocated to attorneys and judges on PTO/leave
Goal: Cases are only allocated to active staff
Goal: Cases should not sit on judges plates to assign or to sign for long times
Option 2: Auto-assign cases to judges from case storage
We wouldn't need any UI for this option. We could approach a full automation option in several ways. Two that come to mind:
A recurring job could run on some interval and ensure each judge has at least N cases on their plate to assign to attorneys.
Each time a judge assigns an appeal to an attorney, if the judge has less than N cases on their plate, we add a batch of new cases from case storage
However, a fully automated option will create some challenges and risks.
Goal: Judges should understand the methodology behind automatic case allocation
Goal: Cases should not be allocated to attorneys and judges on PTO/leave
Goal: Cases are only allocated to active staff
Goal: Cases should not sit on judges plates to assign or to sign for long times
Test Plan
Using FACOLS, we'll be able to thoroughly test the Case Storage assignment to judge logic locally.
Rollout Plan
We can develop this in parallel with the judge queue, but we shouldn't roll it out until we roll out the judge queue. We could test this process change with one judge at first by putting it behind a feature toggle.