Eatyourpeas/issue45

[eatyourpeas]

Overview

Following NPDA meeting the team requested that all csv details that were uploaded be stored in the details, whether invalid data or not. This PR therefore refactors the validation functions on csv_upload to store error messages and state in the Patient and Visit models to reflect this. It then also updates the views and templates to:

1. split patients in listview as having valid or invalid records - patients with any invalid records either in the Patient model or any related Visit models show as red.
2. any invalid records in the patient list have an information icon in the affected field and a pop up detailing the error(s)
3. any invalid records in the visit list the affected category is coloured red and a popup details the error on hover
4. if the invalid visit is opened and corrected, it can be successfully saved and the invalid status is reverted to valid.

This means users can upload their data and get immediate feedback on which patients and visits need correcting, and can do this in the UI, without uploading any further spreadsheets.

Code changes

This is a complicated PR and has:

1. is_valid (booleanfield) and errors (JSONField) added to the Patient and Visit models
2. refactor of the csv_upload function with some error fixes to update_or_create records, storing if valid or not, and if not, the associated errors as a list against affected fields
3. Update the VisitCategory function to add a flag for whether has_errors
4. Refactor patient_list table to have 2 header rows - one for valid, one for invalid with styling.
5. Refactor the Patient listview context to order by valid and invalid records, as well as totals of each in the list
6. Refactor the Visit listview template table to make the category pills small, and recolour to red if invalid, with popups to show invalid fields
7. add new errors_for_category template filter tag to return error strings to the popups in the visit list view
8. update black and gunicorn versions because of security risks highlighted by dependabot.
9. add an dummy sheet with invalid data for testing
10. remove log files apart from the index file from git tracking

Documentation changes (done or required as a result of this PR)

Please describe any changes to documentation here. Will address this in a subsequent commit

Related Issues

closes 45

[eatyourpeas]

New Patient list view with popups showing errors present against a given patient

[eatyourpeas]

New patient visits with errors for categories viewable on hover