[ ] - Time consumption documentation. How long does each function take to process? How long is the process for ep file start to finish? Same for individual file.
[x] Following the use of temporary files, create test_mode = TRUE for only testing one year of data. This will only create temporary files for one year to save space in sourcedev. The idea is to test one year and if everything is working then we can proceed to running all years as the assumption is that the code will work because it should be identical.
[x] #1018
[x] Remove person_id (including comments) from extracts code #1006
[x] handle the issue of file-in-use for extracts tests writing to xlsx #1001
[x] clear out test folder in hscdiip and decide how many old files we would like to keep
[x] create subfolders for ep and ind tests
[x] contact social care team to check if we are extracting data correctly #1000 - meeting set for 25/09/24
[ ] create a marker linking CH to AE #1008
[x] add tests and warning messages for reading social care raw data #1002
[x] check distinct death data by chi while keeping chi==NA records #999
[x] this join_by is not 100% accurate. Consider use ep_file_row_id to join #998
[ ] Investigate 1819 processing #997
[x] Bug - Geography differences #996
[x] Change request completed for substance misuse flag #836
[x] Update documentation: which data are quarterly/annual updated, which data we need to request by ourselves. This includes #955
[ ] Update documentation. We can create flow charts to help visualise the process. This was particularly helpful for understanding the deaths processing due to the complexities.
[ ] Check the substance misuse flag in the December update and add this to the update email to make people aware this CR has completed
[ ] add to draft email - extracts on DD data is based up to XXX month, Also for SC data- Data based up to XXX period
Issues:
check_year_valid()#995test_mode = TRUEfor only testing one year of data. This will only create temporary files for one year to save space in sourcedev. The idea is to test one year and if everything is working then we can proceed to running all years as the assumption is that the code will work because it should be identical.person_id(including comments) from extracts code #1006