donorImportHistory failureCount=0 even when there are failures

[mmelchers]

Describe the bug When importing a file with donors with incorrect HLA, then in the [donorImportHistory] table the failureCount field = 0 even when there are failures according to the [DonorImportFailures] table

To Reproduce Steps to reproduce the behavior:

1. Go to the main DB in live
2. execute select * from [Donors].[DonorImportHistory] where Id = 1380
3. SELECT * FROM [Donors].[DonorImportFailures] where UpdateFile= '/blobServices/default/containers/donors/blobs/missing_donors-D_4068.json' order by Id desc

Expected behaviour The number for failureCount corresponds to the number of unique values for ExternalDonorCode

[zabeen]

This is issue has also been observed in AN LIVE Atlas, which is on version 1.4.2.

[zabeen]

Existing documentation on the data model and changelog shows that the field, DonorImportHistory.FailureCount, is only incremented when the entire file fails to be processed - it does not represent how many individual updates failed within a successful file. This latter information is now being captured in the failures table, and ticket #1051 covers reporting the failed update count within the completion message.

[mmelchers]

@zabeen this does not make sense to me. Why have a counter (FailureCount) in a table when it can only be 0 (for when at least one record has been imported) or 1 (when the entire file has failed). Then it should be a boolean. There is also no easy way to join the donorimportfailures table to the donorimporthistory table. The donorimportFailures table does not a link to the Id of the DonorImportHistory table.

How are we supposed to tell which files had failures and what went wrong for those? Could you please check with us first before closing tickets like this?

[zabeen]

As discussed on standup, the field, Donors.DonorImportHistory.FailedCount, has a different purpose than expected due to ambiguous naming. As part of #1051, a new field, Donors.DonorImportHistory.FailedDonorCount, will be added to capture the requested info. This ticket can now be closed - @mmelchers please close if you agree, thanks.