openlibraryenvironment / reshare-analytics

Community query repository for ReShare analytics based on Metadb/LDP
Apache License 2.0
3 stars 2 forks source link

Discrepancies between reports and user interface in core requesting report #27

Closed kristenwilson closed 2 years ago

kristenwilson commented 2 years ago

I've started doing some more detailed checking of the reports, now that we have a lot of data for PALCI. I'm seeing relatively minor discrepancies between the reported totals and the totals visible in the UI.

As a sample, I reviewed NYU (high volume) and PCOM (low volume). The results of this analysis is below.

requester reqs cancelled unfilled received filled_ratio
nyu - report 5076 103 1437 3215 0.61
nyu - ui 5164 102 1518 3212 0.62
pcom - report 55 2 16 34 0.58
pcom - ui 62 2 17 31 0.5

I'm wondering if it would be possible to pull the actual list of the requests associated with these counts for both PCOM and NYU so we can compare them.

kristenwilson commented 2 years ago

Comments from Sean based on review of PCOM data:

For total requests, I'm counting the number of unique request IDs with the REQ_VALIDATED state, which gets us to 55 for pcom. I suspect the additional 7 that the UI is counting are the following: 10b147c7-bc88-4f59-a83b-2a3553007663 6a6dfee3-4b40-4ff3-80ed-90142d1c6d43 7dbf7758-bcc3-4e51-a0e9-658afe193fd5 c157cc0b-6ec2-47ad-a2a7-9d27528a2295 cf3f1d1e-97cd-4eac-a22d-ad16b005e226 e42d650f-df51-4516-bf63-611672d967aa edadbb20-771d-4085-8384-a088f0112fa5 6 of them have a message stating that the lookupPatron call failed and indicate an unknown user.

For received, I think I am double-counting. When an item has REQ_CHECKED_IN as its current state and someone uses the Message feature, ReShare creates a new transaction where the from state and to state are both REQ_CHECKED_IN. There are 3 examples of this in the pcom data, thus creating the 34/31 discrepancy. I will try to update the code to only count from REQ_SHIPPED to REQ_CHECKED_IN as received instead of what I'm doing now.

For unfilled, PPCO-50 is the culprit. According to this data, it's been sitting in the Supplier Identified state since 10/29.

Action items related to this:

kristenwilson commented 2 years ago

I've confirmed that the fix for determining whether a request has been received has resulted in producing correct received counts.

The discrepancy between total requests is explained by the reports not counting unvalidated requests.

The discrepancy between unfilled requests will be fixed when the following bug is resolved in ReShare: https://openlibraryenvironment.atlassian.net/browse/PR-1237.

At this time, all counts in this report are working as expected, so I will close this issue.