Bug: Rule 8775Q - Not returning errors

[SLornieCYC]

Describe the bug Rule is returning no errors in CIN Validator whereas I get 4 errors in DFE portal from the same data.

Not sure what's going on here because the python code looks right to me, but the failing tests are the ones I'd expect to pass. Somehow the rule is inverted?

Reference date https://github.com/data-to-insight/CIN-validator/blob/dc4559d2362e0efacfe47ebb9b4b3d538c7d0094/cin_validator/rules/cin2022_23/rule_8775Q.py#L160-L162

Older than 25 at reference date https://github.com/data-to-insight/CIN-validator/blob/dc4559d2362e0efacfe47ebb9b4b3d538c7d0094/cin_validator/rules/cin2022_23/rule_8775Q.py#L112-L115

Closure date is after 25th birthday or not closed https://github.com/data-to-insight/CIN-validator/blob/dc4559d2362e0efacfe47ebb9b4b3d538c7d0094/cin_validator/rules/cin2022_23/rule_8775Q.py#L126-L135

[WillLP-code]

Hi @SLornieCYC I've been staring at this a long time now. Let me just check I have the logic right, we want to fail rows for people who are over 25 and who have had their CIN plan close after they are 25, of haven't had it close? That looks to be the case in the failing rows?

[SLornieCYC]

Oops. Sorry @willLPD2I you're right those tests are failing correctly. I must have got mixed up while trying to understand why my data isn't failing. I do have data that fails in the DFE portal but doesn't fail in CIN Validator though:

[WillLP-code]

Oops. Sorry @willLPD2I you're right those tests are failing correctly. I must have got mixed up while trying to understand why my data isn't failing. I do have data that fails in the DFE portal but doesn't fail in CIN Validator though:

@SLornieCYC Is there any chance these people are EXACTLY 25 when their CIN plan closes? That's the only way I've been able to recreate this bug so far?

[SLornieCYC]

They were both 25 but not exactly 25 (i.e. didn't close on their birthday). 25th birthday was before the closure, either the month before or earlier in the same month.

[WillLP-code]

They were both 25 but not exactly 25 (i.e. didn't close on their birthday). 25th birthday was before the closure, either the month before or earlier in the same month.

@SLornieCYC Amazing thanks, I'll keep looking!

[WillLP-code]

@tab1tha I don't know if you want a look but I am really struggling to replicate this bug either using the pytest or the sample data. If I make a child over 25, then give them a con_closure more than 25 years after that, they DO fail. My work is in the branch bugfix/8875Q (https://github.com/data-to-insight/CIN-validator/tree/bugfix/8775Q I don't know if there's a better way to link to branches)

[tab1tha]

Hi both, I'm here now. I'll give this a look.

[tab1tha]

This took me through a number of loops. fun stuff.

Could it be that the DfE tool is using an "or" combination instead of an "and" between conditions? @willLPD2I The data you provided @SLornieCYC has 25 in either column, not both, when the location is flagged. I think that's it.

[tab1tha]

Using the logic we've implemented, those locations wouldn't have been flagged unless both columns had a value of 25 or higher.

[SLornieCYC]

The four rows in my screenshot are two cases each returning two rows (column data for CIN Closure Date and DOB). The "Age At" columns at the end are Excel formulas I added to the output purely to demonstrate that both dates resolve to 25+ (without exposing the actual client data).

[tab1tha]

ahhh I see. Thank you @SLornieCYC To confirm, do you mean that all the children flagged are above 25 and have a cinclosure date that is more than 25 years away from their birthdate?

[SLornieCYC]

Yep! And both records are flagged as queries on this rule in the DFE portal.

They were both 25 but not exactly 25 (i.e. didn't close on their birthday). 25th birthday was before the closure, either the month before or earlier in the same month.

[tab1tha]

Ouch.

Another angle I took was to check whether it had to do with the way pandas calculated date offsets. However, I haven't come across anything that'll make me to believe it is out of normal. I'll keep looking.

[WillLP-code]

A huge part of me is relieved that you haven't found why we can't make our data fail straight away 😅

[SLornieCYC]

So I've managed to run the python from this rule locally, using the dates from my real data, and both children do get returned in the two issues dfs for me there.. but they don't make it through to the front-end on the deployed tool. Does that mean the problem is somewhere else in the rule metadata or somewhere between the rule and the front-end?

[WillLP-code]

@SLornieCYC this is relieving as no matter what I did, the back-end code returned them in error as expected! I wonder if it's somewhere between the issues dfs and the user report that it gets lost

[tab1tha]

Hi both, some extensive comparison between the backend and the frontend showed some differences in the number of data rows processed. The backend produced 1699 rows but the frontend processed 1565 rows. We found that it was due to a difference in the columns used when we de-duplicated the full_issue_df dataset. That has been fixed now and a release will be done at the end of today or early tomorrow morning.

In fixing that data difference, this might have been fixed too. Let's see if this bug goes away with the new release.

[tab1tha]

@SLornieCYC is this fixed now?

[SLornieCYC]

Sadly not. My two records are still not being flagged in the CIN validator despite being picked up by the DFE tool.

[tab1tha]

That's interesting. Data is no longer lost between the backend and frontend. Could this be that this is now a backend problem?. If you do this again, does that backend still give you the correct output?

[No pressure at all, I know how busy this period is. You don't need to do it right now.]

So I've managed to run the python from this rule locally, using the dates from my real data, and both children do get returned in the two issues dfs for me there.. but they don't make it through to the front-end on the deployed tool. Does that mean the problem is somewhere else in the rule metadata or somewhere between the rule and the front-end?

[tab1tha]

This issue might be due to the same reason as https://github.com/data-to-insight/CIN-validator/issues/430

[SLornieCYC]

Aha. My two child records are now returning in FE!! Guess it must have been fixed by one of those latest changes here or on FE.

[tab1tha]

Wow, Today is such a good day 💯