[Lineups] Some lineup bugs (new format, ie 2018+)

Alex-At-Home / cbb-explorer

WIP parser and model for aggregating and playing with CBB stats

Apache License 2.0

0 stars 0 forks source link

[Lineups] Some lineup bugs (new format, ie 2018+) #6

Closed Alex-At-Home closed 4 years ago

Alex-At-Home commented 5 years ago

Using this issue to track the various lineup bugs I've seen that lead to them being dropped

Alex-At-Home commented 5 years ago

Also the v Illinois 2018 (4690813) still has the scores the wrong way round (https://stats.ncaa.org/game/play_by_play/4690813)

Last time round I did that game manually and swapped the LineupEventStats and the ScoreInfo, but post possession stats that's a slightly tricky since you only want to swap the points scored

Not sure what the best solution is, options:

Compare vs the box score (since it seems to happen on a game-by-game basis) ... I think this is the best solution by far
On a lineup-event-by-lineup-event basis use the scoring stats as a sanity check
- (Note there was some score craziness in the @ Purdue game where the score suddenly dropped .. I got lucky and it was in the middle of the event so it was just ignored)

Here's how I fixed it previously:

var good = bad.right.get._1.map(x => { val y = x.opponent_stats; val z = x.team_stats; x.copy(opponent_stats = z, team_stats = y)})

(interestingly i didn't switch the scores, so they'd still be wrong - but it doesn't get used anywhere it's just show)

Alex-At-Home commented 5 years ago

Summary:

Need to somehow calculate the game break from the HTML
~~Use the box score to double check the score is the right way round~~ - fixed by #15
~~Need to find out if it's new format or old format not using the year~~ - fixed by #15
(Maybe ignore X in / X out in the same event?)

Alex-At-Home commented 4 years ago

(EDIT: fixed and committed)

OK more errors:

xXxXxXIN: MaBlazek,AlAndrews
xXxXxXOUT: AlAndrews,SaShewan
xXxXxXON: AlAndrews,AlWittinger,CiRice,CoJoens,J-Ephraim,MaBlazek

This one is nice and easy ... if you have a single player both in and out, then decide whether to include them based on which makes sense!

Alex-At-Home commented 4 years ago

(EDIT: fixed and committed)

There's a bunch of lineup mismatch cases we can fix also:

xXxXxX--------------------------------------------------
xXxXxX CLUMP [+1] size=[1] [?true]
xXxXxX ????????????? FILTERED CANDIDATES [6] vs [1]
xXxXxXIN: GrMitchell
xXxXxXOUT: 
xXxXxXON: AsVeerbeek,GrMitchell,HaWhitish,LeBrown,NiEliely,SaHaiby
...
xXxXxXIN: 
xXxXxXOUT: AsVeerbeek
xXxXxXON: GrMitchell,HaWhitish,LeBrown,NiEliely,SaHaiby
xXxXxXRAW:
xXxXxX-----------
xXxXxX WEIRD CASE - GOOD

I think in this case we can just merge the 2 in/outs

It's not always 1 in/out then 1 out/in as well:

xXxXxX CLUMP [+2] size=[1] [?true]
xXxXxX ????????????? FILTERED CANDIDATES [7] vs [2]
xXxXxXIN: KaRobbins,HaBrown,NaHillmon
xXxXxXOUT: HaThome
xXxXxXON: AkJohnson,ArYoung,DeChurch,HaBrown,KaRobbins,NaHillmon,NiMunger
xXxXxXRAW:
xXxXxX00:08,26-11,CHURCH,DEJA made Free Throw
xXxXxX00:08,27-11,CHURCH,DEJA made Free Throw
xXxXxX-----------
xXxXxX WEIRD CASE 
xXxXxXIN: 
xXxXxXOUT: NiMunger,NaHillmon
xXxXxXON: AkJohnson,ArYoung,DeChurch,HaBrown,KaRobbins

Note in this case I think it should be:

xXxXxXIN: KaRobbins,HaBrown,NaHillmon
xXxXxXOUT: HaThome, NiMunger,NaHillmon

ie
in: KaRobbins, HaBrown
out: HaThome, NiMunger

Alex-At-Home commented 4 years ago

(EDIT: fixed and committed)

Here's a more complicated case:

xXxXxX--------------------------------------------------
xXxXxX CLUMP [+1] size=[4] [?true]
xXxXxX ????????????? FILTERED CANDIDATES [3] vs [1]
xXxXxXIN: Travascio-green,AmCarter
xXxXxXOUT: LaEbo
xXxXxXON: AlSmith~,AmCarter~,KaMcdaniel,SiFrazier~,TePage~,Travascio-green~
(Travascio-green, AmCarter, SiFrazier, TePage)
..
xXxXxXIN: LaEbo
xXxXxXOUT: Travascio-green
xXxXxXON: AlSmith,AmCarter,KaMcdaniel,LaEbo,SiFrazier,TePage
(AlSmith)
..
xXxXxXIN: KaOrtiz
xXxXxXOUT: SiFrazier
xXxXxXON: AlSmith,AmCarter,KaMcdaniel,KaOrtiz,LaEbo,TePage
..
xXxXxXIN: Travascio-green
xXxXxXOUT: AlSmith
xXxXxXON: AmCarter,KaMcdaniel,KaOrtiz,LaEbo,TePage,Travascio-green
..
xXxXxXIN: SiFrazier,AlSmith
xXxXxXOUT: LaEbo,KaOrtiz,Travascio-green
xXxXxXON: AlSmith,AmCarter,KaMcdaniel,SiFrazier,TePage

OK so analysis of who gets mentioned => it has to be KaMacDaniel BUT KaMacDaniel is in the good already .. which means she must also be missing from the IN of the good.

So the subs won't lineup properly, but we can live with that, and still fix the bad lineups...

Alex-At-Home commented 4 years ago

Here's one that's similar to the easy case 2 above, but the final lineup also contains a legit pair which messes everything up

xXxXxX--------------------------------------------------
xXxXxX CLUMP [+1] size=[1] [?true]
xXxXxX ????????????? FILTERED CANDIDATES [6] vs [1]
xXxXxXIN: AbLaszewski,NiBeverley
xXxXxXOUT: KeKarlis
xXxXxXON: AbLaszewski,AlLuehring,ImLewis,KeVaLeeuwen,MaHoward,NiBeverley
xXxXxXRAW:
xXxXxX02:34,30-31,LEWIS,IMANI missed Free Throw
xXxXxX02:34,30-31,TEAM Deadball Rebound
xXxXxX02:34,31-31,LEWIS,IMANI made Free Throw
xXxXxX-----------
xXxXxX WEIRD CASE 
xXxXxXIN: SuGilreath
xXxXxXOUT: MaHoward,AlLuehring
xXxXxXON: AbLaszewski,ImLewis,KeVaLeeuwen,NiBeverley,SuGilreath
xXxXxXRAW:
xXxXxX01:53,31-34,GILREATH,SUZANNE Commits Foul

So one of MaHoward,AlLuehring should be in the first lineup's OUT, but it is impossible to tell which!

(the good/bad news is that these errors are pretty low effect since by definition they only impact 1 lineup)

Alex-At-Home commented 4 years ago

(EDIT: fixed and committed)

These are probably related to the bad names:

xXxXxX CLUMP [+2] size=[1] [?true]
xXxXxX ????????????? FILTERED CANDIDATES [7] vs [2]
xXxXxXIN: Mayer,Butler
xXxXxXOUT: Bandoo,Vital
xXxXxXON: Butler,DeBandoo,KiMcclure,MaMason,MaVital,Mayer,TrClark

Sometimes the names include the first name, other times they don't :(

For teams with no matching surnames, we can fix that

(Is it that we just don't handle 14:37,5-3,MASON,M missed Two Point Jumper)

Alex-At-Home commented 4 years ago

(EDIT: fixed and committed ... EDITEDIT: I think?!)

The logic doesn't currently handle

xXxXxX--------------------------------------------------
xXxXxX CLUMP [+1] size=[3] [?true]
xXxXxX ????????????? FILTERED CANDIDATES [2] vs [1]
xXxXxXIN: DeBandoo
xXxXxXOUT: 
xXxXxXON: DeBandoo,FlThamba,JaButler,KiMcclure,MaMason,MaVital
xXxXxXRAW:
07:38,58-35,VITAL,MARK made Layup
xXxXxX07:38,58-35,MCCLURE,KING Assist
xXxXxX07:24,58-35,THAMBA,FLO Commits Foul
xXxXxX-----------
xXxXxXIN: FrGillespie
xXxXxXOUT: FlThamba
xXxXxXON: DeBandoo,FrGillespie,JaButler,KiMcclure,MaMason,MaVital
xXxXxXRAW:
07:14,60-36,MASON,MAKAI made Layup
xXxXxX06:21,62-39,MASON,MAKAI made Two Point Jumper
xXxXxX06:10,62-39,VITAL,MARK Defensive Rebound
xXxXxX05:54,62-39,VITAL,MARK Turnover
xXxXxX05:53,62-39,MASON,MAKAI Commits Foul
xXxXxX05:42,62-41,TEAM 30 Second Timeout
xXxXxX05:42,62-41,TEAM Timeout
xXxXxX-----------
xXxXxXIN: MaKegler
xXxXxXOUT: MaVital
xXxXxXON: DeBandoo,FrGillespie,JaButler,KiMcclure,MaKegler,MaMason
xXxXxXRAW:
05:33,63-41,KEGLER,MARIO made Free Throw
xXxXxX05:33,64-41,KEGLER,MARIO made Free Throw
xXxXxX05:13,64-41,GILLESPIE,FREDDIE Defensive Rebound
xXxXxX04:39,64-41,MASON,MAKAI missed Free Throw
xXxXxX04:39,64-41,TEAM Deadball Rebound
xXxXxX04:39,65-41,MASON,MAKAI made Free Throw
xXxXxX04:27,65-41,MASON,MAKAI Steal
xXxXxX04:10,65-41,BANDOO,DEVONTE Turnover
xXxXxX04:10,65-41,BANDOO,DEVONTE Commits Foul
xXxXxX03:54,65-41,GILLESPIE,FREDDIE Defensive Rebound
xXxXxX03:37,67-41,BANDOO,DEVONTE made Dunk
xXxXxX03:37,67-41,MASON,MAKAI Assist
xXxXxX03:05,67-43,KEGLER,MARIO missed Free Throw
xXxXxX-----------
xXxXxX WEIRD CASE 
()
xXxXxXIN: MaMayer
xXxXxXOUT: MaMason,KiMcclure
xXxXxXON: DeBandoo,FrGillespie,JaButler,MaKegler,MaMayer
xXxXxXRAW:
03:02,67-43,KEGLER,MARIO Defensive Rebound
xXxXxX02:39,67-43,BUTLER,JARED Turnover
xXxXxX02:31,67-43,MAYER,MATTHEW Commits Foul

Alex-At-Home commented 4 years ago

(EDIT: fixed and committed)

Some play-by-plays have a different name format:

__auc__ CLUMP [+2] size=[1] [?true]
__auc__ [TeamId(Baylor)] vs [TeamId(Ole Miss)] ([Neutral])
__auc__ IN: Mayer,Butler
__auc__ OUT: Bandoo,Vital
__auc__ ON: Butler,DeBandoo,KiMcclure,MaMason,MaVital,Mayer,TrClark
__auc__ RAW:
__auc__ 16:27,2-7,MASON,M missed Three Point Jumper
__auc__16:18,2-7,BUTLER,J Commits Foul
__auc__15:56,5-9,MAYER,M made Three Point Jumper
__auc__15:25,5-11,MAYER,M missed Three Point Jumper
__auc__15:17,5-11,CLARK,T Defensive Rebound
__auc__15:11,5-11,MCCLURE,K Turnover
__auc__14:40,5-11,MCCLURE,K missed Two Point Jumper
__auc__14:03,5-11,MCCLURE,K Commits Foul

Causes both "wrong players" and "size mismatch errors"

Alex-At-Home commented 4 years ago

Cheap fix for 4-player lineups:

__auc__ --------------------------------------------------
__auc__ CLUMP [+-1] size=[5] [?false]
__auc__ [TeamId(Baylor)] vs [TeamId(Texas Southern)] ([Home])
__auc__ IN: 
__auc__ OUT: DaAllen
__auc__ ON: DeBandoo,FlThamba,JaButler,TrClark
__auc__ RAW:
__auc__ 07:52,49-59,MCCLURE,KING missed Three Point Jumper
__auc__ 07:32,51-59,TEAM 30 Second Timeout
__auc__ -----------
__auc__ [TeamId(Baylor)] vs [TeamId(Texas Southern)] ([Home])
__auc__ IN: MaVital
__auc__ OUT: FlThamba
__auc__ ON: DeBandoo,JaButler,MaVital,TrClark
__auc__ RAW:
__auc__ 07:24,51-59,CLARK,TRISTAN Turnover
__auc__ 06:51,53-61,CLARK,TRISTAN made Dunk
__auc__ 06:51,53-61,MCCLURE,KING Assist
^^^^^ DONE!
__auc__ 06:09,55-61,BUTLER,JARED missed Two Point Jumper
__auc__ 05:53,55-61,VITAL,MARK Steal

Alex-At-Home commented 4 years ago

(EDIT: fixed and committed)

Hmm this one is odd

__auc__ CLUMP [+0] size=[3] [?true]
__auc__ valid player codes: [Set(TeLewis, GeConditt, ZiGriffin, TaHorton-tucker, MiJacobson, NiWeiler-babb, MaShayok, ErSteyer, TyHaliburton, LiWigginton)]
__auc__ [TeamId(Iowa St.)] vs [TeamId(Alabama St.)] ([Home])
__auc__ IN: TyHaliburton,GeCondittIv

For some reason the Iv has not been stripped

Hmm OK so in the lineup, he appears as Conditt, George but in the PbP as CONDITT IV,GEORGE Enters Game

Oh gross lol:

__auc__ CLUMP [+0] size=[3] [?true]
__auc__ valid player codes: [Set(XaSneed, ShWilliams, JrBarryBrown, MiMcguirl, DeWade, MaMawien, AuTrice, KaStokes, JaLove, CaDiarra, LeStockard)]
__auc__ [TeamId(Kansas St.)] vs [TeamId(Penn)] ([Neutral])
__auc__ IN: KaStokes
__auc__ OUT: BaBrown
__auc__ ON: AuTrice,CaDiarra,DeWade,KaStokes,XaSneed
__auc__ RAW:
__auc__ 13:23,7-6,TRICE,AUSTIN Blocked Shot
__auc__ 13:21,7-6,WADE,DEAN Defensive Rebound
__auc__ 13:13,7-6,STOKES,KAMAU missed Three Point Jumper
__auc__ 12:55,7-6,DIARRA,CARTIER Commits Foul

JrBarryBrown ... I must have Jr as head so it's not in the right order (and no longer getting filtered out)?

Brown, Jr., Barry in the box score, BROWN,BARRY missed Two Point Jumper in the play-by-play

Alex-At-Home commented 4 years ago

~LOW HANGING TO FIX~ (EDIT: fixed and committed)

Another name combo difference between box score and play:

__auc__ --------------------------------------------------
__auc__ CLUMP [+0] size=[5] [?true] [!Some(Set(UnknownPlayers))]
__auc__ valid player codes: [Set(NiShadd, ShWilliams, JrBarryBrown, MiMcguirl, DeWade, MaMawien, AuTrice, PiMcatee, KaStokes, JaLove, CaDiarra, LeStockard)]
__auc__ [TeamId(Kansas St.)] vs [TeamId(Kennesaw St.)] ([Home])
__auc__ IN: MaMawien,ShNeal-williams,DeWade,BaBrown

ShNeal-williams vs ShWilliams

Alex-At-Home commented 4 years ago

Hmm here's an example of a legit case where it's hard to figure out what's going on:

__fms__ --------------------------------------------------
__fms__ CLUMP [+1] size=[7] [?true]
__fms__ [TeamId(Baylor)] vs [TeamId(Ole Miss)] ([Neutral])
__fms__ IN: JaButler,MaVital
__fms__ OUT: MaMayer
__fms__ ON: DaAllen,JaButler,KiMcclure,MaMason,MaVital,TrClark
__fms__ RAW:
__fms__ 08:10,19-22,CLARK,T made Layup
__fms__ 08:10,19-22,VITAL,M Assist
__fms__ 07:57,19-22,MCCLURE,K Steal
__fms__ 07:49,19-24,BUTLER,J Commits Foul
...
__fms__ -----------
__fms__ candidates=[Set(DaAllen, MaVital, KiMcclure, MaMason, TrClark, JaButler)] - [Set()] - [List(MaVital, KiMcclure, KiMcclure, TrClark)]
__fms__ [TeamId(Baylor)] vs [TeamId(Ole Miss)] ([Neutral])
__fms__ IN: MaMayer
__fms__ OUT: MaMason
__fms__ ON: DaAllen,JaButler,KiMcclure,MaMayer,MaVital,TrClark
__fms__ RAW:
__fms__ 06:22,21-28,MAYER,M made Two Point Jumper
__fms__ 05:52,21-28,MCCLURE,K Turnover
__fms__ 05:30,21-28,VITAL,M Commits Foul
__fms__ -----------
__fms__ candidates=[Set(DaAllen, MaMason, JaButler)] - [List(MaMason)] - [List()]
__fms__ [TeamId(Baylor)] vs [TeamId(Ole Miss)] ([Neutral])
__fms__ IN: MaMason
__fms__ OUT: KiMcclure
__fms__ ON: DaAllen,JaButler,MaMason,MaMayer,MaVital,TrClark
__fms__ RAW:
__fms__ 05:09,22-30,BUTLER,J made Free Throw
__fms__ 05:00,23-30,ALLEN,D Steal
__fms__ 04:41,28-31,CLARK,T made Layup
__fms__ 03:49,30-34,MASON,M made Two Point Jumper
__fms__ 03:35,30-34,MAYER,M Defensive Rebound
...

In that last sequence, everyone but Vital has made a play ... but Vital made a play in the previous lineup

So the first two lineups are really orphans, we can fix the lineup moving forward (but seeing all 5 names mentioned in a lineup seems pretty optimistic I assume?)

(and we don't know which of Allen/Butler was missing from the previous lineup pair, there's not enough data)

Alex-At-Home commented 4 years ago

Here's a case where the preceding lineup is fixed, but we don't pass that knowledge on:

__fms__ candidates=[Set(DaAllen, DeBandoo, KiMcclure, JaButler)] - [List(JaButler)] - [List(KiMcclure)]
__fms__ [TeamId(Baylor)] vs [TeamId(Ole Miss)] ([Neutral])
__fms__ IN: JaButler
__fms__ OUT: MaVital
__fms__ ON: DaAllen,DeBandoo,JaButler,KiMcclure,MaMason,MaMayer,TrClark
__fms__ RAW:
__fms__ 06:46,57-63,MAYER,M Turnover
__fms__ 06:25,57-63,BUTLER,J Defensive Rebound
__fms__ 06:11,59-63,CLARK,T made Layup
__fms__ 05:46,59-63,MAYER,M Commits Foul
__fms__ -----------
__fms__ candidates=[Set(DaAllen, DeBandoo)] - [List()] - [List()]
__fms__ --------------------------------------------------
__fms__ CLUMP [+1] size=[6] [?false]
__fms__ [TeamId(Baylor)] vs [TeamId(Ole Miss)] ([Neutral])
__fms__ IN:
__fms__ OUT: MaMayer
__fms__ ON: DaAllen,DeBandoo,JaButler,KiMcclure,MaMason,TrClark
__fms__ RAW:
__fms__ 05:30,61-65,CLARK,T made Layup
__fms__ 05:30,61-65,MASON,M Assist
__fms__ 05:08,61-65,MCCLURE,K Commits Foul
__fms__ -----------
__fms__ candidates=[Set(DaAllen, DeBandoo, KiMcclure, MaMason, TrClark, JaButler)] - [Set()] - [List(MaMason)]
__fms__ [TeamId(Baylor)] vs [TeamId(Ole Miss)] ([Neutral])
__fms__ IN: MaMayer
__fms__ OUT: DaAllen
__fms__ ON: DeBandoo,JaButler,KiMcclure,MaMason,MaMayer,TrClark

So actually there is a 1-lineup error at the start (because someone - DaAllen- comes in for MaMayer) ie the lineup is:

Xxx,JaButler,KiMcclure,MaMason,TrClark

and then it gets fixed one lineup later as Xxx (who we can identify as DeAllen if we really really want to) comes out, and MaMayer goes back in

So maybe we should have sub-clumps of consecutive events, where we pass preceding sub-clump into the next error and keep applying until they are mentioned in the subs list?

Alex-At-Home commented 4 years ago

LOW HANGING TO FIX

I've seen a few of these, where I miss the player:

__fms__ [TeamId(Baylor)] vs [TeamId(George Mason)] ([Neutral])
__fms__ IN: FlThamba
__fms__ OUT: TrClark
__fms__ ON: DaAllen,FlThamba,JaButler,KiMcclure,MaMason,MaMayer
__fms__ RAW:
__fms__ 12:02,7-10,MAYER,M made Two Point Jumper
__fms__ 11:39,7-10,MCCLURE,K Defensive Rebound
__fms__ 11:29,7-10,MASON,M missed Three Point Jumper
__fms__ 11:14,7-10,THAMBA,F Blocked Shot
__fms__ 11:14,7-10,MASON,M Defensive Rebound
__fms__ 10:50,7-12,MASON,M made Two Point Jumper
__fms__ -----------
__fms__ candidates=[Set(DaAllen, MaMayer, TrClark)] - [List(TrClark)] - [List()]
__fms__ [TeamId(Baylor)] vs [TeamId(George Mason)] ([Neutral])

What about Mayer?!

Alex-At-Home commented 4 years ago

~LOW HANGING TO FIX~ (EDIT: fixed and committed)

1] More bad names:

__auc__ CLUMP [+0] size=[1] [?true] [!Some(Set(UnknownPlayers))]
__auc__ valid player codes: [Set(DeDotson, MaGarrett, ChMoore, QuGrimes, K.Lawson, LaVick, UdAzubuike, MiLightfoot, DeLawson)]
__auc__ [TeamId(Kansas)] vs [TeamId(Vermont)] ([Home])
__auc__ IN: ChMoore,KjLawson
__auc__ OUT: LaVick,MaGarrett

KjLawson vs K.Lawson

Lawson, K.J. in box-score ... LAWSON,KJ Enters Game in game

Maybe get rid of all "."s as part of name building?

2] Seeing lots of Team IN!

__auc__ CLUMP [+0] size=[6] [?true] [!Some(Set(UnknownPlayers))]
__auc__ valid player codes: [Set(XaSneed, NiShadd, ShWilliams, MiMcguirl, DeWade, MaMawien, AuTrice, PiMcatee, KaStokes, JaLove, CaDiarra, LeStockard, BaBrown)]
__auc__ [TeamId(Kansas St.)] vs [TeamId(Marquette)] ([Away])
__auc__ IN: Team
__auc__ OUT: CaDiarra
__auc__ ON: AuTrice,BaBrown,DeWade,KaStokes,Team

Alexs-MBP:cbb-explorer alex$ more log.txt | grep -A 4 "CLUMP .[+]0." | grep "IN:" | grep  "Team" | wc -l
      14
Alexs-MBP:cbb-explorer alex$ more log.txt | grep -A 1 "CLUMP .[+]0." | grep "valid" | grep  "K.Lawson" | wc -l
      73
Alexs-MBP:cbb-explorer alex$ more log.txt | grep -A 4 "CLUMP .[+]0." | grep "IN:" | grep  "DyOsetkowski" | wc -l
      54

Alexs-MBP:cbb-explorer alex$ more log.txt | grep -c "CLUMP .[+]0." 
152

#Remainder:
Alexs-MBP:cbb-explorer alex$ more log.txt | grep -A 4 "CLUMP .[+]0." | grep "IN:" | grep  -v "Osetkowsk" | grep -v "KjLawson" | grep -v "Team" |more
__auc__ IN: ThDziagwa,CuJones,LuMajor
__auc__ IN: CaWagner,JkHadlock
__auc__ IN: DeMitchell
__auc__ IN: ReStreller
__auc__ IN: OwAschieris
__auc__ IN: RuBarlow
__auc__ IN: OwAschieris
__auc__ IN: OwAschieris
__auc__ IN: OwAschieris,KeDavis
__auc__ IN: 
__auc__ IN: JeSims,JaFebres

(Kansas)

(Texas: DyOsetkowski vs DyOstekowski... zomg it's a type in the name!) (TCU: OwAschieris vs OwAscieris ... ditto!)

I like the idea of having a name converter instead of flipping names? Then we can put mis-spellings in there also?

Alex-At-Home commented 4 years ago

~LOW HANGING TO FIX~ (EDIT: fixed and committed)

On the ACC side:

box:
DjVasiljevic
DjFunderburk
PjHorne

PbP:
DeVasiljevic (De vs Dj)
DeFunderburk (De vs Dj)
PrHorne (Pr vs Pj)

eg Preston Horne (bpb) instead of P.J (box) ... annoying ... I think we can handle that in one go by looking at the box score name to see if the first and "3rd onwards" match?

Alex-At-Home commented 4 years ago

In this one, we could have fixed some of the problems but not all? The very first lineup has 4/5 players mentioned, but the next lineup mentions the 5th despite having 1 in, 1 out?

> [INFO] Successful parse: good=[22] bad=[5]
3796,3878c3837
< __auc__ --------------------------------------------------
< __auc__ CLUMP [+1] size=[4] [?true] [!Some(Set(WrongNumberOfPlayers))]
< __auc__ valid player codes: [Set(JuSimon, GrWilliams, JoRoberts, SeKeita, ShPonds, LjFigueroa, MaClark, BrTrimble, MaEarlington)]
< __auc__ [TeamId(St. John's (NY))] vs [TeamId(Xavier)] ([Home])
< __auc__ IN: JoRoberts
< __auc__ OUT: 
< __auc__ ON: JoRoberts,JuSimon,LjFigueroa,MaClark,SeKeita,ShPonds
< __auc__ RAW:
< __auc__ 19:43,0-0,SIMON,JUSTIN Steal
< __auc__ 19:41,0-0,SIMON,JUSTIN Turnover
< __auc__ 19:25,0-0,PONDS,SHAMORIE Steal
< __auc__ 19:23,0-2,FIGUEROA,LJ made Layup
< __auc__ 19:23,0-2,PONDS,SHAMORIE Assist
< __auc__ 18:40,3-2,CLARK II,MARVIN missed Three Point Jumper
< __auc__ 18:08,6-5,CLARK II,MARVIN made Three Point Jumper
< __auc__ 18:08,6-5,PONDS,SHAMORIE Assist
< __auc__ 17:45,6-5,SIMON,JUSTIN Defensive Rebound
< __auc__ 17:28,6-5,FIGUEROA,LJ missed Two Point Jumper
< __auc__ 17:01,8-5,SIMON,JUSTIN Turnover
< __auc__ 16:26,10-5,KEITA,SEDEE missed Two Point Jumper
< __auc__ 16:15,10-5,SIMON,JUSTIN Defensive Rebound
< __auc__ 16:02,10-5,CLARK II,MARVIN missed Two Point Jumper
< __auc__ 15:38,13-5,TEAM 30 Second Timeout
< __auc__ 15:38,13-5,TEAM Media Timeout
< __auc__ -----------

>>>>> Everyone but JoRoberts mentioned

< __auc__ [TeamId(St. John's (NY))] vs [TeamId(Xavier)] ([Home])
< __auc__ IN: BrTrimble
< __auc__ OUT: JuSimon
< __auc__ ON: BrTrimble,JoRoberts,LjFigueroa,MaClark,SeKeita,ShPonds
< __auc__ RAW:
< __auc__ 15:27,13-8,TRIMBLE JR,BRYAN made Three Point Jumper
< __auc__ 15:27,13-8,PONDS,SHAMORIE Assist
< __auc__ 14:37,13-10,FIGUEROA,LJ made Two Point Jumper
< __auc__ 14:04,13-10,FIGUEROA,LJ Defensive Rebound
< __auc__ 13:52,13-13,CLARK II,MARVIN made Three Point Jumper
< __auc__ 13:52,13-13,PONDS,SHAMORIE Assist
< __auc__ 13:17,13-13,ROBERTS,JOSH Commits Foul

^^^^^^^^^^^^^^^^^
??

< __auc__ -----------
< __auc__ [TeamId(St. John's (NY))] vs [TeamId(Xavier)] ([Home])
< __auc__ IN: JuSimon
< __auc__ OUT: LjFigueroa
< __auc__ ON: BrTrimble,JoRoberts,JuSimon,MaClark,SeKeita,ShPonds
< __auc__ RAW:
< __auc__ 12:55,15-13,PONDS,SHAMORIE Turnover
< __auc__ 12:21,17-13,PONDS,SHAMORIE missed Layup
< __auc__ -----------
< __auc__ [TeamId(St. John's (NY))] vs [TeamId(Xavier)] ([Home])
< __auc__ IN: MaEarlington,GrWilliams
< __auc__ OUT: ShPonds,JoRoberts
< __auc__ ON: BrTrimble,GrWilliams,JuSimon,MaClark,MaEarlington,SeKeita
< __auc__ RAW:
< __auc__ 12:04,17-13,SIMON,JUSTIN Blocked Shot
< __auc__ 12:02,17-13,SIMON,JUSTIN Defensive Rebound
< __auc__ 11:59,17-13,SIMON,JUSTIN Turnover
< __auc__ 11:37,19-13,EARLINGTON,MARCELLU missed Two Point Jumper
< __auc__ 10:54,22-13,CLARK II,MARVIN missed Three Point Jumper
< __auc__ 10:54,22-13,EARLINGTON,MARCELLU Offensive Rebound
< __auc__ 10:43,22-13,SIMON,JUSTIN Turnover
< __auc__ 10:34,22-13,SIMON,JUSTIN Defensive Rebound
< __auc__ 10:33,22-13,TEAM Timeout
< __auc__ -----------
< __auc__ Unfixed
< ()
< __auc__ [TeamId(St. John's (NY))] vs [TeamId(Xavier)] ([Home])
< __auc__ IN: LjFigueroa,ShPonds
< __auc__ OUT: MaEarlington,GrWilliams,JuSimon
< __auc__ ON: BrTrimble,LjFigueroa,MaClark,SeKeita,ShPonds
< __auc__ RAW:
< __auc__ 10:17,22-13,PONDS,SHAMORIE missed Three Point Jumper
< __auc__ 10:17,22-13,FIGUEROA,LJ Offensive Rebound
< __auc__ 10:13,22-15,FIGUEROA,LJ made Tip In
< __auc__ 09:35,22-15,TRIMBLE JR,BRYAN missed Three Point Jumper
< __auc__ 09:03,25-15,KEITA,SEDEE missed Two Point Jumper
< __auc__ 08:19,25-18,TRIMBLE JR,BRYAN made Three Point Jumper
< __auc__ 08:19,25-18,PONDS,SHAMORIE Assist
< __auc__ 07:58,25-18,PONDS,SHAMORIE Steal
< __auc__ 07:53,25-20,FIGUEROA,LJ made Layup
< __auc__ 07:33,25-20,FIGUEROA,LJ Defensive Rebound
< __auc__ 07:29,25-20,TEAM Media Timeout
< __auc__ -----------
< __auc__ (next good)

Alex-At-Home commented 4 years ago

This is fun, a single lineup has 6 players mentioned, probably the first is stuck in the wrong lineup:

__auc__ CLUMP [+1] size=[1] [?true] [!Some(Set(WrongNumberOfPlayers))]
__auc__ valid player codes: [Set(JuSimon, GrWilliams, JoRoberts, SeKeita, ShPonds, MiDixon, LjFigueroa, MaClark, BrTrimble, MaEarlington, MuHeron)]
__auc__ [TeamId(St. John's (NY))] vs [TeamId(Loyola Maryland)] ([Home])
__auc__ IN: SeKeita
__auc__ OUT:
__auc__ ON: GrWilliams,JuSimon,MiDixon,SeKeita,ShPonds,MuHeron
__auc__ RAW:
__auc__ 12:46,32-60,PONDS,SHAMORIE missed Three Point Jumper
__auc__ 12:35,32-60,HERON,MUSTAPHA missed Two Point Jumper
__auc__ 12:35,32-60,SIMON,JUSTIN Offensive Rebound
__auc__ 12:31,32-60,SIMON,JUSTIN missed Two Point Jumper
__auc__ 12:21,32-60,KEITA,SEDEE Blocked Shot
__auc__ 12:19,32-60,HERON,MUSTAPHA Defensive Rebound
__auc__ 12:15,32-60,DIXON,MIKEY missed Three Point Jumper
__auc__ 12:15,32-60,WILLIAMS JR,GREG Offensive Rebound
__auc__ 12:12,32-60,WILLIAMS JR,GREG missed Layup
__auc__ 12:01,32-60,SIMON,JUSTIN Steal
__auc__ 11:58,32-62,SIMON,JUSTIN made Layup
__auc__ 11:39,32-62,HERON,MUSTAPHA Defensive Rebound
__auc__ 11:31,32-62,DIXON,MIKEY Turnover
__auc__ 11:02,34-64,SIMON,JUSTIN made Two Point Jumper
__auc__ 10:43,34-64,KEITA,SEDEE Defensive Rebound
__auc__ 10:33,34-64,SIMON,JUSTIN missed Two Point Jumper
__auc__ 10:08,34-64,WILLIAMS JR,GREG Commits Foul

Alex-At-Home commented 4 years ago

~LOW HANGING TO FIX~ (EDIT: fixed and committed)

More fun with names, from the A10 2018/9:

JoAxGudmundsson vs Set(KeGrady, KiPritchett, BaJones, DuKovacevic, LuFrampton, MaWynter, CaCollins, CaFreundlich, NaEkwu, JoAxGudmunsson, LuBrajkovic, DaCzerapowicz)

(Extra d!)

CiSullivan (pbp) vs KeSullivan (box)!

AnLongpr?? (lol) vs AnLongpre

DGuiseppi vs ... no idea!

and SEC:

JoPhilips vs JoPhillips (2 ls)

IgSargiunas (pbp) vs IgSargiūnas (box)

Alex-At-Home commented 4 years ago

Current number of good lineups in 2018/9: 84321 (so approx 245K possessions)

After all processing:

//Lineups: 
good: 93622
bad: 843
total: 94465
//Possessions
good: 278366 
bad: 1705
total: 280071

so 35K/280K =~ 13% were bad before, now <1% (0.6%)

2019:

Old good lineups: 15236
Old good possessions: 45899

After all processing:

good lineups:15657
good possessions: 46718
bad lineups: 130
bad possessions: 309
total lineups: 15787
total possessions: 47027

Alex-At-Home commented 4 years ago

Here's a nice easy one (I think I actually had this in my POC at one point but then removed it while tidying up):

__auc__ CLUMP [+1] size=[1] [?true] [!Some(Set(WrongNumberOfPlayers))]
__auc__ valid player codes: [Set(JaCumberland, NyBrooks, ElNsoseme, TrScott, LoJohnson, KeWilliams, CaBroome, JoKoz, RaFredericks, TrMoore, JuJenifer)]
__auc__ [TeamId(Cincinnati)] vs [TeamId(George Mason)] ([Neutral])
__auc__ IN: NyBrooks,CaBroome
__auc__ OUT: JuJenifer
__auc__ ON: CaBroome,ElNsoseme,JaCumberland,KeWilliams,NyBrooks,TrScott
__auc__ RAW:
__auc__ 15:46,3-6,CUMBERLAND,J Steal
__auc__ 15:38,3-6,CUMBERLAND,J missed Two Point Jumper
__auc__ 15:38,3-6,BROOKS,N Offensive Rebound
__auc__ 15:38,3-6,BROOKS,N missed Free Throw
__auc__ 15:38,3-6,TEAM Deadball Rebound
__auc__ 15:38,3-7,BROOKS,N made Free Throw
__auc__ -----------
__auc__ Unfixed
()
__auc__ [TeamId(Cincinnati)] vs [TeamId(George Mason)] ([Neutral])
__auc__ IN: JoKoz,TrScott
__auc__ OUT: TrScott,JaCumberland
__auc__ ON: CaBroome,ElNsoseme,JoKoz,KeWilliams,NyBrooks
__auc__ RAW:
__auc__ 14:52,6-7,CUMBERLAND,J missed Two Point Jumper
__auc__ 14:52,6-7,TEAM Offensive Rebound

We now TrScott is the problem because he is IN into the good lineup but was already marked as ON in the bad lineup