[BUG] Experience review - including subskill recalculation (7-weeks)

[masterpatje]

Describe the bug I took a deep dive into experience and how it is calculated in HO.

Because hattrick has a page where you can see the drilldown in experience points (click on a player and there is a tab "more information" with historic results of a player including minutes played and on which position and experience gained etc. ). You can almost exactly determine the experience sub-level.

In order to correctly interpreted the results could you (@wsbrenk ) answer a few questions:

• Does HO include the latest match? Is experience subskill recalculated if I played a match and after the match is done I download the HRF file?
• How does HO cope with friendly matches? International friendly matches (0,7 XP) have double the experience of a local match (0,35 XP)? Do you take the lowest, highest or average number? Or does the coding include differentiating between local and international matches?
• What does subskill recalculation (7 weeks) do in relation to experience? What matches are included and which are not? Does it differ from the matches considered for ability subskill recalculation?

Findings:

• I had some unexpected results with regards to experience when I did a comparison between the latest download and the HRF file of the week before. Because I downloaded the file directly after training results were available; the only matches that have been played were the matches corresponding to week 12, but experience seemed to be based not exactly on those matches, but took into consideration also a match prior to week 12?!
• Things became more weird after subskill recalculation 7-weeks. (look at screenshots). It differs completely from what I would expect. Only a few players gain experience (subskill), while almost my whole squad should have got an increase.

By hand I calculated the experience for 7 a 8 weeks in a row. So hope you can look at this as soon as possible so that I can do proper testing and see how we can improve this part.

To Reproduce Steps to reproduce the behavior:

1. Download data
2. Compare data with last week data (look at experience)
3. Do a subskill recalculation (7-weeks)
4. Compare data with last week data (look at experience)

Screenshots Experience increase by comparison (after subskill recalculation 7-weeks)

Matches played between 18th June 2022 10:29 and 24th June 10:57

Platform information:

• OS: Windows10
• Version 6.0.315

Additional context Hattrick change:

• Experience needed for an increase in level was 28,571 and is now 100 ( * 3,5). And the experience gained by matches is also increased by a factor 3,5.
• Experience calculation is thus the same as in the old situation. [Is the code adjusted accordingly so that it is in line with the 'hattrick manual'? I say this for recognisable code purpose for future developers]

[wsbrenk]

select * from spieler 

or

select * from spieler where spielerid=1234566

see example above (29 days ago)

[masterpatje]

Ok I changed the SQL statement to only include the HRF file and the corresponding SUBEXPERIENCE select * from spieler where spieler.datum = '2022-06-11 23:32:08.000000' order by spieler.lastname, DATUM

This corresponds to the initial value for the calculation. But that value is actually the summed calculation of ALL the previous HRF files before that. (I haven't checked if the value actually corresponds to the other HRF files in the database).

In fact that means that the calculated experience value is thus not the latest 7 weeks but is always the total experience from the files in the database. Or am I mistaken? (I think this actually is a good thing, because no matter if you take the 7-weeks of full recalculation you always get the total experience for the player. It would not make to much sense to only calculate it for the last 7 weeks.).

[masterpatje]

These are the HRF files in my test database Do you know why the SUBEXPERIENCE in the OLDEST HRF FILE is 0,0 and not equal to the experience from the matches within that training week?

EDIT 23:30: might be do to the fact that I imported this file and because it is outside the 7-weeks recalculation it does not yet have a calculated SUBEXPERIENCE and is thus 0. EDIT 23:54: yes I tested this and it is true that imported files SUBEXPERIENCE stay 0 untill recalculated

[wsbrenk]

Do you know why the SUBEXPERIENCE in the OLDEST HRF FILE is 0,0 and not equal to the experience from the matches within that training week?

if you recalc only 7 weeks, the hrfs before that won't be changed (remain 0 if never were calculated). if you recalc unlimited they should consider all matches downloaded for the trainings. training has to be set correctly too. if training info of matches are missing the warning should appear in logs.

[masterpatje]

@wsbrenk I figured out what is going on and why I got different outcomes!! Offset in correct calculation occurs only when the calculation coincides with a previous file that holds a lower ERFAHRUNG level. On top of that I had 4 players that leveled in the same week in the 8th week before the calculation. That is why the calculation affected multiple players.

For this I used the following SQL statement: select HRF_ID, DATUM, PLAYERNUMBER, LASTNAME, ERFAHRUNG, SUBEXPERIENCE, LASTMATCHDATE, LASTMATCH_PLAYEDMINUTES from spieler order by HRF_ID, PLAYERNUMBER, DATUM Below a screenshot of the information:

Now I compared the latest HRF files in the database on ERFAHRUNG and SUBEXPERIENCE 9th (and last) oldest HRF: Season 70 Training Week 8 (base file that was not considered in the recalculation and does not hold SUBEXPERIENCE value that influences the calculation, but still is important because if the value ERFAHRUNG) 8th oldest HRF: Season 70 Training Week 9 (base file that was not considered in the recalculation, but because it holds the initial value the subexperience was considered and calculated). It is important to note that HO7 notices an increase in ERFAHRUNG of 1 and sets the SUBEXPERIENCE to 0. 7th oldest HRF: Season 70 Training Week 10 (experience was recalculated)

In my excel it is not Training Week 9 but Training Week 10 when 4 of my players gain experience. Following the logic of the current calculation the SUBEXPERIENCE should be set to 0 in Week 10 (minimum value after increase of experience).

Alternatively the SUBEXPERIENCE can be set for the matches in Training Week 10 (maximum value).

Table below shows that the SUBEXPERIENCE is set to 0 in Training Week 9 (1 week prior to Week 10 that holds the initial value for SUBEXPERIENCE)

It was quite some search, but at least now I know where the calculation starts to differ and why by accident it was affecting 4 players (because in that exact week 4 players increased in experience).

Hopefully by these examples I could explain to you what I found out. Please let me know if something is not clear enough so I can explain better.

[masterpatje]

Today I further thought about the 'cause' that ERFAHRUNHG gain was 1 week early so that we can find a fix.

And I think that it has to do with the following:

• If I download a HRF file today (monday) it includes today's information about the player.
• In the HRF file there is information on the (last) training week (based on the current training setting).
• The experience in HO is based on the (last) training week.
• But today's (monday) HRF file when you look at experience already incorporated sunday's match and the according experience while HO experience calculation is only considered in (next) training week.
• In the code because HO sees the increase (of level) in experience it considers this as a gain of level in that week and thus resets the SUBEXPERIENCE to 0,0 (minimum value).
• Now the result is that this SUBEXPERIENCE is set to 0,0 one week early.

In above ecample HO7 resets the SUBEXPERIENCE in S70 W9 instead of S70 W10.

I think the same will occur if I take a (really) old file where ERFAHRUNG is 1 level lower for all (or many) players. Most probably it will trigger the same that for all players the SUBEXPERIENCE is set to 0.

[wsbrenk]

@masterpatje we should split the experience calculation from the skill calculation. skill on training date and experience on match date

[masterpatje]

@masterpatje we should split the experience calculation from the skill calculation. skill on training date and experience on match date

I agree that would be more correct and accurate. Because the information of the player is actual as well on download.

However the advantage of the current calculation is that no duplication can occur due to the fact that training week is leading and thus the SUBEXPERIENCE value of -last- training week is considered the starting value.

So considering 1) that a calculation error for experience might occur if the ERFAHRUNG increased in level (OR the latest file prior to the one holding the initial value of SUBEXPERIENCE has a lower ERFAHRUNG level), 2) When ERFAHRUNG level was increased due to a match that was in the HRF download but follows the training week considered.

what I would propose is the following: (but maybe you have other ideas)

• For the release HO6 this is not hindering, so it can be released.
• For HO7 development it might be good to just start with a small modification; To add two parameters SUBEXPERIENCE_ACTUAL and SUBEXPERIENCE_TRAININGWEEK (= equal to SUBEXPERIENCE). The ACTUAL value is the one used for the visual in HO and holds the value of actual experience including the latest matches included in the latest HRF download and thus can be calculated as SUBEXPERIENCE_ACTUAL = SUBEXPERIENCE_TRAININGWEEK + EXP latest matches after last training week in HRF download. The SUBEXPERIENCE_TRAININGWEEK is the one that is used for starting point of calculation and has the same function as the current SUBEXPERIENCE.
• Later you can see if you can split the calculation all together.
• Additional suggestion maybe a new option available next to subskill (7-weeks or full) that just does the experience recalculation? In this way all HRF files will be considered and will get a SUBEXPERIENCE calculation.
• Afterwards see how to cope with the considerations 1) and 2), because part of it is choice. These considerations were 'errors' in the calculation that might occur.

@wsbrenk please share your thoughts on this?

And @wsbrenk is it worth understanding why in certain matches the minutes played was incorrect in HO attribute? I encountered a fee examples and presented them with screenshots above.