search

ho-dev / HattrickOrganizer

Assistant for Hattrick online football manager
https://ho-dev.github.io/HattrickOrganizer/
GNU Lesser General Public License v3.0
195 stars 79 forks source link

[BUG] Experience review - including subskill recalculation (7-weeks) #1542

Closed masterpatje closed 1 year ago

masterpatje commented 2 years ago

Describe the bug I took a deep dive into experience and how it is calculated in HO.

Because hattrick has a page where you can see the drilldown in experience points (click on a player and there is a tab "more information" with historic results of a player including minutes played and on which position and experience gained etc. ). You can almost exactly determine the experience sub-level.

In order to correctly interpreted the results could you (@wsbrenk ) answer a few questions:

Findings:

By hand I calculated the experience for 7 a 8 weeks in a row. So hope you can look at this as soon as possible so that I can do proper testing and see how we can improve this part.

To Reproduce Steps to reproduce the behavior:

  1. Download data
  2. Compare data with last week data (look at experience)
  3. Do a subskill recalculation (7-weeks)
  4. Compare data with last week data (look at experience)

Screenshots Experience increase by comparison (after subskill recalculation 7-weeks) image

Matches played between 18th June 2022 10:29 and 24th June 10:57 image

image

Platform information:

Additional context Hattrick change:

masterpatje commented 2 years ago

Additional info: I downloaded today's HRF file (which included Sunday's league game) I noticed that there is no gain in experience, so the conclusion that I can draw from that, is that experience is not directly updated after a match have been played.

Are there certain moments when experience recalculation is triggered? Is it connected to certain days?

wsbrenk commented 2 years ago

@masterpatje experience is calculated together with trainings subskill calculation. the trigger is the training date. each played minute has experience effect with the weights of the match type as you found in the "docs". for some kinds of match the effect is smaller than 0.01 which is not displayed if you compare only one week.

masterpatje commented 2 years ago

General comment Today the training results became available, which includes last week matches. Now I can more easily check experience; I first tested my formula with the information on the hattrick website. And then I looked at the experience gained since last week.

Test 1: Experience on HRF download after training update

Results: Experience is most probably correctly calculated on download. The small differences in 5 players that I noticed might be because of decimal precision shown in HO. Q: Is is true that in the background calculation 3 decimals behind the komma are taking into consideration? And that for visual purposes the decimals are rounded to 2 decimals behind the komma? Q: Does HO in the background distinquishes between (international and national) friendlies? In other words is there coding that looks at the nationality in a friendly match and sets the appropiate experience? The experience gain differs and thus I want to know how HO copes with it? Or does it consider always the minimum or maximum or average? Just to be sure what I am looking at.

image

Test 2: Experience after importing HRF file (hopefully today) Will do this test later, because otherwise I can't do the proper subskill testing.

Test 3: Experience after subskill recalculation (hopefully today) Will do this test later, because I can't redo the action.

masterpatje commented 2 years ago

Test 4: Experience comparison saturday 24 june 2022 10:57 (includes training week 12) versus friday 18 june 10:29 (includes training week 11) after subskill recalculation

Result: Visual representation of experience gained was not correct, the experience subskill level was correct.

Today I had the HRF file from 1 july 2022 (includes training week 13) and when I compare it with 24 june 2022 (includes training week 12) and do a recalculation the experience is exactly the same also the visual representation of the gain. (good)

Last week when I did a subskill recalculation the (visual!) experience gained changed and afterwards was only visible for only 7 players and was incorrectly in amount of subskill gain, but the experience subskill in total is correct for all players.

Q: Do you understand why this might happen? Please note that in this week comparison I compare friday (1st) to friday (24th) and the week before I had a comparison between saturday (24th) and friday (18th). Maybe it helps understanding why the visual representation of the gain is not shown for all players.

image

image

wsbrenk commented 2 years ago

image

HO calculates with higher precision than displayed in the table column. you can check that with sql editor.

wsbrenk commented 2 years ago

HO considers the different match types when calculating the experience:

    public double getExperienceIncrease(int minutes) {
        double p;
        var _matchType = this.getMatchTypeExtended();
        if (_matchType instanceof MatchType){
            p = switch ((MatchType) _matchType){
                case CUP, QUALIFICATION -> 2d;
                case FRIENDLYNORMAL, FRIENDLYCUPRULES -> .1;
                case INTFRIENDLYCUPRULES, INTFRIENDLYNORMAL -> .2;
                case NATIONALFRIENDLY, LEAGUE -> 1d;
                case MASTERS -> 5d;
                case NATIONALCOMPCUPRULES -> 16d;
                case NATIONALCOMPNORMAL -> 8d;
                case TOURNAMENTGROUP -> getTournamentExperienceFactor(TournamentType.getById(this.iMatchContextId), false);
                case TOURNAMENTPLAYOFF -> getTournamentExperienceFactor(TournamentType.getById(this.iMatchContextId), true);
                default -> 0d;
            };
        } else{
            //case MatchTypeExtended.EMERALDCUP, MatchTypeExtended.RUBYCUP, MatchTypeExtended.SAPPHIRECUP -> .5;
            p = 0.5;
        }
        return minutes * p / 90. / 28.571;
    }
wsbrenk commented 2 years ago

just found one of my players showing also missing increments

image

wsbrenk commented 2 years ago

different subs 0.021000315, 0.21000314 is a minor bug. values were stored as double precision values but load as floats.

lastmatch at 06-28 was loaded before training calculation. after downloading current training the 7th skill increment was calculated. result is 0.024 which is still displayed as .02.

masterpatje commented 2 years ago

Comparison between last version and current version (332) gives the following result (see screenshot): In general experience is a slighty higher so that is good I think.

I tested on player that stood out: Szogedi had an increase of 0,007 (90 minutes friendly) in match 29-6-2022 . His history in the last 6 training weeks prior to the 7th training week are: 18-5-2022; 21 minutes cup ; 1,633 [training week 7] 22-5-2022; 21 minutes league; 0,817 [training week 8] 25-5-2022; 11 minutes cup; 0,857 [training week 8] 29-5-2022; 11 minutes league; 0,428 [training week 9] 1-6-2022; 45 minutes friendly (local); 0,175 [training week 9] 8-6-2022; 90 minutes friendly (international); 0,7 [training week 10] 15-6-2022; 90 minutes friendly (international); 0,7 [training week 11] 19-6-2022; 27 minutes league; 1,05 [training week 12] 22-6-2022; 90 minutes friendly (international); 0,7 [training week 12] Total 6 training weeks prior to last week; 7,06 (0,0706 = 0,07 rounded) 7th week: 29-06-2022; 90 minutes friendly (international) [training week 13] Total 7 training weeks: 7,76 (0,0776 = 0,08 rounded)

My conclusion: Increase in 7th week should have been 0,01. (comparison training week 13 versus training week 12)

But I noted experience is indicated as 8,10; Which is higher than 8,08, so I think it takes other matches into consideration was well. Q: Could you tell me which matches are included? Or how I can check which matches were included on subskill recalculation (7-weeks)?

image

masterpatje commented 2 years ago

Code can be changed to:

public double getExperienceIncrease(int minutes) { double p; var _matchType = this.getMatchTypeExtended(); if (_matchType instanceof MatchType){ p = switch ((MatchType) _matchType){ case CUP, QUALIFICATION -> 7d; case FRIENDLYNORMAL, FRIENDLYCUPRULES -> .350; case INTFRIENDLYCUPRULES, INTFRIENDLYNORMAL -> .700; case NATIONALFRIENDLY, LEAGUE -> 3.5d; case MASTERS -> 17,5d; case NATIONALCOMPCUPRULES -> 56d; case NATIONALCOMPNORMAL -> 28d; case TOURNAMENTGROUP -> getTournamentExperienceFactor(TournamentType.getById(this.iMatchContextId), false); case TOURNAMENTPLAYOFF -> getTournamentExperienceFactor(TournamentType.getById(this.iMatchContextId), true); default -> 0d; }; } else{ //case MatchTypeExtended.EMERALDCUP, MatchTypeExtended.RUBYCUP, MatchTypeExtended.SAPPHIRECUP -> 1.75; p = 0.5; } return minutes * p / 90. / 100; }

Maybe you can check if new categories need to be added, because they specify more categories for national teams matches as it seems.

In general experience calculation is the same, except that everything is multiplied by 3,5.

https://www84.hattrick.org/Help/Rules/PlayerAttributes.aspx image

wsbrenk commented 2 years ago

these match types are handled in method getTournamentExperienceFactor:

    private double getTournamentExperienceFactor(@Nullable TournamentType tournamentType, boolean isPlayOff) {
        if ( tournamentType == null) return 0d;
        if ( tournamentType.isWorldCup()){
            // World Cup match  28
            // World Cup (Semi and Final)   56
                if ( isPlayOff){
                    return 56/3.5;
                }
                else {
                    return 28/3.5;
                }
        }
        else if ( tournamentType.isNationsCup()){
            // Nations Cup  7
            // Nations Cup (Knockout)   14
            if ( isPlayOff){
                return 14/3.5;
            }
            else {
                return 7/3.5;
            }
        }
        else if ( tournamentType.isContinentalCup()){
            // Continental Championships    14
            // Continental Championships (Quarter, Semi and Final)  21
            if ( isPlayOff){
                return 21/3.5;
            }
            else {
                return 14/3.5;
            }
        }
        else if ( tournamentType.isNTFriendly()){
            // National Team friendly   3.5
            return 1d;
        }
        return 0d;
    }
wsbrenk commented 2 years ago

trainings are examined from latest download (hrf file) which is older than 7 weeks. if fear if there is a gap of several weeks the considers matches could be a bit older than 7 weeks.

masterpatje commented 2 years ago

trainings are examined from latest download (hrf file) which is older than 7 weeks. if fear if there is a gap of several weeks the considers matches could be a bit older than 7 weeks.

What do you exactly mean with "latest download (hrf file) which is older than 7 weeks?".. do you mean the latest file or te files that is 7 weeks older than the last downloaded hrf file?

In version 7.0.337 (portable) (which works 'experience wise' the same as HO6 most probably) I did a new test. Because it was a clean build I added HRF files one at a time by importing and afterwards recalculating (7-weeks). The experience stays the same?! :

image

image

image

image

image

image

image

wsbrenk commented 2 years ago

if you only import the hrfs the match details are missing, i fear.

wsbrenk commented 2 years ago

the calculation selects all hrfs from database which are downloaded during the last 7 week.

the oldest entry (7 weeks old) loads the hrf loaded before (some minutes or months before him) this entry will give the start values for the skills (and experience). all trainings in past training list that happened before oldest hrf and after the previous one will be taken into account.

this is continued with the following hrfs

masterpatje commented 2 years ago

@wsbrenk; back from holiday and did some more testing. I thought I wanted to test in on a stand alone version (so I took 7.0.350) and I only have 2 HRF files in the database, 19th of July and 8th July (for comparison): TEST 1: HRF download 19th July = CORRECT On downloading 19th July 2022 HRF file the experience is correctly calculated: image It produces the same results as in the screenshot below and thus includedes the 10th July and 13th July matches: image

TEST 2: SUBSKILL RECALCULATION (7-weeks or all) = INCORRECT As you can see the total experience is higher and the gain is incorrectly displayed. image

image

image

image

Reaction other wsbrenk comments:

if you only import the hrfs the match details are missing, i fear.

For me it is important to first fix this issue and later on we can see if a missing HRF file could be tackled with downloadable matches.

the calculation selects all hrfs from database which are downloaded during the last 7 week.

the oldest entry (7 weeks old) loads the hrf loaded before (some minutes or months before him) this entry will give the start values for the skills (and experience). all trainings in past training list that happened before oldest hrf and after the previous one will be taken into account.

this is continued with the following hrfs

Does this mean that the 7 week old HRF is considered as the base reference and the 6 newer weeks are than considered to be the files where the gain of experience is visible?

masterpatje commented 2 years ago

Today new test:

- Now I am going to do subskill recalculation (7-weeks)

Problems are in the (visual) comparison by week and thus the gain is incorrectly shown:

Comparing week 16:

Comparing week 15: image image

Conclusion:

masterpatje commented 2 years ago

image

So it should all matches training week 10 till 16 (so last matches included are training week 10 (5th and 8th June matches) : image

HO experience after recalculation: image image

My calculation (last matches included 5th and 8th June) So there is now a big difference in total experience?! In last test it was correct considering only 3 HRF files that were in the database. image

wsbrenk commented 2 years ago

Does this mean that the 7 week old HRF is considered as the base reference and the 6 newer weeks are than considered to be the files where the gain of experience is visible?

not quite, I'm afraid. base of the calculation is the latest download BEFORE the 7 week old hrf file. all trainings and matches after that download will be considered.

masterpatje commented 2 years ago

@wsbrenk I want to help test this issues because of the excel I made in which I am sure that I correctly calculated the experience week by week, but in order to do so I need some help.

In the above example the total experience of last 7 weeks (see the past training table) was after recalculation no correct in total.

It would be nice if we could fix this for the HO 6.0 release but it is not necessary to do so, but I want to help to make it possible.

Couls you tell how we can move forward on this issue? What weeks do are considered looking at that example?

masterpatje commented 2 years ago

Sorry for the typos ;-)

masterpatje commented 2 years ago

Idea for testing could be the following. I can download the HO 7.0 portable version (which is based on HO 6.0) and where the subskill recalculation functionality works the same.

1) I can start by downloading the current HRF file, which includes 24th (wednesday) and 27th july (sunday) matches.

2) Then do a subskill recalculation (7-weeks).

3) Compare total experience before and after.

4a) Add a HRF file with the training week before and 17th (wed) and 20th (sun) matches.

4b) Then do a subskill recalculation (7-weeks).

4c) Compare total experience before and after.

5) ...repeat proces for another week for step 4...

Problem with this approach is that the altered total experience after subskill recalculation cannot be made undone and I don't know if that influences the test?

But before I do this, I need you to reply on my other comment that I posted today.

wsbrenk commented 2 years ago

@masterpatje many thanks for your help on this.

  1. the hrf files do not include the corresponding match information needed for experience calculation. they have to be downloaded separatly if they are not already in the database.

the result of the calculation should not depend on previous runs of experience calculations - only on the start values given in the latest hrf data before 7 week period. the development of each player can be viewed best with the Simple SQL editor, as shown 28 days ago.

wsbrenk commented 2 years ago

if it would help, i can try to add some logging of experience increments during the calculation. what do you think about this?

masterpatje commented 2 years ago

Yes! This would help a lot. From that I can do debugging and see where the calculations are different and why.

So if you could add this (temporarily) in the newest HO7 version and can continue exploring.

Let me know when it is ready and I will look at it asap.

wsbrenk commented 2 years ago

[Debug] HOVerwaltung: Start full subskill calculation. Sat Jul 30 17:25:34 CEST 2022 [Info] Player: Training 17.06.2022, 00:30:00 Experience increment of Walter Hasemann: 0.0 new sub value=0.0 [Info] Player: Training 17.06.2022, 00:30:00 Experience increment of Franziskus Zentsch: 0.0035000525007875117 new sub value=0.017500262480500475 [Info] Player: Training 17.06.2022, 00:30:00 Experience increment of Daniel-Luca Schlütterf: 0.0 new sub value=0.0035000524949282408 [Info] Player: Training 17.06.2022, 00:30:00 Experience increment of Manolo Klaus: 0.035000525007875115 new sub value=0.8131787996393753 [Info] Player: Training 17.06.2022, 00:30:00 Experience increment of Peter Gensberger: 0.0 new sub value=0.0 [Info] Player: Training 17.06.2022, 00:30:00 Experience increment of Dennis Wölm: 0.0 new sub value=0.0 [Info] Player: Training 17.06.2022, 00:30:00 Experience increment of Georg Hornsteiner: 0.0035000525007875117 new sub value=0.0350005223940046

wsbrenk commented 2 years ago

further information useful for the evaluation?

masterpatje commented 2 years ago

Add the training week number. The training week corresponding to that week experience gain (if it corresponds to the 'past training' week table). I can deduce from the training week the corresponding matches. So that I am absolutely sure which matches were included.

wsbrenk commented 2 years ago

added minutes - the dev is created

further info has to wait - have to feed my bees now.

masterpatje commented 2 years ago

Just started debugging, but right after the first player Jarne Clarebout I got an error. This was on a clean build with only the first HRF file downloaded. I directly checked the 7-weeks recalculation. I will

[Info] Player: Training 29 jul. 2022 01:05:00 Minutes= 90 Experience increment of Jarne Clarebout: 0.07000105001575023 new sub value=0.07000105001575023 [Debug] HOVerwaltung: recalcSubskills : [Error] HOVerwaltung: java.lang.NullPointerException: Cannot invoke "core.training.TrainingPoints.getTrainingDuration()" because the return value of "core.training.TrainingPerPlayer.getTrainingPair()" is null [Error] HOVerwaltung: core.model.player.Player.calcSubskills(Player.java:2318) [Error] HOVerwaltung: core.model.HOModel.calcSubskills(HOModel.java:548) [Error] HOVerwaltung: core.model.HOVerwaltung.recalcSubskills(HOVerwaltung.java:165) [Error] HOVerwaltung: core.gui.HOMainFrame.actionPerformed(HOMainFrame.java:326) [Error] HOVerwaltung: java.desktop/javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1972) [Error] HOVerwaltung: java.desktop/javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2313) [Error] HOVerwaltung: java.desktop/javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:405) [Error] HOVerwaltung: java.desktop/javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:262) [Error] HOVerwaltung: java.desktop/javax.swing.AbstractButton.doClick(AbstractButton.java:374) [Error] HOVerwaltung: java.desktop/javax.swing.plaf.basic.BasicMenuItemUI.doClick(BasicMenuItemUI.java:1028) [Error] HOVerwaltung: java.desktop/javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(BasicMenuItemUI.java:1072) [Error] HOVerwaltung: java.desktop/java.awt.Component.processMouseEvent(Component.java:6626) [Error] HOVerwaltung: java.desktop/javax.swing.JComponent.processMouseEvent(JComponent.java:3389) [Error] HOVerwaltung: java.desktop/java.awt.Component.processEvent(Component.java:6391) [Error] HOVerwaltung: java.desktop/java.awt.Container.processEvent(Container.java:2266) [Error] HOVerwaltung: java.desktop/java.awt.Component.dispatchEventImpl(Component.java:5001) [Error] HOVerwaltung: java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2324) [Error] HOVerwaltung: java.desktop/java.awt.Component.dispatchEvent(Component.java:4833) [Error] HOVerwaltung: java.desktop/java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4948) [Error] HOVerwaltung: java.desktop/java.awt.LightweightDispatcher.processMouseEvent(Container.java:4575) [Error] HOVerwaltung: java.desktop/java.awt.LightweightDispatcher.dispatchEvent(Container.java:4516) [Error] HOVerwaltung: java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2310) [Error] HOVerwaltung: java.desktop/java.awt.Window.dispatchEventImpl(Window.java:2780) [Error] HOVerwaltung: java.desktop/java.awt.Component.dispatchEvent(Component.java:4833) [Error] HOVerwaltung: java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:773) [Error] HOVerwaltung: java.desktop/java.awt.EventQueue$4.run(EventQueue.java:722) [Error] HOVerwaltung: java.desktop/java.awt.EventQueue$4.run(EventQueue.java:716) [Error] HOVerwaltung: java.base/java.security.AccessController.doPrivileged(AccessController.java:399) [Error] HOVerwaltung: java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86) [Error] HOVerwaltung: java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:97) [Error] HOVerwaltung: java.desktop/java.awt.EventQueue$5.run(EventQueue.java:746) [Error] HOVerwaltung: java.desktop/java.awt.EventQueue$5.run(EventQueue.java:744) [Error] HOVerwaltung: java.base/java.security.AccessController.doPrivileged(AccessController.java:399) [Error] HOVerwaltung: java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86) [Error] HOVerwaltung: java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:743) [Error] HOVerwaltung: java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203) [Error] HOVerwaltung: java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124) [Error] HOVerwaltung: java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113) [Error] HOVerwaltung: java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109) [Error] HOVerwaltung: java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) [Error] HOVerwaltung: java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90) [Debug] TrainingManager: HOVerwaltung model changed => TrainingManager and TrainingWeekManager are reinitialized

masterpatje commented 2 years ago

And I accidently overwrote a HRF file (from last training week). It is still in the database of a H0 6,0 version. Is it possible to extract a HRF file from the HO database? To recreate the HRF file?

masterpatje commented 2 years ago

Just started debugging, but right after the first player Jarne Clarebout I got an error. This was on a clean build with only the first HRF file downloaded. I directly checked the 7-weeks recalculation. I will

[Info] Player: Training 29 jul. 2022 01:05:00 Minutes= 90 Experience increment of Jarne Clarebout: 0.07000105001575023 new sub value=0.07000105001575023 [Debug] HOVerwaltung: recalcSubskills : [Error] HOVerwaltung: java.lang.NullPointerException: Cannot invoke "core.training.TrainingPoints.getTrainingDuration()" because the return value of "core.training.TrainingPerPlayer.getTrainingPair()" is null [Error] HOVerwaltung: core.model.player.Player.calcSubskills(Player.java:2318) [Error] HOVerwaltung: core.model.HOModel.calcSubskills(HOModel.java:548) [Error] HOVerwaltung: core.model.HOVerwaltung.recalcSubskills(HOVerwaltung.java:165) [Error] HOVerwaltung: core.gui.HOMainFrame.actionPerformed(HOMainFrame.java:326) [Error] HOVerwaltung: java.desktop/javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1972) [Error] HOVerwaltung: java.desktop/javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2313) [Error] HOVerwaltung: java.desktop/javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:405) [Error] HOVerwaltung: java.desktop/javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:262) [Error] HOVerwaltung: java.desktop/javax.swing.AbstractButton.doClick(AbstractButton.java:374) [Error] HOVerwaltung: java.desktop/javax.swing.plaf.basic.BasicMenuItemUI.doClick(BasicMenuItemUI.java:1028) [Error] HOVerwaltung: java.desktop/javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(BasicMenuItemUI.java:1072) [Error] HOVerwaltung: java.desktop/java.awt.Component.processMouseEvent(Component.java:6626) [Error] HOVerwaltung: java.desktop/javax.swing.JComponent.processMouseEvent(JComponent.java:3389) [Error] HOVerwaltung: java.desktop/java.awt.Component.processEvent(Component.java:6391) [Error] HOVerwaltung: java.desktop/java.awt.Container.processEvent(Container.java:2266) [Error] HOVerwaltung: java.desktop/java.awt.Component.dispatchEventImpl(Component.java:5001) [Error] HOVerwaltung: java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2324) [Error] HOVerwaltung: java.desktop/java.awt.Component.dispatchEvent(Component.java:4833) [Error] HOVerwaltung: java.desktop/java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4948) [Error] HOVerwaltung: java.desktop/java.awt.LightweightDispatcher.processMouseEvent(Container.java:4575) [Error] HOVerwaltung: java.desktop/java.awt.LightweightDispatcher.dispatchEvent(Container.java:4516) [Error] HOVerwaltung: java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2310) [Error] HOVerwaltung: java.desktop/java.awt.Window.dispatchEventImpl(Window.java:2780) [Error] HOVerwaltung: java.desktop/java.awt.Component.dispatchEvent(Component.java:4833) [Error] HOVerwaltung: java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:773) [Error] HOVerwaltung: java.desktop/java.awt.EventQueue$4.run(EventQueue.java:722) [Error] HOVerwaltung: java.desktop/java.awt.EventQueue$4.run(EventQueue.java:716) [Error] HOVerwaltung: java.base/java.security.AccessController.doPrivileged(AccessController.java:399) [Error] HOVerwaltung: java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86) [Error] HOVerwaltung: java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:97) [Error] HOVerwaltung: java.desktop/java.awt.EventQueue$5.run(EventQueue.java:746) [Error] HOVerwaltung: java.desktop/java.awt.EventQueue$5.run(EventQueue.java:744) [Error] HOVerwaltung: java.base/java.security.AccessController.doPrivileged(AccessController.java:399) [Error] HOVerwaltung: java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86) [Error] HOVerwaltung: java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:743) [Error] HOVerwaltung: java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203) [Error] HOVerwaltung: java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124) [Error] HOVerwaltung: java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113) [Error] HOVerwaltung: java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109) [Error] HOVerwaltung: java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) [Error] HOVerwaltung: java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90) [Debug] TrainingManager: HOVerwaltung model changed => TrainingManager and TrainingWeekManager are reinitialized

This only occurred when I had 1 HRF file. When I have multiple HRF file in the log file I get all the details of all players.

Interesting note 1: When I download the first HRF file the experience is not taking into consideration. This might be connected to the fact that this file counts as the base line on a clean installation. Interesting note 2: When I do a subskill recalculation (7-weeks ) on 1 HRF file, I can see that in the background HO did calculate experience; [Info] Player: Training 29 jul. 2022 01:05:00 Minutes= 90 Experience increment of Jarne Clarebout: 0.07000105001575023 new sub value=0.07000105001575023 but in the application this experience was not shown (I mean in the total experience).

I will look further into this at around 13:30. And now that I have the information so that I can follow it step by step I will present the conclusions hopefully around 14:45.

wsbrenk commented 2 years ago

@masterpatje new dev is created. your error log shows missing match info - now you should see a warning instead of stack trace in log.

wsbrenk commented 2 years ago

i don't know a possibility to recreate hrf files.

masterpatje commented 2 years ago

Oke @wsbrenk. I can see that the first step of the experience recalculation is rather strange. The starting value is not set at 0,0. It starts at a certain value (don't know where it is based on and then add experience (new sub value).

In the example below I have two players both correctly have a 0,035000525 gain, but started with a different starting experience value. However I can't see how it is related to a certain training week. So that I understand where the starting value came from. Although in my oppinion it should have started at 0,0.

image

@wsbrenk for the next version it would be nice if you could put a semicolon (";") after "[Info] Player" "Training .....00" "Minutes=" "[Minutes Experience]" "Experience increment of [PLAYER]=" "[Increment Value]" "New sub value=" "[New Sub Value].

In this way I can parse the output file in excel on the semicolomn (;)

wsbrenk commented 2 years ago

@masterpatje initial sub values should come from latest download before 7 weeks period.

; will come later ;-)

masterpatje commented 2 years ago

Oke good to know. Thank you for the fast response. I can proceed.

I will re-check if the initial values correspond to that week.

Will -start- to do so after 15 min. Expect reaction in 1h. Watching interesting phase F1 at the moment :-)

masterpatje commented 2 years ago

I will not combine my findings, but instead will post them one at a time. So that you can look at them when you can.

Oke, I calculated the initial experience from the latest downloaded by New Value subtracted (-/-) by the Experience Increment. And doing a comparison.

Finding Titus Calcagnille:

So how is his initial value determined?

His latest match was from 13/7/2022 (S70 W15), 45 minutes international friendly and he didn't play other matches that training week. This would mean a 0,0035 (50%*0,7/100) if that match is considered. But that is from a HRF file that is 3 weeks old.

masterpatje commented 2 years ago

Findings Initial Experience image

So the initial value can't be on the latest download (S71 Training Week 1) which is from 31/7/2022 and included 24/7/2022 (sunday INT friendly) 27/7/2022 (wednesday CUP) and

masterpatje commented 2 years ago

Finding Titus Calcagnille:

  • This is a player that I sold on 16/7/2022 so last possible training update is from S70 (training)W15
  • Is not in the latest download.
  • Still he has an initial experience: Starting Experience of Titus Calcagnile 0,021622547
  • original output: [Info] Player: Training 17 jun. 2022 01:05:00 Minutes= 114 Experience increment of Titus Calcagnile: 0.016333578337008388 new sub value=0.037956124897429014

So how is his initial value determined?

Extra information: Found out that I downloaded the HRF file on 19/7/2022 which did not include him or that training week. So now I will check the week before that (S70/W14) which is from a 8/7/2022 HRF file.

image

He was substituted after 60 minutes in the league game (3/7/2022) and got (60/90)*3,5/100 = 0,0233 gain there. HO initial experience value shows 0,02162.

masterpatje commented 2 years ago

Now I am going to look at the bigger picture and look at the experience gain from week to week. Will post results shortly.

wsbrenk commented 2 years ago

semicolons still useful? anything else?

the initial value should be found in latest hrf before 7 weeks period. you can check them in sql editor.

masterpatje commented 2 years ago

I am now working backwards. Fully checked the experience gain in S71 W1 and S70 W16 they fully matched (minutes experience and experience gain). Moving forward.

masterpatje commented 2 years ago

Experience gain

D. Wellinga image

[Info] Player: Training 8 jul. 2022 01:05:00 Minutes= 69 Experience increment of Dean Wellinga: 0.012522410058373097 new sub value=0.03087824095139205

masterpatje commented 2 years ago

Experience gain

J. Dieleman (27 + 19 = 46 minutes) image

[Info] Player: Training 1 jul. 2022 01:05:00 Minutes= 65 Experience increment of Jasper Dieleman: 0.016878030948242 new sub value=0.13377978447454486

masterpatje commented 2 years ago

Experience gain

J. Dieleman 2 minute difference HO7 121 en Hattrick 123 (55+68) image [Info] Player: Training 17 jun. 2022 01:05:00 Minutes= 121 Experience increment of Jasper Dieleman: 0.030567125173544266 new sub value=0.09341251229879557

A. Gamboa 2 minute difference HO7 171 en Hattrick 173 (83+90) In one match he exceeds the 90 minutes (65 +27 = 92) so I limited that one to 90 minutes for correct experience calculation, but still 2 minutes difference. image [Info] Player: Training 17 jun. 2022 01:05:00 Minutes= 121 Experience increment of Jasper Dieleman: 0.030567125173544266 new sub value=0.09341251229879557

C. Vredenbrecht. 2 minute difference HO7 121 en Hattrick 123 (33+90) In one match he exceeds the 90 minutes (92) so I limited that one to 90 minutes for correct experience calculation, but still 2 minutes difference with HO7. image [Info] Player: Training 17 jun. 2022 01:05:00 Minutes= 121 Experience increment of Carel Vredenbregt: 0.0374116722861954 new sub value=0.10741272230194562

masterpatje commented 2 years ago

semicolons still useful? anything else?

the initial value should be found in latest hrf before 7 weeks period. you can check them in sql editor.

That one is based on the HRF before the 7 weeks periode. It is Season 70 Training Week 10, which is June 10th (and includes 8th and 5th June matches). And it doesn't hold up.

image

Conclusion: Main difference is in the initial experience value, the gain in experience calculates correctly. On a few (minor) exceptions the minutes are incorrectly calculated for reasons unknown yet.

To fix this I would advise to set the initial experience value to 0,0 on recalculation (for both the 7-week and the total recalculation of subskills). For the 7-weeks recalculation the total experience is as a result exactly the combined experience of the last 7 training weeks, so that is what is good right?

masterpatje commented 2 years ago

semicolons still useful? anything else?

the initial value should be found in latest hrf before 7 weeks period. you can check them in sql editor.

How can I check them. Do i need to use commands or drag the HRF file to that window (didn't work)? image

masterpatje commented 2 years ago

In the log also with the 7-weeks subskill recalculation I saw in the output file that because when a warning is issued that no previous training was found apparently the code is searching for a previous training (far beyond the latest 7 weeks).

[Info] TrainingWeekManager: Next week training date: 2022-08-05 01:05:00 [Info] Player: Training 29 jul. 2022 01:05:00; Minutes= 90; Experience increment of Jarne Clarebout; increment: 0.07000105001575023; new sub value=0.07000105001575023 [Warning] Player: no training info found [Info] Player: Training 22 jul. 2022 01:05:00; Minutes= 0; Experience increment of Jarne Clarebout; increment: 0.0; new sub value=0.07000105001575023 [Warning] Player: no training info found ... [Info] Player: Training 17 sep. 2004 01:05:00; Minutes= 0; Experience increment of Jarne Clarebout; increment: 0.0; new sub value=0.07000105001575023 [Warning] Player: no training info found [Info] Player: Training 10 sep. 2004 01:05:00; Minutes= 0; Experience increment of Jarne Clarebout; increment: 0.0; new sub value=0.07000105001575023

masterpatje commented 2 years ago

Side note: In the output log sometimes the order in which the players are processed is reversed. This is quite interesting because normally in a loop you would expect the same order and calculation for all of the HRF files.
Jarne Clarebout has shirt number 1 and Alwin Arjun number 99.

Sometimes in a training week Clarebout is first processed and sometimes he is last. And vice versa with Alwin Arjun. output.log