search

guitargeek / XGBoost-FastForest

Minimal library code to deploy XGBoost models in C++.
MIT License
89 stars 30 forks source link

Different confidence score between FastForest and predict_proba of python? #23

Closed inkzk closed 7 months ago

inkzk commented 7 months ago

With XGBClassifier in python, I got [[0.05979478 0.9402052 ]] for binary classification by predict_proba. However, 0.9898 gotten in FastForest after logistic transformation while loading the xgb_model.txt dumped from python with absolute same input with python inference.

Why is the difference?

guitargeek commented 7 months ago

Hello @inkzk!

Which XGBoost version are you using? Depending on the version, you might have to change the base response that is assumed implicitly by XGBoost depending on it's version (that's why FastForest might make the wrong guess):

https://github.com/guitargeek/XGBoost-FastForest/blob/master/include/fastforest.h#L62

Does that solve your problem? If yes, where should I have in your opinion documented this so the question would not have come up?

Thanks! Jonas

inkzk commented 7 months ago

Hello @inkzk!

Which XGBoost version are you using? Depending on the version, you might have to change the base response that is assumed implicitly by XGBoost depending on it's version (that's why FastForest might make the wrong guess):

https://github.com/guitargeek/XGBoost-FastForest/blob/master/include/fastforest.h#L62

Does that solve your problem? If yes, where should I have in your opinion documented this so the question would not have come up?

Thanks! Jonas

@guitargeek Thanks for ur reply. xgboost=2.0.3, and I dumpped it with xgbc.get_booster().dump_model('file_name.txt')

guitargeek commented 7 months ago

Ok! Does it work if you call the FastForest with baseResponse=0.5?

inkzk commented 7 months ago

Ok! Does it work if you call the FastForest with baseResponse=0.5?

@guitargeek Still mismatch. below is my boost tree txt, could u please help verify that? 

booster[0]:
0:[f0<0.464745909] yes=1,no=2,missing=2
    1:[f1<0.688540101] yes=3,no=4,missing=4
        3:leaf=-0.343334645
        4:leaf=-0.0685302913
    2:[f5<0.925550044] yes=5,no=6,missing=6
        5:[f4<0.363732725] yes=7,no=8,missing=8
            7:[f1<0.46713987] yes=11,no=12,missing=12
                11:[f6<0.29543516] yes=15,no=16,missing=16
                    15:leaf=-0.11647056
                    16:leaf=0.578968108
                12:[f7<0.630501032] yes=17,no=18,missing=18
                    17:[f1<0.550881445] yes=23,no=24,missing=24
                        23:leaf=0.490999013
                        24:leaf=1.44462729
                    18:leaf=0.287130892
            8:[f0<0.957570314] yes=13,no=14,missing=14
                13:[f4<0.428245842] yes=19,no=20,missing=20
                    19:[f0<0.540145934] yes=25,no=26,missing=26
                        25:leaf=-0.249268338
                        26:leaf=0.506970406
                    20:[f6<0.446160495] yes=27,no=28,missing=28
                        27:leaf=-0.32993868
                        28:leaf=0.101085864
                14:[f4<0.40901643] yes=21,no=22,missing=22
                    21:leaf=0.1081504
                    22:leaf=0.976575971
        6:[f5<0.963987529] yes=9,no=10,missing=10
            9:leaf=-0.036273066
            10:leaf=-0.33453387
booster[1]:
0:[f0<0.464745909] yes=1,no=2,missing=2
    1:[f1<0.477993608] yes=3,no=4,missing=4
        3:leaf=-0.331933796
        4:[f2<0.698272228] yes=7,no=8,missing=8
            7:leaf=0.271315485
            8:leaf=-0.287922323
    2:[f5<0.963987529] yes=5,no=6,missing=6
        5:[f4<0.549994588] yes=9,no=10,missing=10
            9:[f1<0.608612716] yes=11,no=12,missing=12
                11:[f4<0.371538043] yes=15,no=16,missing=16
                    15:[f6<0.339155525] yes=19,no=20,missing=20
                        19:leaf=0.0493167825
                        20:leaf=0.690361202
                    16:[f3<0.322769463] yes=21,no=22,missing=22
                        21:leaf=0.139887109
                        22:leaf=-0.230679497
                12:[f2<0.99801147] yes=17,no=18,missing=18
                    17:[f7<0.627484322] yes=23,no=24,missing=24
                        23:leaf=0.689768255
                        24:leaf=0.238464281
                    18:[f1<0.747833788] yes=25,no=26,missing=26
                        25:leaf=-0.061183922
                        26:leaf=0.425076067
            10:[f5<0.684047699] yes=13,no=14,missing=14
                13:leaf=-0.312829792
                14:leaf=-0.0313760452
        6:leaf=-0.317647189
booster[2]:
0:[f0<0.464745909] yes=1,no=2,missing=2
    1:[f1<0.477993608] yes=3,no=4,missing=4
        3:leaf=-0.321564645
        4:[f4<0.338961661] yes=7,no=8,missing=8
            7:leaf=0.275449038
            8:leaf=-0.273527741
    2:[f5<0.925550044] yes=5,no=6,missing=6
        5:[f4<0.549994588] yes=9,no=10,missing=10
            9:[f1<0.637513697] yes=13,no=14,missing=14
                13:[f2<0.903187752] yes=17,no=18,missing=18
                    17:[f1<0.4708969] yes=21,no=22,missing=22
                        21:leaf=-0.0330066793
                        22:leaf=0.37014392
                    18:[f4<0.304078698] yes=23,no=24,missing=24
                        23:leaf=0.106471747
                        24:leaf=-0.294448912
                14:[f2<0.9998734] yes=19,no=20,missing=20
                    19:[f4<0.485309005] yes=25,no=26,missing=26
                        25:leaf=0.449053407
                        26:leaf=0.0443474092
                    20:[f2<0.999961495] yes=27,no=28,missing=28
                        27:leaf=-0.0995458886
                        28:leaf=0.247802898
            10:[f5<0.684047699] yes=15,no=16,missing=16
                15:leaf=-0.295679003
                16:leaf=-0.0476977304
        6:[f4<0.566825867] yes=11,no=12,missing=12
            11:leaf=-0.306028247
            12:leaf=-0.0505427159
booster[3]:
0:[f0<0.464745909] yes=1,no=2,missing=2
    1:[f1<0.477993608] yes=3,no=4,missing=4
        3:leaf=-0.313885659
        4:[f2<0.67366159] yes=7,no=8,missing=8
            7:leaf=0.259458095
            8:leaf=-0.260770559
    2:[f5<0.963987529] yes=5,no=6,missing=6
        5:[f4<0.576931775] yes=9,no=10,missing=10
            9:[f6<0.446160495] yes=11,no=12,missing=12
                11:[f1<0.706568837] yes=15,no=16,missing=16
                    15:[f2<0.825392783] yes=19,no=20,missing=20
                        19:leaf=0.12347234
                        20:leaf=-0.213362724
                    16:[f3<0.289854676] yes=21,no=22,missing=22
                        21:leaf=0.132648513
                        22:leaf=0.351812243
                12:[f1<0.439495146] yes=17,no=18,missing=18
                    17:leaf=-0.19111
                    18:[f6<0.596998692] yes=23,no=24,missing=24
                        23:leaf=0.521652699
                        24:leaf=0.0923499465
            10:[f3<0.528982401] yes=13,no=14,missing=14
                13:leaf=-0.0462808385
                14:leaf=-0.282250613
        6:leaf=-0.292778313
booster[4]:
0:[f0<0.464745909] yes=1,no=2,missing=2
    1:[f1<0.477993608] yes=3,no=4,missing=4
        3:leaf=-0.307864249
        4:[f2<0.603130698] yes=7,no=8,missing=8
            7:leaf=0.258893311
            8:leaf=-0.249272823
    2:[f5<0.963987529] yes=5,no=6,missing=6
        5:[f3<0.373620301] yes=9,no=10,missing=10
            9:[f6<0.34447065] yes=11,no=12,missing=12
                11:[f4<0.319153905] yes=15,no=16,missing=16
                    15:[f6<0.161429822] yes=21,no=22,missing=22
                        21:leaf=-0.0873716623
                        22:leaf=0.332722068
                    16:[f1<0.645725667] yes=23,no=24,missing=24
                        23:leaf=-0.173120648
                        24:leaf=0.112454645
                12:[f3<0.280642003] yes=17,no=18,missing=18
                    17:leaf=-0.00607322948
                    18:[f4<0.402831346] yes=25,no=26,missing=26
                        25:leaf=0.504794717
                        26:leaf=0.11193727
            10:[f1<0.481658846] yes=13,no=14,missing=14
                13:leaf=-0.312682331
                14:[f6<0.392236084] yes=19,no=20,missing=20
                    19:[f2<0.825392783] yes=27,no=28,missing=28
                        27:leaf=0.133586153
                        28:leaf=-0.233711541
                    20:[f4<0.576931775] yes=29,no=30,missing=30
                        29:leaf=0.284917057
                        30:leaf=-0.0806663111
        6:leaf=-0.281438321
booster[5]:
0:[f0<0.422372401] yes=1,no=2,missing=2
    1:[f1<0.577572346] yes=3,no=4,missing=4
        3:leaf=-0.304021776
        4:leaf=-0.0361476243
    2:[f6<0.705945671] yes=5,no=6,missing=6
        5:[f3<0.534836769] yes=7,no=8,missing=8
            7:[f1<0.372503698] yes=9,no=10,missing=10
                9:[f7<0.464734882] yes=13,no=14,missing=14
                    13:leaf=-0.0432436578
                    14:leaf=-0.2421837
                10:[f2<0.614885151] yes=15,no=16,missing=16
                    15:[f4<0.298274696] yes=19,no=20,missing=20
                        19:leaf=0.459163845
                        20:leaf=0.0409142524
                    16:[f1<0.637513697] yes=21,no=22,missing=22
                        21:leaf=-0.00313727115
                        22:leaf=0.172455892
            8:[f5<0.74511838] yes=11,no=12,missing=12
                11:[f6<0.427681804] yes=17,no=18,missing=18
                    17:leaf=-0.272538722
                    18:leaf=-0.0781817734
                12:leaf=0.0316307619
        6:leaf=-0.278460264
booster[6]:
0:[f0<0.422372401] yes=1,no=2,missing=2
    1:[f1<0.577572346] yes=3,no=4,missing=4
        3:leaf=-0.299147278
        4:leaf=-0.0309733022
    2:[f5<0.963987529] yes=5,no=6,missing=6
        5:[f3<0.534836769] yes=7,no=8,missing=8
            7:[f1<0.462907314] yes=9,no=10,missing=10
                9:[f3<0.369019806] yes=13,no=14,missing=14
                    13:[f5<0.325174898] yes=19,no=20,missing=20
                        19:leaf=-0.176679865
                        20:leaf=0.200613305
                    14:leaf=-0.274248511
                10:[f6<0.34447065] yes=15,no=16,missing=16
                    15:[f5<0.308020294] yes=21,no=22,missing=22
                        21:leaf=0.282350868
                        22:leaf=-0.0354972519
                    16:[f7<0.615278244] yes=23,no=24,missing=24
                        23:leaf=0.254819155
                        24:leaf=-0.0218059495
            8:[f5<0.592235029] yes=11,no=12,missing=12
                11:leaf=-0.259386897
                12:[f0<0.612285852] yes=17,no=18,missing=18
                    17:leaf=0.0781116933
                    18:leaf=-0.0929537565
        6:leaf=-0.261656344
booster[7]:
0:[f0<0.422372401] yes=1,no=2,missing=2
    1:[f1<0.577572346] yes=3,no=4,missing=4
        3:leaf=-0.294328511
        4:leaf=-0.0265360698
    2:[f6<0.705945671] yes=5,no=6,missing=6
        5:[f3<0.675634325] yes=7,no=8,missing=8
            7:[f6<0.455611378] yes=9,no=10,missing=10
                9:[f4<0.441228211] yes=13,no=14,missing=14
                    13:[f3<0.365643382] yes=17,no=18,missing=18
                        17:leaf=0.114852354
                        18:leaf=-0.0613230355
                    14:[f0<0.993976593] yes=19,no=20,missing=20
                        19:leaf=-0.280128807
                        20:leaf=0.148033738
                10:[f2<0.9998734] yes=15,no=16,missing=16
                    15:[f1<0.477993608] yes=21,no=22,missing=22
                        21:leaf=-0.0729891807
                        22:leaf=0.355017543
                    16:leaf=-0.0998855606
            8:[f2<0.807594597] yes=11,no=12,missing=12
                11:leaf=0.00971503649
                12:leaf=-0.24911049
        6:leaf=-0.258191258
booster[8]:
0:[f1<0.372503698] yes=1,no=2,missing=2
    1:[f5<0.419235408] yes=3,no=4,missing=4
        3:leaf=-0.291896462
        4:leaf=-0.0631172657
    2:[f2<0.903187752] yes=5,no=6,missing=6
        5:[f4<0.40580824] yes=7,no=8,missing=8
            7:[f3<0.369019806] yes=11,no=12,missing=12
                11:[f2<0.788840115] yes=19,no=20,missing=20
                    19:[f6<0.226668999] yes=31,no=32,missing=32
                        31:leaf=0.0228453334
                        32:leaf=0.366166145
                    20:[f1<0.533979893] yes=33,no=34,missing=34
                        33:leaf=-0.110894263
                        34:leaf=0.230455503
                12:[f7<0.724237621] yes=21,no=22,missing=22
                    21:[f7<0.597474337] yes=35,no=36,missing=36
                        35:leaf=0.022377545
                        36:leaf=-0.253474891
                    22:leaf=0.282094359
            8:[f5<0.526986599] yes=13,no=14,missing=14
                13:leaf=-0.257934153
                14:[f1<0.49794367] yes=23,no=24,missing=24
                    23:leaf=-0.169750243
                    24:leaf=0.260894358
        6:[f0<0.908376873] yes=9,no=10,missing=10
            9:[f2<0.969576836] yes=15,no=16,missing=16
                15:[f1<0.698552966] yes=25,no=26,missing=26
                    25:[f7<0.520251691] yes=37,no=38,missing=38
                        37:leaf=-0.22391817
                        38:leaf=-0.0584116913
                    26:[f7<0.606402755] yes=39,no=40,missing=40
                        39:leaf=0.209930107
                        40:leaf=-0.00777901663
                16:[f5<0.511613727] yes=27,no=28,missing=28
                    27:leaf=-0.321986586
                    28:leaf=-0.120414227
            10:[f5<0.963987529] yes=17,no=18,missing=18
                17:[f2<0.99801147] yes=29,no=30,missing=30
                    29:leaf=0.25882706
                    30:[f1<0.747833788] yes=41,no=42,missing=42
                        41:leaf=-0.152321249
                        42:leaf=0.0989169925
                18:leaf=-0.215295702
booster[9]:
0:[f1<0.372503698] yes=1,no=2,missing=2
    1:[f5<0.419235408] yes=3,no=4,missing=4
        3:leaf=-0.285553634
        4:leaf=-0.054535538
    2:[f2<0.903187752] yes=5,no=6,missing=6
        5:[f1<0.602751374] yes=7,no=8,missing=8
            7:[f3<0.404040575] yes=11,no=12,missing=12
                11:[f4<0.40580824] yes=19,no=20,missing=20
                    19:[f7<0.597474337] yes=25,no=26,missing=26
                        25:leaf=0.206440032
                        26:leaf=-0.0488133095
                    20:leaf=-0.20935376
                12:[f1<0.49421373] yes=21,no=22,missing=22
                    21:leaf=-0.23904404
                    22:[f7<0.534033895] yes=27,no=28,missing=28
                        27:leaf=-0.142795011
                        28:leaf=0.103450797
            8:[f7<0.485662758] yes=13,no=14,missing=14
                13:leaf=0.078453891
                14:leaf=0.287345886
        6:[f1<0.637513697] yes=9,no=10,missing=10
            9:[f4<0.304078698] yes=15,no=16,missing=16
                15:leaf=-0.0134606585
                16:leaf=-0.26535356
            10:[f5<0.963987529] yes=17,no=18,missing=18
                17:[f5<0.428650796] yes=23,no=24,missing=24
                    23:[f6<0.261970997] yes=29,no=30,missing=30
                        29:leaf=-0.17456913
                        30:leaf=0.0537675507
                    24:[f7<0.714149952] yes=31,no=32,missing=32
                        31:leaf=0.20446153
                        32:leaf=-0.113113686
                18:leaf=-0.210487455
booster[10]:
0:[f1<0.372503698] yes=1,no=2,missing=2
    1:[f5<0.419235408] yes=3,no=4,missing=4
        3:leaf=-0.278420001
        4:leaf=-0.0471023694
    2:[f4<0.226391301] yes=5,no=6,missing=6
        5:[f7<0.508040965] yes=7,no=8,missing=8
            7:leaf=0.000757406524
            8:leaf=0.319387585
        6:[f4<0.576931775] yes=9,no=10,missing=10
            9:[f6<0.705945671] yes=11,no=12,missing=12
                11:[f6<0.339155525] yes=13,no=14,missing=14
                    13:[f1<0.550881445] yes=15,no=16,missing=16
                        15:leaf=-0.187881082
                        16:leaf=0.023978075
                    14:[f7<0.597474337] yes=17,no=18,missing=18
                        17:leaf=0.129458472
                        18:leaf=-0.0704982281
                12:leaf=-0.22258167
            10:leaf=-0.203735173
booster[11]:
0:[f0<0.422372401] yes=1,no=2,missing=2
    1:[f1<0.49794367] yes=3,no=4,missing=4
        3:leaf=-0.270729005
        4:leaf=-0.0416518636
    2:[f4<0.280614644] yes=5,no=6,missing=6
        5:[f0<0.633971274] yes=7,no=8,missing=8
            7:[f6<0.161429822] yes=11,no=12,missing=12
                11:leaf=-0.0264336802
                12:leaf=0.2811459
            8:leaf=-0.0918180496
        6:[f0<0.908376873] yes=9,no=10,missing=10
            9:[f2<0.969576836] yes=13,no=14,missing=14
                13:[f5<0.385503411] yes=17,no=18,missing=18
                    17:[f5<0.291640192] yes=25,no=26,missing=26
                        25:leaf=0.0533528067
                        26:leaf=-0.200874999
                    18:[f7<0.538562596] yes=27,no=28,missing=28
                        27:leaf=-0.0923695266
                        28:leaf=0.165786266
                14:[f5<0.511613727] yes=19,no=20,missing=20
                    19:leaf=-0.299741477
                    20:leaf=-0.075053677
            10:[f2<0.999821126] yes=15,no=16,missing=16
                15:[f5<0.74511838] yes=21,no=22,missing=22
                    21:[f0<0.97059834] yes=29,no=30,missing=30
                        29:leaf=0.0192103647
                        30:leaf=0.23894991
                    22:leaf=-0.0374556482
                16:[f4<0.462543845] yes=23,no=24,missing=24
                    23:leaf=-0.212168157
                    24:leaf=0.0329220667
booster[12]:
0:[f2<0.494544029] yes=1,no=2,missing=2
    1:leaf=-0.262809455
    2:[f4<0.327130586] yes=3,no=4,missing=4
        3:[f2<0.848099828] yes=5,no=6,missing=6
            5:[f1<0.405181319] yes=9,no=10,missing=10
                9:leaf=-0.0547375083
                10:[f3<0.400596529] yes=17,no=18,missing=18
                    17:[f7<0.471964955] yes=29,no=30,missing=30
                        29:leaf=0.0823116228
                        30:leaf=0.317718297
                    18:leaf=0.0225857422
            6:[f7<0.60982132] yes=11,no=12,missing=12
                11:[f1<0.533979893] yes=19,no=20,missing=20
                    19:leaf=-0.076042816
                    20:[f6<0.265769839] yes=31,no=32,missing=32
                        31:leaf=0.0277713481
                        32:leaf=0.193506137
                12:leaf=-0.178375423
        4:[f1<0.61749512] yes=7,no=8,missing=8
            7:[f3<0.322769463] yes=13,no=14,missing=14
                13:[f4<0.402831346] yes=21,no=22,missing=22
                    21:leaf=0.180170134
                    22:leaf=-0.114291616
                14:[f4<0.363732725] yes=23,no=24,missing=24
                    23:[f6<0.269844413] yes=33,no=34,missing=34
                        33:leaf=-0.111085869
                        34:leaf=0.0691967607
                    24:[f2<0.67366159] yes=35,no=36,missing=36
                        35:leaf=-0.0159534644
                        36:leaf=-0.269385934
            8:[f2<0.99801147] yes=15,no=16,missing=16
                15:[f0<0.908376873] yes=25,no=26,missing=26
                    25:[f2<0.969576836] yes=37,no=38,missing=38
                        37:leaf=0.14410919
                        38:leaf=-0.114789091
                    26:leaf=0.249786764
                16:[f3<0.2983073] yes=27,no=28,missing=28
                    27:[f7<0.528133452] yes=39,no=40,missing=40
                        39:leaf=-0.313391507
                        40:leaf=-0.00153360132
                    28:[f7<0.522722125] yes=41,no=42,missing=42
                        41:leaf=0.156105861
                        42:leaf=-0.119221091
booster[13]:
0:[f2<0.494544029] yes=1,no=2,missing=2
    1:leaf=-0.251960576
    2:[f6<0.705945671] yes=3,no=4,missing=4
        3:[f3<0.675634325] yes=5,no=6,missing=6
            5:[f1<0.637513697] yes=7,no=8,missing=8
                7:[f4<0.40580824] yes=11,no=12,missing=12
                    11:[f6<0.34447065] yes=15,no=16,missing=16
                        15:leaf=-0.0412482992
                        16:leaf=0.148420811
                    12:[f6<0.455611378] yes=17,no=18,missing=18
                        17:leaf=-0.23686485
                        18:leaf=-0.0280051958
                8:[f2<0.99801147] yes=13,no=14,missing=14
                    13:[f7<0.627484322] yes=19,no=20,missing=20
                        19:leaf=0.259605259
                        20:leaf=0.0205010045
                    14:[f3<0.294231057] yes=21,no=22,missing=22
                        21:leaf=-0.135383695
                        22:leaf=0.0869348422
            6:[f2<0.807594597] yes=9,no=10,missing=10
                9:leaf=-0.0315424502
                10:leaf=-0.208580092
        4:leaf=-0.209767893
booster[14]:
0:[f1<0.325174659] yes=1,no=2,missing=2
    1:leaf=-0.241520524
    2:[f4<0.327130586] yes=3,no=4,missing=4
        3:[f3<0.437603265] yes=5,no=6,missing=6
            5:[f2<0.883008003] yes=9,no=10,missing=10
                9:[f1<0.49794367] yes=15,no=16,missing=16
                    15:[f6<0.200197995] yes=27,no=28,missing=28
                        27:leaf=-0.165324524
                        28:leaf=0.179644644
                    16:leaf=0.308889866
                10:[f3<0.201739192] yes=17,no=18,missing=18
                    17:leaf=0.114547119
                    18:[f5<0.413971484] yes=29,no=30,missing=30
                        29:leaf=-0.209084675
                        30:leaf=0.0405442677
            6:leaf=-0.156731039
        4:[f0<0.795038283] yes=7,no=8,missing=8
            7:[f4<0.401736468] yes=11,no=12,missing=12
                11:[f2<0.788840115] yes=19,no=20,missing=20
                    19:[f4<0.356014639] yes=31,no=32,missing=32
                        31:leaf=-0.054115057
                        32:leaf=0.129571512
                    20:[f0<0.633971274] yes=33,no=34,missing=34
                        33:leaf=-0.213016257
                        34:leaf=0.0229289196
                12:[f5<0.642284513] yes=21,no=22,missing=22
                    21:leaf=-0.22311987
                    22:leaf=-0.016692128
            8:[f2<0.999821126] yes=13,no=14,missing=14
                13:[f4<0.373637646] yes=23,no=24,missing=24
                    23:leaf=-0.0862897709
                    24:[f4<0.446527421] yes=35,no=36,missing=36
                        35:leaf=0.177606404
                        36:leaf=-0.0188792907
                14:[f2<0.999969244] yes=25,no=26,missing=26
                    25:leaf=-0.200768694
                    26:leaf=0.0429412611
booster[15]:
0:[f0<0.422372401] yes=1,no=2,missing=2
    1:[f3<0.208352372] yes=3,no=4,missing=4
        3:leaf=-0.0619732291
        4:leaf=-0.229850158
    2:[f6<0.705945671] yes=5,no=6,missing=6
        5:[f4<0.29255116] yes=7,no=8,missing=8
            7:[f6<0.165122598] yes=9,no=10,missing=10
                9:leaf=-0.0872740075
                10:[f5<0.428650796] yes=13,no=14,missing=14
                    13:leaf=0.228381619
                    14:leaf=0.00277113914
            8:[f1<0.46713987] yes=11,no=12,missing=12
                11:[f3<0.369019806] yes=15,no=16,missing=16
                    15:[f1<0.409300059] yes=19,no=20,missing=20
                        19:leaf=0.0222900845
                        20:leaf=-0.0629349649
                    16:leaf=-0.202644885
                12:[f6<0.34447065] yes=17,no=18,missing=18
                    17:[f6<0.317662865] yes=21,no=22,missing=22
                        21:leaf=0.010082203
                        22:leaf=-0.196786657
                    18:[f2<0.997871041] yes=23,no=24,missing=24
                        23:leaf=0.136315882
                        24:leaf=-0.0428045839
        6:leaf=-0.19570154
booster[16]:
0:[f2<0.494544029] yes=1,no=2,missing=2
    1:leaf=-0.219898552
    2:[f6<0.705945671] yes=3,no=4,missing=4
        3:[f3<0.675634325] yes=5,no=6,missing=6
            5:[f1<0.571113765] yes=7,no=8,missing=8
                7:[f2<0.690423727] yes=11,no=12,missing=12
                    11:[f1<0.4708969] yes=15,no=16,missing=16
                        15:leaf=-0.0377374217
                        16:leaf=0.17511788
                    12:[f6<0.34447065] yes=17,no=18,missing=18
                        17:leaf=-0.175342739
                        18:leaf=0.00834679697
                8:[f0<0.600520432] yes=13,no=14,missing=14
                    13:leaf=0.278150171
                    14:[f4<0.4001638] yes=19,no=20,missing=20
                        19:leaf=-0.0722937435
                        20:leaf=0.0902283937
            6:[f7<0.551590323] yes=9,no=10,missing=10
                9:leaf=-0.194653437
                10:leaf=-0.0113157313
        4:leaf=-0.181837022
booster[17]:
0:[f1<0.325174659] yes=1,no=2,missing=2
    1:leaf=-0.20747526
    2:[f4<0.383024812] yes=3,no=4,missing=4
        3:[f7<0.448171556] yes=5,no=6,missing=6
            5:leaf=-0.109604843
            6:[f3<0.437603265] yes=9,no=10,missing=10
                9:[f2<0.977534413] yes=15,no=16,missing=16
                    15:[f6<0.151116922] yes=19,no=20,missing=20
                        19:leaf=-0.0337385722
                        20:leaf=0.159264073
                    16:[f6<0.286427647] yes=21,no=22,missing=22
                        21:leaf=0.102646679
                        22:leaf=-0.202553332
                10:leaf=-0.0927646011
        4:[f5<0.426278144] yes=7,no=8,missing=8
            7:[f3<0.269215077] yes=11,no=12,missing=12
                11:leaf=-0.00906175561
                12:leaf=-0.222224444
            8:[f4<0.392702997] yes=13,no=14,missing=14
                13:leaf=-0.128316253
                14:[f5<0.500192344] yes=17,no=18,missing=18
                    17:[f0<0.957570314] yes=23,no=24,missing=24
                        23:leaf=0.0148146516
                        24:leaf=0.237045392
                    18:[f3<0.512349129] yes=25,no=26,missing=26
                        25:leaf=-0.111909963
                        26:leaf=0.0763963088
booster[18]:
0:[f1<0.46713987] yes=1,no=2,missing=2
    1:[f0<0.540145934] yes=3,no=4,missing=4
        3:leaf=-0.19723703
        4:[f2<0.737321436] yes=7,no=8,missing=8
            7:leaf=0.132516995
            8:leaf=-0.0996495336
    2:[f2<0.848099828] yes=5,no=6,missing=6
        5:[f0<0.612285852] yes=9,no=10,missing=10
            9:[f1<0.543526292] yes=13,no=14,missing=14
                13:[f2<0.645647883] yes=21,no=22,missing=22
                    21:leaf=0.141625986
                    22:leaf=-0.149567693
                14:leaf=0.265230954
            10:[f1<0.539158821] yes=15,no=16,missing=16
                15:leaf=0.105466016
                16:[f2<0.720247805] yes=23,no=24,missing=24
                    23:leaf=-0.216264173
                    24:leaf=0.0121331559
        6:[f1<0.673115373] yes=11,no=12,missing=12
            11:[f0<0.948889911] yes=17,no=18,missing=18
                17:[f6<0.34447065] yes=25,no=26,missing=26
                    25:leaf=-0.226914674
                    26:leaf=-0.0403409936
                18:leaf=0.0620711297
            12:[f2<0.99801147] yes=19,no=20,missing=20
                19:[f5<0.39393115] yes=27,no=28,missing=28
                    27:leaf=-0.00607117312
                    28:leaf=0.166878998
                20:[f3<0.2983073] yes=29,no=30,missing=30
                    29:[f4<0.327130586] yes=31,no=32,missing=32
                        31:leaf=-0.0180181284
                        32:leaf=-0.194830343
                    30:[f3<0.326372564] yes=33,no=34,missing=34
                        33:leaf=0.197316736
                        34:leaf=-0.0569760092
booster[19]:
0:[f0<0.422372401] yes=1,no=2,missing=2
    1:leaf=-0.164744183
    2:[f3<0.373620301] yes=3,no=4,missing=4
        3:[f2<0.977534413] yes=5,no=6,missing=6
            5:[f0<0.570021033] yes=9,no=10,missing=10
                9:[f3<0.306287706] yes=15,no=16,missing=16
                    15:leaf=0.121178553
                    16:leaf=-0.119341843
                10:[f4<0.402831346] yes=17,no=18,missing=18
                    17:[f6<0.29543516] yes=25,no=26,missing=26
                        25:leaf=0.0329033472
                        26:leaf=0.20990245
                    18:leaf=-0.0107636759
            6:[f0<0.963036537] yes=11,no=12,missing=12
                11:[f1<0.688540101] yes=19,no=20,missing=20
                    19:leaf=-0.176529199
                    20:leaf=-0.0322653912
                12:[f6<0.232030913] yes=21,no=22,missing=22
                    21:leaf=-0.146197587
                    22:[f7<0.528133452] yes=27,no=28,missing=28
                        27:leaf=0.0144899823
                        28:leaf=0.170358211
        4:[f1<0.477993608] yes=7,no=8,missing=8
            7:leaf=-0.196281388
            8:[f0<0.557111859] yes=13,no=14,missing=14
                13:leaf=0.175396875
                14:[f0<0.894690931] yes=23,no=24,missing=24
                    23:[f6<0.383571625] yes=29,no=30,missing=30
                        29:leaf=-0.199342474
                        30:leaf=-0.0318984278
                    24:[f7<0.538562596] yes=31,no=32,missing=32
                        31:leaf=0.143692642
                        32:leaf=-0.101043411
guitargeek commented 7 months ago

Thanks! Can you please also attach the output of:

xgbc.get_booster().save_model('model.json')

Maybe attach the file instead of copy-pasting the output, because this quite a long one.

Also, what are the feature values at the point where you get the mismatch?

inkzk commented 7 months ago 
c.get_booster().save_model('model.json')

@guitargeek Sorry for my network regulation, no attachment file could be uploaded...

Let feature be X = [[0.5]*8] Output of python is [[0.5282326 , 0.47176743]], while fastforest got 0.90 after sigmoid with baseResponse=0.5

guitargeek commented 7 months ago

@guitargeek Sorry for my network regulation, no attachment file could be uploaded...

Then maybe you could just copy-paste the content of a JSON file to a post here, just like you did with the txt dump?

If I don't have access to the original xgboost model, it's very hard to debug these kind of problems, even if I would like to :slightly_smiling_face:

The problem with the txt files is that they can't be read back in to xgboost.

inkzk commented 7 months ago

@guitargeek Nice idea! Json content as below:

{"learner":{"attributes":{},"feature_names":[],"feature_types":[],"gradient_booster":{"model":{"gbtree_model_param":{"num_parallel_tree":"1","num_trees":"20"},"iteration_indptr":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20],"tree_info":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"trees":[{"base_weights":[-7.896569E-1,-1.1392244E0,1.5847175E-1,-3.4333465E-1,-6.853029E-2,8.785152E-1,-1.0787609E0,3.0550668E0,4.3979958E-2,-3.6273066E-2,-3.3453387E-1,8.2333624E-1,3.7162564E0,-4.3495327E-1,2.7238045E0,-1.1647056E-1,5.789681E-1,4.1005197E0,2.871309E-1,8.139033E-1,-8.807087E-1,1.081504E-1,9.76576E-1,4.90999E-1,1.4446273E0,-2.4926834E-1,5.069704E-1,-3.2993868E-1,1.01085864E-1],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":0,"left_children":[1,3,5,-1,-1,7,9,11,13,-1,-1,15,17,19,21,-1,-1,23,-1,25,27,-1,-1,-1,-1,-1,-1,-1,-1],"loss_changes":[8.500905E1,6.28952E-1,6.315004E1,0E0,0E0,8.103633E1,7.751961E-1,1.5194458E1,4.394534E1,0E0,0E0,6.2620783E0,5.891861E0,1.6931664E1,5.3439865E0,0E0,0E0,5.897354E0,0E0,1.3192354E1,6.2252064E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,1,1,2,2,5,5,6,6,7,7,8,8,11,11,12,12,13,13,14,14,17,17,19,19,20,20],"right_children":[2,4,6,-1,-1,8,10,12,14,-1,-1,16,18,20,22,-1,-1,24,-1,26,28,-1,-1,-1,-1,-1,-1,-1,-1],"split_conditions":[4.647459E-1,6.885401E-1,9.2555004E-1,-3.4333465E-1,-6.853029E-2,3.6373273E-1,9.639875E-1,4.6713987E-1,9.575703E-1,-3.6273066E-2,-3.3453387E-1,2.9543516E-1,6.3050103E-1,4.2824584E-1,4.0901643E-1,-1.1647056E-1,5.789681E-1,5.5088145E-1,2.871309E-1,5.4014593E-1,4.461605E-1,1.081504E-1,9.76576E-1,4.90999E-1,1.4446273E0,-2.4926834E-1,5.069704E-1,-3.2993868E-1,1.01085864E-1],"split_indices":[0,1,5,0,0,4,5,1,0,0,0,6,7,4,4,0,0,1,0,0,6,0,0,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[2.549432E2,1.8612051E2,6.88227E1,1.8480391E2,1.3166081E0,4.368745E1,2.5135246E1,1.1370707E1,3.2316746E1,1.0772249E0,2.4058022E1,3.1119828E0,8.258723E0,2.8127537E1,4.1892076E0,1.7953748E0,1.3166081E0,6.822424E0,1.4362998E0,7.181499E0,2.094604E1,1.0772249E0,3.1119828E0,2.2741413E0,4.5482826E0,2.5135245E0,4.6679745E0,1.7834055E1,3.1119828E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"29","size_leaf_vector":"1"}},{"base_weights":[-7.147649E-1,-1.0864061E0,5.4863807E-2,-3.319338E-1,-6.0393775E-1,4.7413844E-1,-3.176472E-1,2.713155E-1,-2.8792232E-1,9.3374795E-1,-9.192565E-1,2.6414096E-1,1.734023E0,-3.128298E-1,-3.1376045E-2,1.1312582E0,-4.7864425E-1,2.1647131E0,8.1582403E-1,4.9316783E-2,6.903612E-1,1.3988711E-1,-2.306795E-1,6.8976825E-1,2.3846428E-1,-6.1183922E-2,4.2507607E-1],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":1,"left_children":[1,3,5,-1,7,9,-1,-1,-1,11,13,15,17,-1,-1,19,21,23,25,-1,-1,-1,-1,-1,-1,-1,-1],"loss_changes":[6.1380104E1,7.3005676E-1,3.354506E1,0E0,5.1448402E0,3.4224724E1,0E0,0E0,0E0,2.0802185E1,1.2419329E0,1.5213824E1,5.310299E0,0E0,0E0,1.1847307E1,3.9169288E0,3.0535507E-1,4.774895E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,1,1,2,2,4,4,5,5,9,9,10,10,11,11,12,12,15,15,16,16,17,17,18,18],"right_children":[2,4,6,-1,8,10,-1,-1,-1,12,14,16,18,-1,-1,20,22,24,26,-1,-1,-1,-1,-1,-1,-1,-1],"split_conditions":[4.647459E-1,4.779936E-1,9.639875E-1,-3.319338E-1,6.982722E-1,5.499946E-1,-3.176472E-1,2.713155E-1,-2.8792232E-1,6.086127E-1,6.840477E-1,3.7153804E-1,9.9801147E-1,-3.128298E-1,-3.1376045E-2,3.3915552E-1,3.2276946E-1,6.274843E-1,7.478338E-1,4.9316783E-2,6.903612E-1,1.3988711E-1,-2.306795E-1,6.8976825E-1,2.3846428E-1,-6.1183922E-2,4.2507607E-1],"split_indices":[0,1,5,0,2,4,0,0,0,1,5,4,2,0,0,6,3,7,1,0,0,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[2.1338803E2,1.4362314E2,6.976489E1,1.3665382E2,6.9693213E0,5.1100964E1,1.8663929E1,1.0359415E0,5.9333797E0,3.8674206E1,1.2426761E1,2.1780262E1,1.6893942E1,1.0547675E1,1.8790858E0,9.807789E0,1.1972473E1,1.0575728E1,6.318214E0,5.9943967E0,3.8133917E0,2.6711435E0,9.30133E0,9.012272E0,1.5634563E0,2.588124E0,3.7300901E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"27","size_leaf_vector":"1"}},{"base_weights":[-6.497301E-1,-1.0466244E0,2.3880836E-2,-3.2156464E-1,-5.1621586E-1,3.458042E-1,-9.66296E-1,2.7544904E-1,-2.7352774E-1,6.504652E-1,-9.1720057E-1,-3.0602825E-1,-5.0542716E-2,2.2202057E-1,1.2656575E0,-2.95679E-1,-4.769773E-2,6.3851434E-1,-7.1812445E-1,1.4039876E0,3.3696324E-1,-3.300668E-2,3.7014392E-1,1.0647175E-1,-2.944489E-1,4.490534E-1,4.434741E-2,-9.954589E-2,2.478029E-1],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":2,"left_children":[1,3,5,-1,7,9,11,-1,-1,13,15,-1,-1,17,19,-1,-1,21,23,25,27,-1,-1,-1,-1,-1,-1,-1,-1],"loss_changes":[4.7167603E1,9.3344116E-1,2.1484993E1,0E0,4.793619E0,2.0147688E1,5.381985E-1,0E0,0E0,1.0662827E1,3.7688828E-1,0E0,0E0,1.0516996E1,1.6893501E0,0E0,0E0,8.304064E0,2.7100616E0,1.4507828E0,1.4181658E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,1,1,2,2,4,4,5,5,6,6,9,9,10,10,13,13,14,14,17,17,18,18,19,19,20,20],"right_children":[2,4,6,-1,8,10,12,-1,-1,14,16,-1,-1,18,20,-1,-1,22,24,26,28,-1,-1,-1,-1,-1,-1,-1,-1],"split_conditions":[4.647459E-1,4.779936E-1,9.2555004E-1,-3.2156464E-1,3.3896166E-1,5.499946E-1,5.6682587E-1,2.7544904E-1,-2.7352774E-1,6.375137E-1,6.840477E-1,-3.0602825E-1,-5.0542716E-2,9.0318775E-1,9.998734E-1,-2.95679E-1,-4.769773E-2,4.708969E-1,3.040787E-1,4.85309E-1,9.999615E-1,-3.300668E-2,3.7014392E-1,1.0647175E-1,-2.944489E-1,4.490534E-1,4.434741E-2,-9.954589E-2,2.478029E-1],"split_indices":[0,1,5,0,4,4,4,0,0,1,5,0,0,2,2,0,0,1,4,4,2,0,0,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[1.7533136E2,1.09978065E2,6.535329E1,1.0400457E2,5.973495E0,4.980998E1,1.5543308E1,1.0055919E0,4.967903E0,4.0520615E1,9.289363E0,1.430041E1,1.2428982E0,2.4687881E1,1.5832734E1,8.245642E0,1.043721E0,1.7330935E1,7.3569465E0,1.3334724E1,2.49801E0,8.038455E0,9.29248E0,1.3813797E0,5.975567E0,1.2238444E1,1.0962801E0,1.1909093E0,1.3071007E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"29","size_leaf_vector":"1"}},{"base_weights":[-5.917498E-1,-1.0143491E0,-2.0134293E-3,-3.1388566E-1,-4.3686932E-1,2.326622E-1,-2.927783E-1,2.594581E-1,-2.6077056E-1,4.327741E-1,-8.490039E-1,2.1819961E-1,1.2096853E0,-4.628084E-2,-2.822506E-1,-8.399033E-2,8.6102366E-1,-1.9111E-1,1.5827035E0,1.2347234E-1,-2.1336272E-1,1.3264851E-1,3.5181224E-1,5.216527E-1,9.234995E-2],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":3,"left_children":[1,3,5,-1,7,9,-1,-1,-1,11,13,15,17,-1,-1,19,21,-1,23,-1,-1,-1,-1,-1,-1],"loss_changes":[3.6284786E1,1.1071701E0,1.4347766E1,0E0,4.2197356E0,1.1318657E1,0E0,0E0,0E0,6.963955E0,3.8032484E-1,6.9089603E0,7.3203964E0,0E0,0E0,8.0171385E0,9.6233463E-1,0E0,1.0393353E0,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,1,1,2,2,4,4,5,5,9,9,10,10,11,11,12,12,15,15,16,16,18,18],"right_children":[2,4,6,-1,8,10,-1,-1,-1,12,14,16,18,-1,-1,20,22,-1,24,-1,-1,-1,-1,-1,-1],"split_conditions":[4.647459E-1,4.779936E-1,9.639875E-1,-3.1388566E-1,6.736616E-1,5.769318E-1,-2.927783E-1,2.594581E-1,-2.6077056E-1,4.461605E-1,5.289824E-1,7.0656884E-1,4.3949515E-1,-4.628084E-2,-2.822506E-1,8.253928E-1,2.8985468E-1,-1.9111E-1,5.969987E-1,1.2347234E-1,-2.1336272E-1,1.3264851E-1,3.5181224E-1,5.216527E-1,9.234995E-2],"split_indices":[0,1,5,0,2,4,0,0,0,6,3,1,1,0,0,2,3,0,6,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[1.4460031E2,8.3817505E1,6.0782806E1,7.865577E1,5.1617355E0,4.9593742E1,1.1189065E1,1.0375804E0,4.124155E0,4.2357384E1,7.2363586E0,3.4194176E1,8.163206E0,1.1577238E0,6.078635E0,2.3851152E1,1.0343024E1,1.2528552E0,6.9103513E0,1.3515993E1,1.0335159E1,5.444694E0,4.8983297E0,5.8321295E0,1.0782218E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"25","size_leaf_vector":"1"}},{"base_weights":[-5.3273785E-1,-9.859245E-1,-1.365614E-2,-3.0786425E-1,-3.6682275E-1,1.6801894E-1,-2.8143832E-1,2.588933E-1,-2.4927282E-1,5.710793E-1,-2.5233057E-1,2.7632946E-1,1.0830032E0,-3.1268233E-1,6.756418E-2,7.719474E-1,-3.441404E-2,-6.0732295E-3,1.2865145E0,-4.1657162E-1,7.0806307E-1,-8.737166E-2,3.3272207E-1,-1.7312065E-1,1.12454645E-1,5.047947E-1,1.1193727E-1,1.3358615E-1,-2.3371154E-1,2.8491706E-1,-8.066631E-2],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":4,"left_children":[1,3,5,-1,7,9,-1,-1,-1,11,13,15,17,-1,19,21,23,-1,25,27,29,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1],"loss_changes":[2.8527653E1,1.2389183E0,9.826062E0,0E0,3.863677E0,8.439132E0,0E0,0E0,0E0,3.5182695E0,6.3018036E0,2.7060857E0,2.024724E0,0E0,6.0287294E0,2.7414594E0,2.8038557E0,0E0,2.0970716E0,3.8333516E0,2.2220855E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,1,1,2,2,4,4,5,5,9,9,10,10,11,11,12,12,14,14,15,15,16,16,18,18,19,19,20,20],"right_children":[2,4,6,-1,8,10,-1,-1,-1,12,14,16,18,-1,20,22,24,-1,26,28,30,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1],"split_conditions":[4.647459E-1,4.779936E-1,9.639875E-1,-3.0786425E-1,6.031307E-1,3.736203E-1,-2.8143832E-1,2.588933E-1,-2.4927282E-1,3.4447065E-1,4.8165885E-1,3.191539E-1,2.80642E-1,-3.1268233E-1,3.9223608E-1,1.6142982E-1,6.4572567E-1,-6.0732295E-3,4.0283135E-1,8.253928E-1,5.769318E-1,-8.737166E-2,3.3272207E-1,-1.7312065E-1,1.12454645E-1,5.047947E-1,1.1193727E-1,1.3358615E-1,-2.3371154E-1,2.8491706E-1,-8.066631E-2],"split_indices":[0,1,5,0,2,3,0,0,0,6,1,4,3,0,6,6,1,0,4,2,4,0,0,0,0,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[1.2032719E2,6.3760994E1,5.656619E1,5.9247707E1,4.5132885E0,4.795954E1,8.6066475E0,1.00105E0,3.5122385E0,2.4300219E1,2.3659325E1,1.6398346E1,7.9018726E0,6.1685677E0,1.7490757E1,5.747392E0,1.0650953E1,1.3708638E0,6.5310087E0,1.0160399E1,7.3303576E0,1.4164581E0,4.330934E0,4.402937E0,6.2480164E0,3.9679294E0,2.5630794E0,2.9403841E0,7.2200155E0,5.8990316E0,1.431326E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"31","size_leaf_vector":"1"}},{"base_weights":[-4.759222E-1,-9.9438477E-1,-4.1747212E-2,-3.0402178E-1,-3.6147624E-2,9.22965E-2,-2.7846026E-1,2.643184E-1,-6.9974834E-1,-6.822539E-1,3.5981783E-1,-8.559838E-1,3.1630762E-2,-4.3243658E-2,-2.421837E-1,1.1589437E0,2.6797223E-1,-2.7253872E-1,-7.818177E-2,4.5916384E-1,4.0914252E-2,-3.1372712E-3,1.7245589E-1],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":5,"left_children":[1,3,5,-1,-1,7,-1,9,11,13,15,17,-1,-1,-1,19,21,-1,-1,-1,-1,-1,-1],"loss_changes":[2.2883474E1,6.4870834E-1,6.7956605E0,0E0,0E0,6.992277E0,0E0,4.0216637E0,1.2462282E0,1.6441822E-1,2.5225024E0,7.129669E-3,0E0,0E0,0E0,1.1889586E0,3.0753055E0,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,1,1,2,2,5,5,7,7,8,8,9,9,10,10,11,11,15,15,16,16],"right_children":[2,4,6,-1,-1,8,-1,10,12,14,16,18,-1,-1,-1,20,22,-1,-1,-1,-1,-1,-1],"split_conditions":[4.223724E-1,5.7757235E-1,7.059457E-1,-3.0402178E-1,-3.6147624E-2,5.3483677E-1,-2.7846026E-1,3.725037E-1,7.451184E-1,4.6473488E-1,6.1488515E-1,4.276818E-1,3.1630762E-2,-4.3243658E-2,-2.421837E-1,2.982747E-1,6.375137E-1,-2.7253872E-1,-7.818177E-2,4.5916384E-1,4.0914252E-2,-3.1372712E-3,1.7245589E-1],"split_indices":[0,1,6,0,0,3,0,1,5,7,2,6,0,0,0,4,1,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[1.00841995E2,4.537178E1,5.5470215E1,4.424901E1,1.1227717E0,4.8962334E1,6.5078816E0,4.07732E1,8.18913E0,3.1735458E0,3.7599655E1,6.805517E0,1.3836126E0,1.0042374E0,2.1693084E0,2.678273E0,3.4921383E1,5.770985E0,1.0345318E0,1.600039E0,1.078234E0,1.8815863E1,1.610552E1],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"23","size_leaf_vector":"1"}},{"base_weights":[-4.1776738E-1,-9.7250473E-1,-4.355241E-2,-2.9914728E-1,-3.0973302E-2,7.0069075E-2,-2.6165634E-1,2.0444803E-1,-6.3239014E-1,-2.966366E-1,3.4221393E-1,-2.593869E-1,-5.5697262E-2,3.112167E-1,-2.742485E-1,8.155665E-2,7.167108E-1,7.811169E-2,-9.295376E-2,-1.7667986E-1,2.006133E-1,2.8235087E-1,-3.5497252E-2,2.5481915E-1,-2.180595E-2],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":6,"left_children":[1,3,5,-1,-1,7,-1,9,11,13,15,-1,17,19,-1,21,23,-1,-1,-1,-1,-1,-1,-1,-1],"loss_changes":[1.819202E1,6.5431213E-1,5.098237E0,0E0,0E0,4.657916E0,0E0,2.9336908E0,1.0163069E0,3.8065333E0,3.1280875E0,0E0,3.4982878E-1,2.1555214E0,0E0,3.6123235E0,1.4514728E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,1,1,2,2,5,5,7,7,8,8,9,9,10,10,12,12,13,13,15,15,16,16],"right_children":[2,4,6,-1,-1,8,-1,10,12,14,16,-1,18,20,-1,22,24,-1,-1,-1,-1,-1,-1,-1,-1],"split_conditions":[4.223724E-1,5.7757235E-1,9.639875E-1,-2.9914728E-1,-3.0973302E-2,5.3483677E-1,-2.6165634E-1,4.629073E-1,5.92235E-1,3.690198E-1,3.4447065E-1,-2.593869E-1,6.1228585E-1,3.251749E-1,-2.742485E-1,3.080203E-1,6.1527824E-1,7.811169E-2,-9.295376E-2,-1.7667986E-1,2.006133E-1,2.8235087E-1,-3.5497252E-2,2.5481915E-1,-2.180595E-2],"split_indices":[0,1,5,0,0,3,0,1,5,3,6,0,0,5,0,5,7,0,0,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[8.683796E1,3.433736E1,5.2500595E1,3.3247314E1,1.0900472E0,4.6974895E1,5.5257006E0,4.0026806E1,6.948089E0,8.382958E0,3.1643847E1,4.597435E0,2.3506541E0,4.474532E0,3.9084263E0,1.937573E1,1.2268117E1,1.0367815E0,1.3138725E0,1.0878923E0,3.3866396E0,2.9557025E0,1.6420027E1,1.0437684E1,1.8304323E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"25","size_leaf_vector":"1"}},{"base_weights":[-3.6616084E-1,-9.4929737E-1,-4.7397055E-2,-2.943285E-1,-2.653607E-2,4.3888416E-2,-2.5819126E-1,1.4120397E-1,-6.415347E-1,2.7183123E-2,6.5404207E-1,9.7150365E-3,-2.4911049E-1,1.8365052E-1,-5.929771E-1,8.667714E-1,-9.988556E-2,1.14852354E-1,-6.1323036E-2,-2.801288E-1,1.4803374E-1,-7.298918E-2,3.5501754E-1],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":7,"left_children":[1,3,5,-1,-1,7,-1,9,11,13,15,-1,-1,17,19,21,-1,-1,-1,-1,-1,-1,-1],"loss_changes":[1.420888E1,6.5364265E-1,3.7965732E0,0E0,0E0,3.1905138E0,0E0,2.4098177E0,7.851434E-1,3.5058625E0,1.864996E0,0E0,0E0,2.2981153E0,3.1595948E0,2.5563955E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,1,1,2,2,5,5,7,7,8,8,9,9,10,10,13,13,14,14,15,15],"right_children":[2,4,6,-1,-1,8,-1,10,12,14,16,-1,-1,18,20,22,-1,-1,-1,-1,-1,-1,-1],"split_conditions":[4.223724E-1,5.7757235E-1,7.059457E-1,-2.943285E-1,-2.653607E-2,6.756343E-1,-2.5819126E-1,4.5561138E-1,8.075946E-1,4.412282E-1,9.998734E-1,9.7150365E-3,-2.4911049E-1,3.6564338E-1,9.939766E-1,4.779936E-1,-9.988556E-2,1.14852354E-1,-6.1323036E-2,-2.801288E-1,1.4803374E-1,-7.298918E-2,3.5501754E-1],"split_indices":[0,1,6,0,0,3,0,6,2,4,2,0,0,3,0,1,0,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[7.5682045E1,2.6049623E1,4.9632427E1,2.4986969E1,1.0626537E0,4.547404E1,4.1583853E0,4.0515663E1,4.9583755E0,3.400765E1,6.5080175E0,1.2665963E0,3.6917794E0,2.7718174E1,6.289473E0,5.4542546E0,1.0537628E0,1.8325293E1,9.392881E0,4.8946514E0,1.3948215E0,1.2618387E0,4.1924157E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"23","size_leaf_vector":"1"}},{"base_weights":[-3.1917623E-1,-9.4074917E-1,-4.4039696E-2,-2.9189646E-1,-6.3117266E-2,2.7929157E-1,-3.284983E-1,4.935452E-1,-4.1546312E-1,-6.9547695E-1,4.5343105E-2,8.181726E-1,5.2223813E-2,-2.5793415E-1,2.2925828E-1,-1.728001E-1,-9.6922433E-1,2.648842E-1,-2.152957E-1,9.7994673E-1,2.1603519E-1,-2.11595E-1,2.8209436E-1,-1.6975024E-1,2.6089436E-1,-6.422835E-1,4.5184666E-1,-3.219866E-1,-1.2041423E-1,2.5882706E-1,-6.4872103E-3,2.2845333E-2,3.6616614E-1,-1.1089426E-1,2.304555E-1,2.2377545E-2,-2.534749E-1,-2.2391817E-1,-5.841169E-2,2.099301E-1,-7.7790166E-3,-1.5232125E-1,9.891699E-2],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":8,"left_children":[1,3,5,-1,-1,7,9,11,13,15,17,19,21,-1,23,25,27,29,-1,31,33,35,-1,-1,-1,37,39,-1,-1,-1,41,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1],"loss_changes":[1.154967E1,2.8379822E-1,4.5093102E0,0E0,0E0,3.63162E0,3.5999844E0,2.5433502E0,1.9069128E0,1.7193341E0,2.5056772E0,7.947769E-1,2.241698E0,0E0,2.1339731E0,1.9763536E0,6.778717E-2,1.8657832E0,0E0,1.6166077E0,1.4063923E0,1.4292974E0,0E0,0E0,0E0,2.6872754E-2,3.7991023E-1,0E0,0E0,0E0,1.6636546E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,1,1,2,2,5,5,6,6,7,7,8,8,9,9,10,10,11,11,12,12,14,14,15,15,16,16,17,17,19,19,20,20,21,21,25,25,26,26,30,30],"right_children":[2,4,6,-1,-1,8,10,12,14,16,18,20,22,-1,24,26,28,30,-1,32,34,36,-1,-1,-1,38,40,-1,-1,-1,42,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1],"split_conditions":[3.725037E-1,4.192354E-1,9.0318775E-1,-2.9189646E-1,-6.3117266E-2,4.0580824E-1,9.083769E-1,3.690198E-1,5.269866E-1,9.6957684E-1,9.639875E-1,7.888401E-1,7.242376E-1,-2.5793415E-1,4.9794367E-1,6.9855297E-1,5.116137E-1,9.9801147E-1,-2.152957E-1,2.26669E-1,5.339799E-1,5.9747434E-1,2.8209436E-1,-1.6975024E-1,2.6089436E-1,5.202517E-1,6.0640275E-1,-3.219866E-1,-1.2041423E-1,2.5882706E-1,7.478338E-1,2.2845333E-2,3.6616614E-1,-1.1089426E-1,2.304555E-1,2.2377545E-2,-2.534749E-1,-2.2391817E-1,-5.841169E-2,2.099301E-1,-7.7790166E-3,-1.5232125E-1,9.891699E-2],"split_indices":[1,5,2,0,0,4,0,3,5,2,5,2,7,0,1,1,5,2,0,6,1,7,0,0,0,7,7,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[6.6777306E1,1.9746937E1,4.7030373E1,1.8593967E1,1.1529689E0,2.2019716E1,2.5010656E1,1.705101E1,4.9687057E0,1.2187013E1,1.2823644E1,9.332379E0,7.7186313E0,2.7440453E0,2.2246602E0,4.7496176E0,7.437395E0,1.0465252E1,2.3583913E0,6.861482E0,2.4708977E0,6.5381355E0,1.180496E0,1.0443912E0,1.1802691E0,2.6954713E0,2.054146E0,5.533247E0,1.9041485E0,2.5868278E0,7.878424E0,1.7193934E0,5.1420884E0,1.3594335E0,1.111464E0,5.110852E0,1.4272835E0,1.6451632E0,1.0503081E0,1.0464875E0,1.0076586E0,2.9580872E0,4.920337E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"43","size_leaf_vector":"1"}},{"base_weights":[-2.7842876E-1,-9.106354E-1,-4.40708E-2,-2.8555363E-1,-5.4535538E-2,2.1710758E-1,-2.865139E-1,3.283283E-2,8.643452E-1,-7.638563E-1,-4.9245775E-2,2.8699523E-1,-4.566208E-1,7.845389E-2,2.873459E-1,-1.34606585E-2,-2.6535356E-1,9.750168E-2,-2.1048746E-1,4.8694068E-1,-2.0935376E-1,-2.3904404E-1,-4.575202E-2,-2.7701673E-1,4.803577E-1,2.0644003E-1,-4.881331E-2,-1.4279501E-1,1.034508E-1,-1.7456913E-1,5.376755E-2,2.0446153E-1,-1.1311369E-1],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":9,"left_children":[1,3,5,-1,-1,7,9,11,13,15,17,19,21,-1,-1,-1,-1,23,-1,25,-1,-1,27,29,31,-1,-1,-1,-1,-1,-1,-1,-1],"loss_changes":[8.838219E0,3.1219864E-1,2.8969533E0,0E0,0E0,2.609886E0,2.640688E0,2.402298E0,2.3525476E-2,6.354637E-1,1.6829858E0,2.8270953E0,8.897232E-1,0E0,0E0,0E0,0E0,2.233113E0,0E0,1.56131E0,0E0,0E0,8.4677184E-1,1.2223526E0,1.5693258E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,1,1,2,2,5,5,6,6,7,7,8,8,9,9,10,10,11,11,12,12,17,17,19,19,22,22,23,23,24,24],"right_children":[2,4,6,-1,-1,8,10,12,14,16,18,20,22,-1,-1,-1,-1,24,-1,26,-1,-1,28,30,32,-1,-1,-1,-1,-1,-1,-1,-1],"split_conditions":[3.725037E-1,4.192354E-1,9.0318775E-1,-2.8555363E-1,-5.4535538E-2,6.027514E-1,6.375137E-1,4.0404058E-1,4.8566276E-1,3.040787E-1,9.639875E-1,4.0580824E-1,4.9421373E-1,7.845389E-2,2.873459E-1,-1.34606585E-2,-2.6535356E-1,4.286508E-1,-2.1048746E-1,5.9747434E-1,-2.0935376E-1,-2.3904404E-1,5.340339E-1,2.61971E-1,7.1414995E-1,2.0644003E-1,-4.881331E-2,-1.4279501E-1,1.034508E-1,-1.7456913E-1,5.376755E-2,2.0446153E-1,-1.1311369E-1],"split_indices":[1,5,2,0,0,1,1,3,7,4,5,4,1,0,0,0,0,5,0,7,0,0,7,6,7,0,0,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[5.8922993E1,1.5155008E1,4.3767982E1,1.4054402E1,1.100606E0,2.112019E1,2.2647795E1,1.7257532E1,3.862657E0,6.782732E0,1.5865063E1,1.1631299E1,5.626233E0,1.02835E0,2.8343072E0,1.1718061E0,5.6109257E0,1.3646039E1,2.2190232E0,1.0088637E1,1.542662E0,2.56397E0,3.0622633E0,7.037876E0,6.608163E0,7.6578255E0,2.4308116E0,1.3531455E0,1.7091178E0,4.053644E0,2.984232E0,5.5184336E0,1.0897293E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"33","size_leaf_vector":"1"}},{"base_weights":[-2.4714364E-1,-8.763082E-1,-5.062544E-2,-2.7842E-1,-4.710237E-2,7.543735E-1,-1.19552284E-1,7.574065E-4,3.1938758E-1,-4.371181E-2,-2.0373517E-1,2.483345E-2,-2.2258167E-1,-1.7171516E-1,2.755534E-1,-1.8788108E-1,2.3978075E-2,1.2945847E-1,-7.049823E-2],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":10,"left_children":[1,3,5,-1,-1,7,9,-1,-1,11,-1,13,-1,15,17,-1,-1,-1,-1],"loss_changes":[6.5946207E0,3.2779503E-1,2.4193032E0,0E0,0E0,8.014866E-1,1.6478374E0,0E0,0E0,1.7356374E0,0E0,1.7050481E0,0E0,2.2775037E0,1.3091717E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,1,1,2,2,5,5,6,6,9,9,11,11,13,13,14,14],"right_children":[2,4,6,-1,-1,8,10,-1,-1,12,-1,14,-1,16,18,-1,-1,-1,-1],"split_conditions":[3.725037E-1,4.192354E-1,2.263913E-1,-2.7842E-1,-4.710237E-2,5.0804096E-1,5.769318E-1,7.574065E-4,3.1938758E-1,7.059457E-1,-2.0373517E-1,3.3915552E-1,-2.2258167E-1,5.5088145E-1,5.9747434E-1,-1.8788108E-1,2.3978075E-2,1.2945847E-1,-7.049823E-2],"split_indices":[1,5,4,0,0,7,4,0,0,6,0,6,0,1,7,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[5.2695103E1,1.1718492E1,4.097661E1,1.0661118E1,1.0573736E0,2.4475095E0,3.85291E1,1.0094342E0,1.4380754E0,3.4879795E1,3.6493058E0,3.263995E1,2.2398448E0,1.8473223E1,1.4166727E1,6.0521507E0,1.2421071E1,1.0970085E1,3.1966424E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"19","size_leaf_vector":"1"}},{"base_weights":[-2.093679E-1,-8.383107E-1,-3.8133312E-2,-2.70729E-1,-4.1651864E-2,4.820996E-1,-1.2948623E-1,7.0432407E-1,-9.181805E-2,-2.6518196E-1,1.4481182E-1,-2.643368E-2,2.811459E-1,-9.373519E-2,-8.6552984E-1,4.504736E-1,-3.832398E-1,-4.9572918E-1,1.2109424E-1,-2.9974148E-1,-7.505368E-2,5.5877584E-1,-3.745565E-2,-2.1216816E-1,3.2922067E-2,5.3352807E-2,-2.00875E-1,-9.236953E-2,1.6578627E-1,1.9210365E-2,2.3894991E-1],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":11,"left_children":[1,3,5,-1,-1,7,9,11,-1,13,15,-1,-1,17,19,21,23,25,27,-1,-1,29,-1,-1,-1,-1,-1,-1,-1,-1,-1],"loss_changes":[5.3016424E0,3.478856E-1,1.9621809E0,0E0,0E0,1.2984482E0,1.3333719E0,1.0282876E0,0E0,2.3510141E0,2.135599E0,0E0,0E0,1.7442248E0,1.7161155E-1,5.7836604E-1,8.7606794E-1,9.3746173E-1,2.671785E0,0E0,0E0,7.7840185E-1,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,1,1,2,2,5,5,6,6,7,7,9,9,10,10,13,13,14,14,15,15,16,16,17,17,18,18,21,21],"right_children":[2,4,6,-1,-1,8,10,12,-1,14,16,-1,-1,18,20,22,24,26,28,-1,-1,30,-1,-1,-1,-1,-1,-1,-1,-1,-1],"split_conditions":[4.223724E-1,4.9794367E-1,2.8061464E-1,-2.70729E-1,-4.1651864E-2,6.339713E-1,9.083769E-1,1.6142982E-1,-9.181805E-2,9.6957684E-1,9.998211E-1,-2.643368E-2,2.811459E-1,3.855034E-1,5.116137E-1,7.451184E-1,4.6254385E-1,2.916402E-1,5.385626E-1,-2.9974148E-1,-7.505368E-2,9.7059834E-1,-3.745565E-2,-2.1216816E-1,3.2922067E-2,5.3352807E-2,-2.00875E-1,-9.236953E-2,1.6578627E-1,1.9210365E-2,2.3894991E-1],"split_indices":[0,1,4,0,0,0,0,6,0,2,2,0,0,5,5,5,4,5,7,0,0,0,0,0,0,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[4.8524326E1,9.550341E0,3.8973984E1,8.482595E0,1.0677446E0,5.182659E0,3.3791325E1,4.1257563E0,1.0569028E0,2.262964E1,1.1161687E1,1.0780743E0,3.047682E0,1.8501993E1,4.127646E0,7.162563E0,3.9991236E0,5.988542E0,1.2513452E1,2.8788798E0,1.2487662E0,6.0520587E0,1.1105044E0,2.1509314E0,1.8481921E0,1.223876E0,4.764666E0,6.4188175E0,6.0946345E0,2.3761904E0,3.675868E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"31","size_leaf_vector":"1"}},{"base_weights":[-1.7783941E-1,-2.6280946E-1,-4.522828E-2,2.8178048E-1,-1.8178296E-1,5.6707233E-1,-1.1160982E-1,-4.130132E-1,5.498848E-2,-5.473751E-2,7.762877E-1,2.4319409E-1,-1.7837542E-1,2.0960744E-1,-5.918109E-1,4.5296678E-1,-2.786943E-1,9.2566663E-1,2.2585742E-2,-7.6042816E-2,4.879086E-1,1.8017013E-1,-1.14291616E-1,-5.4439977E-2,-7.3371685E-1,1.799831E-1,2.4978676E-1,-7.349939E-1,2.9146744E-2,8.231162E-2,3.177183E-1,2.7771348E-2,1.9350614E-1,-1.1108587E-1,6.919676E-2,-1.5953464E-2,-2.6938593E-1,1.4410919E-1,-1.1478909E-1,-3.133915E-1,-1.5336013E-3,1.5610586E-1,-1.1922109E-1],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":12,"left_children":[1,-1,3,5,7,9,11,13,15,-1,17,19,-1,21,23,25,27,29,-1,-1,31,-1,-1,33,35,37,-1,39,41,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1],"loss_changes":[4.2416515E0,0E0,1.8335531E0,1.4268718E0,1.5975002E0,1.2717819E0,1.1629734E0,1.7571311E0,2.1307263E0,0E0,5.402286E-1,6.2634003E-1,0E0,1.1549796E0,8.4908533E-1,6.0306394E-1,1.2616687E0,1.1887264E-1,0E0,0E0,1.3361478E-1,0E0,0E0,4.0198335E-1,9.7412777E-1,1.11361E0,0E0,8.0717695E-1,1.5130165E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,2,2,3,3,4,4,5,5,6,6,7,7,8,8,10,10,11,11,13,13,14,14,15,15,16,16,17,17,20,20,23,23,24,24,25,25,27,27,28,28],"right_children":[2,-1,4,6,8,10,12,14,16,-1,18,20,-1,22,24,26,28,30,-1,-1,32,-1,-1,34,36,38,-1,40,42,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1],"split_conditions":[4.9454403E-1,-2.6280946E-1,3.271306E-1,8.480998E-1,6.174951E-1,4.0518132E-1,6.098213E-1,3.2276946E-1,9.9801147E-1,-5.473751E-2,4.0059653E-1,5.339799E-1,-1.7837542E-1,4.0283135E-1,3.6373273E-1,9.083769E-1,2.983073E-1,4.7196496E-1,2.2585742E-2,-7.6042816E-2,2.6576984E-1,1.8017013E-1,-1.14291616E-1,2.698444E-1,6.736616E-1,9.6957684E-1,2.4978676E-1,5.2813345E-1,5.227221E-1,8.231162E-2,3.177183E-1,2.7771348E-2,1.9350614E-1,-1.1108587E-1,6.919676E-2,-1.5953464E-2,-2.6938593E-1,1.4410919E-1,-1.1478909E-1,-3.133915E-1,-1.5336013E-3,1.5610586E-1,-1.1922109E-1],"split_indices":[2,0,4,2,1,1,7,3,2,0,3,1,0,4,4,0,3,7,0,0,6,0,0,6,2,2,0,7,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[4.524002E1,6.3262987E0,3.891372E1,1.1149768E1,2.7763952E1,6.206924E0,4.942844E0,1.3669757E1,1.4094195E1,1.3823689E0,4.824555E0,3.1357324E0,1.8071114E0,3.0113003E0,1.0658457E1,6.2647943E0,7.829401E0,3.7128747E0,1.1116803E0,1.0232174E0,2.112515E0,1.8016851E0,1.2096152E0,2.515682E0,8.142775E0,4.501879E0,1.7629153E0,2.5951483E0,5.2342525E0,1.150829E0,2.5620458E0,1.0527377E0,1.0597774E0,1.0517153E0,1.4639666E0,1.8404344E0,6.3023405E0,3.0302253E0,1.471654E0,1.5193415E0,1.075807E0,2.3305418E0,2.9037106E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"43","size_leaf_vector":"1"}},{"base_weights":[-1.4830647E-1,-2.5196058E-1,-3.5669938E-2,2.053479E-2,-2.097679E-1,9.6656E-2,-5.527384E-1,-4.8255462E-2,2.8858396E-1,-3.154245E-2,-2.0858009E-1,1.2536141E-1,-6.356933E-1,6.5051365E-1,-5.9960745E-2,-4.12483E-2,1.4842081E-1,-2.3686485E-1,-2.8005196E-2,2.5960526E-1,2.0501005E-2,-1.353837E-1,8.693484E-2],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":13,"left_children":[1,-1,3,5,-1,7,9,11,13,-1,-1,15,17,19,21,-1,-1,-1,-1,-1,-1,-1,-1],"loss_changes":[3.273443E0,0E0,1.4134029E0,1.604987E0,0E0,9.105614E-1,2.0227516E-1,2.0405288E0,1.8066723E0,0E0,0E0,1.5899707E0,3.0601537E-1,8.112104E-1,1.2330357E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,2,2,3,3,5,5,6,6,7,7,8,8,11,11,12,12,13,13,14,14],"right_children":[2,-1,4,6,-1,8,10,12,14,-1,-1,16,18,20,22,-1,-1,-1,-1,-1,-1,-1,-1],"split_conditions":[4.9454403E-1,-2.5196058E-1,7.059457E-1,6.756343E-1,-2.097679E-1,6.375137E-1,8.075946E-1,4.0580824E-1,9.9801147E-1,-3.154245E-2,-2.0858009E-1,3.4447065E-1,4.5561138E-1,6.274843E-1,2.9423106E-1,-4.12483E-2,1.4842081E-1,-2.3686485E-1,-2.8005196E-2,2.5960526E-1,2.0501005E-2,-1.353837E-1,8.693484E-2],"split_indices":[2,0,6,3,0,1,2,4,2,0,0,6,6,7,3,0,0,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[4.1408295E1,4.895376E0,3.6512917E1,3.4555077E1,1.95784E0,3.1238516E1,3.3165624E0,1.8225945E1,1.301257E1,1.2207144E0,2.0958478E0,1.4675274E1,3.550671E0,5.9586854E0,7.0538845E0,8.940656E0,5.734618E0,2.4109116E0,1.1397595E0,3.9972134E0,1.9614722E0,3.1920621E0,3.8618224E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"23","size_leaf_vector":"1"}},{"base_weights":[-1.3287574E-1,-2.4152052E-1,-3.5345364E-2,2.1941875E-1,-1.4888161E-1,3.5157079E-1,-1.5673104E-1,-3.3055854E-1,4.774324E-2,6.211255E-1,-5.5983867E-2,-3.9998718E-2,-6.41983E-1,2.3012267E-1,-3.9653742E-1,1.9118984E-1,3.0888987E-1,1.1454712E-1,-3.4679532E-1,2.0026557E-1,-4.3992868E-1,-2.2311987E-1,-1.6692128E-2,-8.628977E-2,3.5845506E-1,-2.007687E-1,4.294126E-2,-1.6532452E-1,1.7964464E-1,-2.0908467E-1,4.0544268E-2,-5.4115057E-2,1.2957151E-1,-2.1301626E-1,2.292892E-2,1.776064E-1,-1.887929E-2],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":14,"left_children":[1,-1,3,5,7,9,-1,11,13,15,17,19,21,23,25,27,-1,-1,29,31,33,-1,-1,-1,35,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1],"loss_changes":[2.5516896E0,0E0,1.0570492E0,1.2936268E0,9.121751E-1,1.1561596E0,0E0,1.1659223E0,1.1385151E0,8.967445E-1,7.665823E-1,8.451871E-1,3.3386302E-1,7.539958E-1,7.0340085E-1,1.6194813E0,0E0,0E0,6.8768823E-1,5.7249033E-1,5.103462E-1,0E0,0E0,0E0,8.5370326E-1,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,2,2,3,3,4,4,5,5,7,7,8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,18,18,19,19,20,20,24,24],"right_children":[2,-1,4,6,8,10,-1,12,14,16,18,20,22,24,26,28,-1,-1,30,32,34,-1,-1,-1,36,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1],"split_conditions":[3.2517466E-1,-2.4152052E-1,3.271306E-1,4.3760327E-1,7.950383E-1,8.83008E-1,-1.5673104E-1,4.0173647E-1,9.998211E-1,4.9794367E-1,2.0173919E-1,7.888401E-1,6.422845E-1,3.7363765E-1,9.9996924E-1,2.00198E-1,3.0888987E-1,1.1454712E-1,4.1397148E-1,3.5601464E-1,6.339713E-1,-2.2311987E-1,-1.6692128E-2,-8.628977E-2,4.4652742E-1,-2.007687E-1,4.294126E-2,-1.6532452E-1,1.7964464E-1,-2.0908467E-1,4.0544268E-2,-5.4115057E-2,1.2957151E-1,-2.1301626E-1,2.292892E-2,1.776064E-1,-1.887929E-2],"split_indices":[1,0,4,3,0,2,0,4,2,1,3,2,5,4,2,6,0,0,5,4,0,0,0,0,4,0,0,0,0,0,0,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[3.8355953E1,3.9408107E0,3.4415142E1,1.0321694E1,2.4093449E1,9.207576E0,1.1141188E0,1.216869E1,1.1924759E1,5.226659E0,3.9809172E0,6.879001E0,5.289689E0,8.795993E0,3.1287658E0,3.2617903E0,1.9648683E0,1.4639804E0,2.5169368E0,4.6091995E0,2.2698014E0,4.2787557E0,1.0109335E0,1.500587E0,7.295406E0,1.9190385E0,1.2097274E0,1.0315124E0,2.230278E0,1.1993966E0,1.31754E0,1.8274527E0,2.7817469E0,1.2439357E0,1.0258658E0,4.4332185E0,2.8621874E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"37","size_leaf_vector":"1"}},{"base_weights":[-1.1996663E-1,-6.9649214E-1,-2.559628E-2,-6.197323E-2,-2.2985016E-1,2.7993634E-2,-1.9570154E-1,3.509341E-1,-4.341349E-2,-8.727401E-2,5.641472E-1,-4.3269545E-1,4.130297E-2,2.2838162E-1,2.7711391E-3,-9.036216E-2,-2.0264488E-1,-1.4238606E-1,3.0236763E-1,2.2290085E-2,-6.2934965E-2,1.0082203E-2,-1.9678666E-1,1.3631588E-1,-4.2804584E-2],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":15,"left_children":[1,3,5,-1,-1,7,-1,9,11,-1,13,15,17,-1,-1,19,-1,21,23,-1,-1,-1,-1,-1,-1],"loss_changes":[1.9880676E0,1.7448902E-2,1.1172471E0,0E0,0E0,7.324543E-1,0E0,9.499024E-1,8.8718843E-1,0E0,5.016568E-1,3.3777678E-1,1.1249899E0,0E0,0E0,7.743976E-2,0E0,1.2804952E0,7.169263E-1,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,1,1,2,2,5,5,7,7,8,8,10,10,11,11,12,12,15,15,17,17,18,18],"right_children":[2,4,6,-1,-1,8,-1,10,12,-1,14,16,18,-1,-1,20,-1,22,24,-1,-1,-1,-1,-1,-1],"split_conditions":[4.223724E-1,2.0835237E-1,7.059457E-1,-6.197323E-2,-2.2985016E-1,2.9255116E-1,-1.9570154E-1,1.651226E-1,4.6713987E-1,-8.727401E-2,4.286508E-1,3.690198E-1,3.4447065E-1,2.2838162E-1,2.7711391E-3,4.0930006E-1,-2.0264488E-1,3.1766286E-1,9.9787104E-1,2.2290085E-2,-6.2934965E-2,1.0082203E-2,-1.9678666E-1,1.3631588E-1,-4.2804584E-2],"split_indices":[0,3,6,0,0,4,0,6,1,0,5,3,6,0,0,1,0,6,2,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[3.5868385E1,4.1478724E0,3.1720512E1,1.010125E0,3.1377475E0,3.0101963E1,1.6185492E0,4.7419276E0,2.5360035E1,1.0915495E0,3.650378E0,3.7983952E0,2.156164E1,2.4191291E0,1.231249E0,2.1454585E0,1.6529368E0,1.2923244E1,8.638395E0,1.0607222E0,1.0847362E0,1.0320952E1,2.6022916E0,6.4233494E0,2.2150464E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"25","size_leaf_vector":"1"}},{"base_weights":[-1.1096737E-1,-2.1989855E-1,-3.7910055E-2,6.5409127E-3,-1.8183702E-1,6.115311E-2,-4.41912E-1,-1.0834896E-1,1.9062546E-1,-1.9465344E-1,-1.1315731E-2,2.4111639E-1,-2.7889177E-1,2.7815017E-1,3.133203E-2,-3.773742E-2,1.7511788E-1,-1.7534274E-1,8.346797E-3,-7.229374E-2,9.022839E-2],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":16,"left_children":[1,-1,3,5,-1,7,9,11,13,-1,-1,15,17,-1,19,-1,-1,-1,-1,-1,-1],"loss_changes":[1.5743102E0,0E0,8.2745636E-1,7.9181665E-1,0E0,6.508485E-1,2.6923764E-1,8.463903E-1,1.9235806E0,0E0,0E0,6.7981184E-1,8.8317496E-1,0E0,1.1523441E0,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,2,2,3,3,5,5,6,6,7,7,8,8,11,11,12,12,14,14],"right_children":[2,-1,4,6,-1,8,10,12,14,-1,-1,16,18,-1,20,-1,-1,-1,-1,-1,-1],"split_conditions":[4.9454403E-1,-2.1989855E-1,7.059457E-1,6.756343E-1,-1.8183702E-1,5.7111377E-1,5.515903E-1,6.904237E-1,6.0052043E-1,-1.9465344E-1,-1.1315731E-2,4.708969E-1,3.4447065E-1,2.7815017E-1,4.001638E-1,-3.773742E-2,1.7511788E-1,-1.7534274E-1,8.346797E-3,-7.229374E-2,9.022839E-2],"split_indices":[2,0,6,3,0,1,7,2,0,0,0,1,6,0,4,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[3.425459E1,2.6509135E0,3.1603676E1,3.0227488E1,1.3761884E0,2.7715904E1,2.511583E0,1.2073162E1,1.5642742E1,1.260808E0,1.2507751E0,3.82382E0,8.249343E0,1.9243631E0,1.3718379E1,2.152021E0,1.6717992E0,3.6786494E0,4.570693E0,6.875217E0,6.8431625E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"21","size_leaf_vector":"1"}},{"base_weights":[-9.4168775E-2,-2.0747526E-1,-3.0759266E-2,1.2561601E-1,-1.9713527E-1,-1.0960484E-1,2.0839782E-1,-5.8910674E-1,3.7529327E-2,3.0479756E-1,-9.27646E-2,-9.061756E-3,-2.2222444E-1,-1.2831625E-1,1.525812E-1,4.5733222E-1,-1.8826725E-1,5.086296E-1,-9.278719E-2,-3.3738572E-2,1.5926407E-1,1.0264668E-1,-2.0255333E-1,1.4814652E-2,2.3704539E-1,-1.1190996E-1,7.639631E-2],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":17,"left_children":[1,-1,3,5,7,-1,9,11,13,15,-1,-1,-1,-1,17,19,21,23,25,-1,-1,-1,-1,-1,-1,-1,-1],"loss_changes":[1.2571039E0,0E0,8.457288E-1,7.620741E-1,1.4641769E0,0E0,8.4664667E-1,4.8028708E-1,6.405162E-1,1.0799962E0,0E0,0E0,0E0,0E0,8.6431736E-1,4.9488354E-1,1.2203588E0,4.7428453E-1,7.164974E-1,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,2,2,3,3,4,4,6,6,7,7,8,8,9,9,14,14,15,15,16,16,17,17,18,18],"right_children":[2,-1,4,6,8,-1,10,12,14,16,-1,-1,-1,-1,18,20,22,24,26,-1,-1,-1,-1,-1,-1,-1,-1],"split_conditions":[3.2517466E-1,-2.0747526E-1,3.830248E-1,4.4817156E-1,4.2627814E-1,-1.0960484E-1,4.3760327E-1,2.6921508E-1,3.92703E-1,9.775344E-1,-9.27646E-2,-9.061756E-3,-2.2222444E-1,-1.2831625E-1,5.0019234E-1,1.5111692E-1,2.8642765E-1,9.575703E-1,5.123491E-1,-3.3738572E-2,1.5926407E-1,1.0264668E-1,-2.0255333E-1,1.4814652E-2,2.3704539E-1,-1.1190996E-1,7.639631E-2],"split_indices":[1,0,4,7,5,0,3,3,4,2,0,0,0,0,5,6,6,0,3,0,0,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[3.2746445E1,2.1915963E0,3.0554848E1,1.5877097E1,1.4677752E1,1.7982955E0,1.4078801E1,4.930942E0,9.746809E0,1.22150345E1,1.8637674E0,1.308271E0,3.6226711E0,1.3936186E0,8.35319E0,9.384357E0,2.8306766E0,2.9702322E0,5.382958E0,1.0123492E0,8.372008E0,1.4970915E0,1.3335851E0,1.5755093E0,1.394723E0,2.9388375E0,2.4441206E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"27","size_leaf_vector":"1"}},{"base_weights":[-8.856826E-2,-3.982697E-1,8.095417E-3,-1.9723703E-1,5.5801455E-2,2.4818717E-1,-1.3579993E-1,1.32517E-1,-9.964953E-2,4.777257E-1,-1.0905718E-1,-4.3520287E-1,5.3853914E-2,-2.1715648E-2,2.6523095E-1,1.05466016E-1,-4.2132786E-1,-6.00426E-1,6.207113E-2,4.0663484E-1,-1.5166068E-1,1.4162599E-1,-1.495677E-1,-2.1626417E-1,1.2133156E-2,-2.2691467E-1,-4.0340994E-2,-6.071173E-3,1.66879E-1,-4.9497128E-1,1.2939325E-1,-1.8018128E-2,-1.9483034E-1,1.9731674E-1,-5.697601E-2],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":18,"left_children":[1,3,5,-1,7,9,11,-1,-1,13,15,17,19,21,-1,-1,23,25,-1,27,29,-1,-1,-1,-1,-1,-1,-1,-1,31,33,-1,-1,-1,-1],"loss_changes":[9.638505E-1,9.373536E-1,9.124509E-1,0E0,7.011135E-1,8.6271167E-1,9.6320194E-1,0E0,0E0,1.2784224E0,8.214274E-1,8.168106E-1,8.6077994E-1,1.1205833E0,0E0,0E0,4.7796988E-1,2.942207E-1,0E0,2.877568E-1,8.063345E-1,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0,2.0755035E-1,9.722743E-1,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,1,1,2,2,4,4,5,5,6,6,9,9,10,10,11,11,12,12,13,13,16,16,17,17,19,19,20,20,29,29,30,30],"right_children":[2,4,6,-1,8,10,12,-1,-1,14,16,18,20,22,-1,-1,24,26,-1,28,30,-1,-1,-1,-1,-1,-1,-1,-1,32,34,-1,-1,-1,-1],"split_conditions":[4.6713987E-1,5.4014593E-1,8.480998E-1,-1.9723703E-1,7.3732144E-1,6.1228585E-1,6.731154E-1,1.32517E-1,-9.964953E-2,5.435263E-1,5.391588E-1,9.488899E-1,9.9801147E-1,6.456479E-1,2.6523095E-1,1.05466016E-1,7.202478E-1,3.4447065E-1,6.207113E-2,3.9393115E-1,2.983073E-1,1.4162599E-1,-1.495677E-1,-2.1626417E-1,1.2133156E-2,-2.2691467E-1,-4.0340994E-2,-6.071173E-3,1.66879E-1,3.271306E-1,3.2637256E-1,-1.8018128E-2,-1.9483034E-1,1.9731674E-1,-5.697601E-2],"split_indices":[1,0,2,0,2,0,1,0,0,1,1,0,2,2,0,0,2,6,0,5,3,0,0,0,0,0,0,0,0,4,3,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[3.1088396E1,6.652915E0,2.443548E1,3.9501984E0,2.7027168E0,8.885351E0,1.5550129E1,1.2854655E0,1.4172512E0,5.2042418E0,3.6811094E0,5.5281777E0,1.0021952E1,2.7594352E0,2.4448066E0,1.4364688E0,2.2446406E0,4.4484425E0,1.0797352E0,3.3289485E0,6.6930027E0,1.3605119E0,1.3989232E0,1.0211188E0,1.2235218E0,2.8659227E0,1.58252E0,1.0884581E0,2.2404902E0,2.6701987E0,4.0228043E0,1.0637913E0,1.6064075E0,1.1161847E0,2.9066193E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"35","size_leaf_vector":"1"}},{"base_weights":[-7.2814785E-2,-1.6474418E-1,-1.566705E-2,1.2236403E-1,-2.0580128E-1,2.8556347E-1,-1.16573215E-1,-1.9628139E-1,-7.4677385E-2,-6.41229E-2,4.222566E-1,-4.6426573E-1,1.0075119E-1,1.7539687E-1,-2.7092683E-1,1.2117855E-1,-1.1934184E-1,5.205817E-1,-1.0763676E-2,-1.765292E-1,-3.226539E-2,-1.4619759E-1,3.5336623E-1,-4.6036962E-1,9.177708E-2,3.2903347E-2,2.0990245E-1,1.4489982E-2,1.7035821E-1,-1.9934247E-1,-3.1898428E-2,1.4369264E-1,-1.0104341E-1],"categories":[],"categories_nodes":[],"categories_segments":[],"categories_sizes":[],"default_left":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"id":19,"left_children":[1,-1,3,5,7,9,11,-1,13,15,17,19,21,-1,23,-1,-1,25,-1,-1,-1,-1,27,29,31,-1,-1,-1,-1,-1,-1,-1,-1],"loss_changes":[8.1279093E-1,0E0,7.592829E-1,6.8970156E-1,6.6492647E-1,5.190231E-1,6.1663926E-1,0E0,1.5197434E0,7.4801934E-1,3.5793126E-1,7.309091E-2,9.8370767E-1,0E0,6.462979E-1,0E0,0E0,3.8429987E-1,0E0,0E0,0E0,0E0,2.6054054E-1,3.5865712E-1,8.00228E-1,0E0,0E0,0E0,0E0,0E0,0E0,0E0,0E0],"parents":[2147483647,0,0,2,2,3,3,4,4,5,5,6,6,8,8,9,9,10,10,11,11,12,12,14,14,17,17,22,22,23,23,24,24],"right_children":[2,-1,4,6,8,10,12,-1,14,16,18,20,22,-1,24,-1,-1,26,-1,-1,-1,-1,28,30,32,-1,-1,-1,-1,-1,-1,-1,-1],"split_conditions":[4.223724E-1,-1.6474418E-1,3.736203E-1,9.775344E-1,4.779936E-1,5.7002103E-1,9.6303654E-1,-1.9628139E-1,5.5711186E-1,3.062877E-1,4.0283135E-1,6.885401E-1,2.3203091E-1,1.7539687E-1,8.9469093E-1,1.2117855E-1,-1.1934184E-1,2.9543516E-1,-1.0763676E-2,-1.765292E-1,-3.226539E-2,-1.4619759E-1,5.2813345E-1,3.8357162E-1,5.385626E-1,3.2903347E-2,2.0990245E-1,1.4489982E-2,1.7035821E-1,-1.9934247E-1,-3.1898428E-2,1.4369264E-1,-1.0104341E-1],"split_indices":[0,0,3,2,1,0,0,0,0,3,4,1,6,0,0,0,0,6,0,0,0,0,7,6,7,0,0,0,0,0,0,0,0],"split_type":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sum_hessian":[2.9174671E1,2.2030191E0,2.6971653E1,1.5833498E1,1.1138155E1,9.291831E0,6.5416665E0,1.6172203E0,9.520934E0,2.760601E0,6.53123E0,2.0790925E0,4.462574E0,1.7298981E0,7.791036E0,1.06135E0,1.6992509E0,5.264959E0,1.2662716E0,1.0604035E0,1.018689E0,1.0617486E0,3.4008255E0,4.9410305E0,2.8500056E0,2.0876184E0,3.1773403E0,1.9097912E0,1.4910342E0,2.5779893E0,2.3630414E0,1.4355372E0,1.4144684E0],"tree_param":{"num_deleted":"0","num_feature":"8","num_nodes":"33","size_leaf_vector":"1"}}]},"name":"gbtree"},"learner_model_param":{"base_score":"1.3901752E-1","boost_from_average":"1","num_class":"0","num_feature":"8","num_target":"1"},"objective":{"name":"binary:logistic","reg_loss_param":{"scale_pos_weight":"1"}}},"version":[2,0,3]}
guitargeek commented 7 months ago

Thanks for sharing! It has indeed something do do with the right value for the base_score, which is not dumped to the txt file that FastForest takes as an input.

I have added a section in the README that explains how to work around this problem: https://github.com/guitargeek/XGBoost-FastForest?tab=readme-ov-file#models-with-non-zero-base-response

I hope this helps you! If you have further problems, feel free to re-open this issues or open a new one.

inkzk commented 7 months ago

@guitargeek nice! NICE!

Small bug, base_score should acutally be base_response at third row in the README?

float base_score = /* the output of get_basescore from Python */;
float base_response = std::log(base_score / (1.0 - base_score));
float score = 1./(1. + std::exp(-fastForest(input.data(), base_score)));
guitargeek commented 7 months ago

Thanks! I have updated that