[Model] XGBoost - Githubissues

13aek commented 1 month ago

import xgboost as xgb
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error
# # 학습에 사용할 특징 열
# feature_columns = [col for col in data.columns if col not in ['deposit', '_type', 'index']]
# 학습 및 테스트 데이터 준비
X_train = train_data.drop(columns=['deposit'])
y_train = train_data['deposit']
X_test = test_data.copy()
# 학습 데이터를 검증용으로 분할
X_train_split, X_val_split, y_train_split, y_val_split = train_test_split(X_train, y_train, test_size=0.2, random_state=42)
# XGBoost를 위한 DMatrix 생성
dtrain = xgb.DMatrix(X_train_split, label=y_train_split)
dval = xgb.DMatrix(X_val_split, label=y_val_split)
dtest = xgb.DMatrix(X_test)
# XGBoost 파라미터 설정
params = {
    'objective': 'reg:absoluteerror',
    'eval_metric': 'mae',
    'max_depth': 6,
    'eta': 0.05,
    'subsample': 0.8,
    'colsample_bytree': 0.8,
    'seed': 42
}
# 모델 학습
evals = [(dtrain, 'train'), (dval, 'eval')]
model = xgb.train(params, dtrain, num_boost_round=500, early_stopping_rounds=50, evals=evals, verbose_eval=10)
# 테스트 세트에 대한 예측
y_pred = model.predict(dtest)

13aek commented 1 month ago

XGBoost 1

data set: v4

columns:

   'area_m2', 'contract_year_month', 'contract_day',
   'contract_type', 'floor', 'built_year', 'latitude', 'longitude', 'age', 'deposit', 
   'apt_idx', 
   'area',
   'grid_id', 
   'nearest_subway_distance', 'nearest_subway_idx',
   'num_subway_within_1km', 'category_interchange_within_1km',
   'num_subway_within_500m', 'category_interchange_within_500m',
   'nearest_elementary_school_distance', 
   'nearest_elementary_school_within_1km',
   'has_elementary_school_within_1km', 'nearest_middle_school_distance',
   'nearest_middle_school_within_1km', 'has_middle_school_within_1km',
   'nearest_high_school_distance', 'nearest_high_school_within_1km',
   'has_high_school_within_1km', 'nearest_park_distance',
   'nearest_park_within_500.0m', 'has_park_within_500.0m',
   'apt_deposit_rank', 'apt_area_deposit_rank', 
   'recent_deposit',

deposit mean: 39049.292969
local mae: 3909.01869
ai stage mae: 3723.2652
Feature Importance

13aek commented 1 month ago

XGBoost 2

data set: v4
change: + 'arima_deposit_index' columns

columns:

   'area_m2', 'contract_year_month', 'contract_day',
   'contract_type', 'floor', 'built_year', 'latitude', 'longitude', 'age', 'deposit', 
   'apt_idx', 
   'area',
   'grid_id', 
   'nearest_subway_distance', 'nearest_subway_idx',
   'num_subway_within_1km', 'category_interchange_within_1km',
   'num_subway_within_500m', 'category_interchange_within_500m',
   'nearest_elementary_school_distance', 'nearest_elementary_school_within_1km',
   'has_elementary_school_within_1km', 'nearest_middle_school_distance',
   'nearest_middle_school_within_1km', 'has_middle_school_within_1km',
   'nearest_high_school_distance', 'nearest_high_school_within_1km',
   'has_high_school_within_1km', 'nearest_park_distance',
   'nearest_park_within_500.0m', 'has_park_within_500.0m',
   'apt_deposit_rank', 'apt_area_deposit_rank', 
   'recent_deposit',
   'arima_deposit_index'

deposit mean: 39061.113281
local mae: 3905.36761
ai stage mae:
Feature Importance

13aek commented 1 month ago

XGBoost 3

data set: v4
change:
- 'contract_type' 원핫인코딩 (`contract_0', 'contract_1') 추가

columns:

   'area_m2', 'contract_year_month', 'contract_day',
   'floor', 'built_year', 'latitude', 'longitude', 'age',
   'contract_0', 'contract_1',
   'deposit', 
   'apt_idx', 
   'area',
   'grid_id', 
   'nearest_subway_distance', 'nearest_subway_idx',
   'num_subway_within_1km', 'category_interchange_within_1km',
   'num_subway_within_500m', 'category_interchange_within_500m',
   'nearest_elementary_school_distance', 'nearest_elementary_school_within_1km',
   'has_elementary_school_within_1km', 'nearest_middle_school_distance',
   'nearest_middle_school_within_1km', 'has_middle_school_within_1km',
   'nearest_high_school_distance', 'nearest_high_school_within_1km',
   'has_high_school_within_1km', 'nearest_park_distance',
   'nearest_park_within_500.0m', 'has_park_within_500.0m',
   'apt_deposit_rank', 'apt_area_deposit_rank', 
   'recent_deposit',

deposit mean: 39069.425781
local mae: 3901.35951
ai stage mae:
Feature Importance

13aek commented 1 month ago

XGBoost 4

data set: v4
change:
- 'contract_type' 원핫인코딩 (`contract_0', 'contract_1') 추가
- 'arima_deposit_index' 추가

columns:

   'area_m2', 'contract_year_month', 'contract_day',
   'floor', 'built_year', 'latitude', 'longitude', 'age',
   'contract_0', 'contract_1',
   'deposit', 
   'apt_idx', 
   'area',
   'grid_id', 
   'nearest_subway_distance', 'nearest_subway_idx',
   'num_subway_within_1km', 'category_interchange_within_1km',
   'num_subway_within_500m', 'category_interchange_within_500m',
   'nearest_elementary_school_distance', 'nearest_elementary_school_within_1km',
   'has_elementary_school_within_1km', 'nearest_middle_school_distance',
   'nearest_middle_school_within_1km', 'has_middle_school_within_1km',
   'nearest_high_school_distance', 'nearest_high_school_within_1km',
   'has_high_school_within_1km', 'nearest_park_distance',
   'nearest_park_within_500.0m', 'has_park_within_500.0m',
   'apt_deposit_rank', 'apt_area_deposit_rank', 
   'recent_deposit',
   'arima_deposit_index'

deposit mean: 39116.851562
local mae: 3894.83773
ai stage mae:
Feature Importance

13aek commented 1 month ago

XGBoost 5

data set: v4
change:
- 'contract_type' 원핫인코딩 (`contract_0', 'contract_1') 추가
- 'arima_deposit_index' 추가
- 'grid_id' 삭제
- 'grid_area_deposit' 추가

columns:

   'area_m2', 'contract_year_month', 'contract_day',
   'floor', 'built_year', 'latitude', 'longitude', 'age',
   'contract_0', 'contract_1',
   'deposit', 
   'apt_idx', 
   'area', 
   'nearest_subway_distance', 'nearest_subway_idx',
   'num_subway_within_1km', 'category_interchange_within_1km',
   'num_subway_within_500m', 'category_interchange_within_500m',
   'nearest_elementary_school_distance', 'nearest_elementary_school_within_1km',
   'has_elementary_school_within_1km', 'nearest_middle_school_distance',
   'nearest_middle_school_within_1km', 'has_middle_school_within_1km',
   'nearest_high_school_distance', 'nearest_high_school_within_1km',
   'has_high_school_within_1km', 'nearest_park_distance',
   'nearest_park_within_500.0m', 'has_park_within_500.0m',
   'apt_deposit_rank', 'apt_area_deposit_rank', 
   'recent_deposit',
   'arima_deposit_index',
   'grid_area_deposit',

deposit mean: 39094.605469
local mae: 3894.30676
ai stage mae:
Feature Importance

13aek commented 1 month ago

XGBoost 6

data set: v4
change:
- 'contract_type' 원핫인코딩 (`contract_0', 'contract_1') 추가
- 'arima_deposit_index' 추가
- 'grid_id' 삭제
- 'grid_area_deposit' 추가
- 'grid' 추가 (grid_x + grid_y)

columns:

   'area_m2', 'contract_year_month', 'contract_day',
   'floor', 'built_year', 'latitude', 'longitude', 'age',
   'contract_0', 'contract_1',
   'deposit', 
   'apt_idx', 
   'area', 
   'nearest_subway_distance', 'nearest_subway_idx',
   'num_subway_within_1km', 'category_interchange_within_1km',
   'num_subway_within_500m', 'category_interchange_within_500m',
   'nearest_elementary_school_distance', 'nearest_elementary_school_within_1km',
   'has_elementary_school_within_1km', 'nearest_middle_school_distance',
   'nearest_middle_school_within_1km', 'has_middle_school_within_1km',
   'nearest_high_school_distance', 'nearest_high_school_within_1km',
   'has_high_school_within_1km', 'nearest_park_distance',
   'nearest_park_within_500.0m', 'has_park_within_500.0m',
   'apt_deposit_rank', 'apt_area_deposit_rank', 
   'recent_deposit',
   'arima_deposit_index',
   'grid_area_deposit',
   'grid',

deposit mean: 39099.472656
local mae: 3892.63524
ai stage mae:
Feature Importance

13aek commented 1 month ago

XGBoost 6

data set: v4
change:
- 'contract_type' 원핫인코딩 (`contract_0', 'contract_1') 추가
- 'arima_deposit_index' 추가
- 'grid_id' 삭제
- 'grid_area_deposit' 추가
- 'grid' 추가 (grid_x + grid_y)

columns:

   'area_m2', 'contract_year_month', 'contract_day',
   'floor', 'built_year', 'latitude', 'longitude', 'age',
   'contract_0', 'contract_1',
   'deposit', 
   'apt_idx', 
   'area', 
   'nearest_subway_distance', 'nearest_subway_idx',
   'num_subway_within_1km', 'category_interchange_within_1km',
   'num_subway_within_500m', 'category_interchange_within_500m',
   'nearest_elementary_school_distance', 'nearest_elementary_school_within_1km',
   'has_elementary_school_within_1km', 'nearest_middle_school_distance',
   'nearest_middle_school_within_1km', 'has_middle_school_within_1km',
   'nearest_high_school_distance', 'nearest_high_school_within_1km',
   'has_high_school_within_1km', 'nearest_park_distance',
   'nearest_park_within_500.0m', 'has_park_within_500.0m',
   'apt_deposit_rank', 'apt_area_deposit_rank', 
   'recent_deposit',
   'arima_deposit_index',
   'grid_area_deposit',
   'grid',

deposit mean: 39099.472656
local mae: 3892.63524
ai stage mae: 3736.9398
Feature Importance

13aek commented 1 month ago

XGBoost 7

data set: v4
change:
- 'contract_type' 원핫인코딩 (`contract_0', 'contract_1') 추가
- 'deposit_xgb' 추가 (xgboost 2번 모델링)

columns:

   'area_m2', 'contract_year_month', 'contract_day',
   'floor', 'built_year', 'latitude', 'longitude', 'age',
   'contract_0', 'contract_1',
   'deposit', 
   'apt_idx', 
   'area', 
   'nearest_subway_distance', 'nearest_subway_idx',
   'num_subway_within_1km', 'category_interchange_within_1km',
   'num_subway_within_500m', 'category_interchange_within_500m',
   'nearest_elementary_school_distance', 'nearest_elementary_school_within_1km',
   'has_elementary_school_within_1km', 'nearest_middle_school_distance',
   'nearest_middle_school_within_1km', 'has_middle_school_within_1km',
   'nearest_high_school_distance', 'nearest_high_school_within_1km',
   'has_high_school_within_1km', 'nearest_park_distance',
   'nearest_park_within_500.0m', 'has_park_within_500.0m',
   'apt_deposit_rank', 'apt_area_deposit_rank', 
   'recent_deposit',
   'deposit_xgb'

deposit mean: 39101.99
local mae: 3870.47160
ai stage mae: 3680.0482
Feature Importance

13aek commented 1 month ago

XGBoost 8

data set: v4
change:
- 'contract_type' 원핫인코딩 (`contract_0', 'contract_1') 추가
- 'contract_date_numeric' 추가 (contract_year_month + contract_day)

columns:

   'area_m2', 
   'floor', 'built_year', 'latitude', 'longitude', 'age',
   'contract_0', 'contract_1',
   'deposit', 
   'apt_idx', 
   'area', 
   'nearest_subway_distance', 'nearest_subway_idx',
   'num_subway_within_1km', 'category_interchange_within_1km',
   'num_subway_within_500m', 'category_interchange_within_500m',
   'nearest_elementary_school_distance', 'nearest_elementary_school_within_1km',
   'has_elementary_school_within_1km', 'nearest_middle_school_distance',
   'nearest_middle_school_within_1km', 'has_middle_school_within_1km',
   'nearest_high_school_distance', 'nearest_high_school_within_1km',
   'has_high_school_within_1km', 'nearest_park_distance',
   'nearest_park_within_500.0m', 'has_park_within_500.0m',
   'apt_deposit_rank', 'apt_area_deposit_rank', 
   'recent_deposit',
   'contract_date_numeric'

deposit mean: 39047.761719
local mae: 3897.71814
ai stage mae:
Feature Importance

13aek commented 1 month ago

XGBoost 9

data set: v4
change:
- 'contract_type' 원핫인코딩 (`contract_0', 'contract_1') 추가
- 'contract_date_numeric' 추가 (contract_year_month + contract_day)
- 'apt_floor_rank' 추가
- 불필요하다 보여지는 컬럼들 삭제
- 'interchange_2' 환승역이 주변에 있는지에 대한 컬럼 추가

columns:

'area_m2', 'contract_date_numeric',
'floor', 'built_year', 'latitude', 'longitude', 'age',
'contract_0', 'contract_1',
'deposit',
'apt_idx',
'area',
'grid_id',
'nearest_subway_distance',
'nearest_subway_idx',
'nearest_elementary_school_distance',
'nearest_middle_school_distance',
'nearest_high_school_distance',
'nearest_park_distance',
'apt_deposit_rank', 'apt_area_deposit_rank',
'recent_deposit',
'interchange_2',
'apt_floor_deposit_rank',

deposit mean: 38953.000000
local mae: 3799.81658
ai stage mae: 3784.0241
Feature Importance

13aek commented 1 month ago

241021 기준점

XGBoost 10

data set: v4
change:
- 'contract_type' 원핫인코딩 (`contract_0', 'contract_1') 추가
- 'contract_date_numeric' 추가 (contract_year_month + contract_day)

columns:

'area_m2', 'contract_date_numeric',
'floor', 'built_year', 'latitude', 'longitude', 'age',
'contract_0', 'contract_1',
'deposit',
'apt_idx',
'area',
'grid_id',
'nearest_subway_distance',
'nearest_subway_idx',
'nearest_elementary_school_distance',
'nearest_middle_school_distance',
'nearest_high_school_distance',
'nearest_park_distance',
'apt_deposit_rank', 'apt_area_deposit_rank',
'recent_deposit',
]

deposit mean: 39073.144531
local mae: 3889.70660
ai stage mae:
Feature Importance

13aek commented 1 month ago

XGBoost 11

data set: v4
change:
- 'contract_type' 원핫인코딩 (`contract_0', 'contract_1') 추가
- 'contract_date_numeric' 추가 (contract_year_month + contract_day)
- 'nearest_park_idx', 'nearest_school_idx', 'nearest_school_distance' 추가

columns:

'area_m2', 'contract_date_numeric',
'floor', 'built_year', 'latitude', 'longitude', 'age',
'contract_0', 'contract_1',
'deposit',
'apt_idx',
'area',
'grid_id',
'nearest_subway_distance',
'nearest_subway_idx',
'nearest_elementary_school_distance',
'nearest_middle_school_distance',
'nearest_high_school_distance',
'nearest_park_distance',
'apt_deposit_rank', 'apt_area_deposit_rank',
'recent_deposit',
'nearest_park_idx',
'nearest_school_idx',
'nearest_school_distance',
]

deposit mean: 39075.296875
local mae: 3886.03884
ai stage mae:
Feature Importance

13aek commented 1 month ago

XGBoost 12

data set: v4
change:
- 'contract_type' 원핫인코딩 (`contract_0', 'contract_1') 추가
- 'contract_date_numeric' 추가 (contract_year_month + contract_day)
- 'nearest_park_idx', 'nearest_school_idx', 'nearest_school_distance' 추가
- 학교별 거리 컬럼 삭제

columns:

'area_m2', 'contract_date_numeric',
'floor', 'built_year', 'latitude', 'longitude', 'age',
'contract_0', 'contract_1',
'deposit',
'apt_idx',
'area',
'grid_id',
'nearest_subway_distance',
'nearest_subway_idx',
'nearest_park_distance',
'apt_deposit_rank', 'apt_area_deposit_rank',
'recent_deposit',
'nearest_park_idx',
'nearest_school_idx',
'nearest_school_distance',
]

deposit mean: 39088.398438
local mae: 3895.29090
ai stage mae:
Feature Importance

13aek commented 1 month ago

XGBoost 12

data set: v4
change:
- 'contract_type' 원핫인코딩 (`contract_0', 'contract_1') 추가
- 'contract_date_numeric' 추가 (contract_year_month + contract_day)
- 'nearest_park_idx', 'nearest_school_idx', 'nearest_school_distance' 추가
- 'park_area' 추가

columns:

'area_m2', 'contract_date_numeric',
'floor', 'built_year', 'latitude', 'longitude', 'age',
'contract_0', 'contract_1',
'deposit',
'apt_idx',
'area',
'grid_id',
'nearest_subway_distance',
'nearest_subway_idx',
'nearest_park_distance',
'apt_deposit_rank', 'apt_area_deposit_rank',
'recent_deposit',
'nearest_park_idx',
'nearest_school_idx',
'nearest_school_distance',
'park_area',
]

deposit mean: 39106.253906
local mae: 3888.68319
ai stage mae: 3697.9864
Feature Importance

13aek commented 1 month ago

XGBoost 13

data set: v8
change:
- 'interest_rate' 추가 및 이전 3개월분의 금리 데이터 추가

columns:

'contract_date_numeric', 
'floor', 'built_year', 'latitude', 'longitude','age', 
'contract_0', 'contract_1', 
'deposit', 
'apt_idx', 
'area',
'grid_id', 
'nearest_subway_distance', 
'nearest_subway_idx',
'nearest_park_distance',
'apt_deposit_rank', 'apt_area_deposit_rank',
'recent_deposit', 
'nearest_park_idx', 
'nearest_school_idx',
'nearest_school_distance',
'park_area', 
'before_1month_interest','before_2month_interest', 'before_3month_interest',
'interest_rate',
]

deposit mean: 38481.238281
local mae: 3892.10695
ai stage mae: 3799.5995
Feature Importance

13aek commented 1 month ago

XGBoost 14

data set: v8
change:
- 'interest_rate' 추가 및 이전 6개월분의 금리 데이터 추가

columns:

'contract_date_numeric', 
'floor', 'built_year', 'latitude', 'longitude','age', 
'contract_0', 'contract_1', 
'deposit', 
'apt_idx', 
'area',
'grid_id', 
'nearest_subway_distance', 
'nearest_subway_idx',
'nearest_park_distance',
'apt_deposit_rank', 'apt_area_deposit_rank',
'recent_deposit', 
'nearest_park_idx', 
'nearest_school_idx',
'nearest_school_distance',
'park_area', 
'before_1month_interest','before_2month_interest', 'before_3month_interest',
'before_4month_interest', 'before_5month_interest','before_6month_interest',
'interest_rate',
]

deposit mean: 37937.738281
local mae: 3888.20883
ai stage mae: 3955.3361
Feature Importance

13aek commented 1 month ago

XGBoost 15

data set: temp_real_v8
change:
- 'interest_rate' 추가
- park, subway, school 관련 피쳐 추가

columns:

'contract_date_numeric', 
'floor', 'built_year', 'latitude', 'longitude','age', 
'contract_0', 'contract_1', 
'deposit', 
'apt_idx', 
'area',
'grid_deposit', 
'apt_deposit_rank', 'apt_area_deposit_rank',
'recent_deposit', 
'interest_rate', 
'grid_id', 
'nearest_park_distance','nearest_park_idx', 'park_grid_id', 'park_area',
'nearest_school_distance', 'nearest_school_idx', 'school_grid_id',
'nearest_subway_distance', 'nearest_subway_idx', 'subway_grid_id',
'park_count', 'school_count', 'subway_count'
]

deposit mean: 38758.683594
local mae: 3886.28229
ai stage mae: ****
Feature Importance

13aek commented 1 month ago

XGBoost 16

data set: temp_real_v8
change:
- 'interest_rate' 추가
- park, subway, school 관련 피쳐 추가
- timestamp 변환

columns:

'area_m2',
'floor', 'built_year', 'latitude', 'longitude','age', 
'contract_0', 'contract_1', 
'deposit', 
'apt_idx', 
'area',
'grid_deposit', 
'apt_deposit_rank', 'apt_area_deposit_rank',
'recent_deposit', 
'interest_rate', 
'grid_id', 
'nearest_park_distance','nearest_park_idx', 'park_grid_id', 'park_area',
'nearest_school_distance', 'nearest_school_idx', 'school_grid_id',
'nearest_subway_distance', 'nearest_subway_idx', 'subway_grid_id',
'park_count', 'school_count', 'subway_count',
'contract_timestamp',
]

deposit mean: 38829.437500
local mae: 3882.16348
ai stage mae: ****
Feature Importance

13aek commented 1 month ago

XGBoost 17

data set: temp_real_v8
change:
- num_boost_round: 500 -> 1000
- park, subway, school count 추가

columns:

'contract_date_numeric', 
'area_m2',
'floor', 'built_year', 'latitude', 'longitude','age', 
'contract_0', 'contract_1', 
'deposit', 
'apt_idx', 
'area',
'grid_deposit', 
'apt_deposit_rank', 'apt_area_deposit_rank',
'recent_deposit', 
'nearest_park_distance','nearest_park_idx', 'park_area',
'nearest_school_distance', 'nearest_school_idx',
'nearest_subway_distance', 'nearest_subway_idx',
'park_count', 'school_count', 'subway_count',
]

deposit mean: 39195.167969
local mae: 3800.06324
ai stage mae: ****
Feature Importance

13aek commented 1 month ago

XGBoost 18

data set: v9
change: num_boost_round 1000 -> 5000

columns:

'contract_date_numeric', 
'area_m2',
'floor', 'built_year', 'latitude', 'longitude','age', 
'contract_0', 'contract_1', 
'deposit', 
'apt_idx', 
'area',
'grid_deposit', 
'apt_deposit_rank', 'apt_area_deposit_rank',
'recent_deposit', 
'nearest_park_distance','nearest_park_idx', 'park_area',
'nearest_school_distance', 'nearest_school_idx',
'nearest_subway_distance', 'nearest_subway_idx',
'park_count', 'school_count', 'subway_count',
]

deposit mean: 39294.285156
local mae: 3674.86146
ai stage mae: 3550.1043
Feature Importance

13aek commented 1 month ago

XGBoost 19

data set: v9
change:
- train_test_split -> K-fold
- num_boost_round: 1000
- recent_deposit 결측값을 grid_deposit으로 처리

columns:

'contract_date_numeric', 
'area_m2',
'floor', 'built_year', 'latitude', 'longitude','age', 
'contract_0', 'contract_1', 
'deposit', 
'apt_idx', 
'area',
'grid_deposit', 
'apt_deposit_rank', 'apt_area_deposit_rank',
'recent_deposit', 
'nearest_park_distance','nearest_park_idx', 'park_area',
'nearest_school_distance', 'nearest_school_idx',
'nearest_subway_distance', 'nearest_subway_idx',
'park_count', 'school_count', 'subway_count',
]

deposit mean: 39146.307116
local mae: 3817.8985314416605
ai stage mae: 3610.6061
Feature Importance

13aek commented 1 month ago

XGBoost 20

data set: v9
change:
- train_test_split -> K-fold
- num_boost_round: 1000
- 결측값 미처리

columns:

'contract_date_numeric', 
'area_m2',
'floor', 'built_year', 'latitude', 'longitude','age', 
'contract_0', 'contract_1', 
'deposit', 
'apt_idx', 
'area',
'grid_deposit', 
'apt_deposit_rank', 'apt_area_deposit_rank',
'recent_deposit', 
'nearest_park_distance','nearest_park_idx', 'park_area',
'nearest_school_distance', 'nearest_school_idx',
'nearest_subway_distance', 'nearest_subway_idx',
'park_count', 'school_count', 'subway_count',
]

deposit mean: 39184.431767
local mae: 3812.8852546000844
ai stage mae: 3631.4194
Feature Importance

boostcampaitech7 / level2-competitiveds-recsys-06

[Model] XGBoost #47

XGBoost 1

XGBoost 2

XGBoost 3

XGBoost 4

XGBoost 5

XGBoost 6

XGBoost 6

XGBoost 7

XGBoost 8

XGBoost 9

241021 기준점

XGBoost 10

XGBoost 11

XGBoost 12

XGBoost 12

XGBoost 13

XGBoost 14

XGBoost 15

XGBoost 16

XGBoost 17

XGBoost 18

XGBoost 19

XGBoost 20