search

Seondong / Customs-Fraud-Detection

Simulation framework for customs fraud detection using import declarations.
MIT License
30 stars 4 forks source link

Experiment plan #88

Open john-mai-2605 opened 3 years ago

john-mai-2605 commented 3 years ago

Building experiment plan and realize with bash code

john-mai-2605 commented 3 years ago

Main settings

Main data: t-data Main subsamplings: xgb/random

Expected work:

Code Analysis ipynbs

Expected results:

Weekly performance Weekly weights (+ada pmf)

Experiments

Exp 1: Tuning adahybrid (ada-)

Data: data-t Subsamplings: xgb/random By independent search or grid search (grid might be too time-consuming) Hyperparam: Level of randomness, learning rate, decay rate

Exp 2: Main exp: (main-)

Data: data-t Subsamplings: xgb/random Strategies: pot, pv, ada, csi, kats Hypothesis: Approaching or outperforming hybrid ada+pot > ada, pot > best hybrid

2.1. Adaptive with performance signal (main-ada-)

best of exp 1

2.2. Adaptive with concept drift signal (main-cd-)

pot, csi, pv, kats (DriftHybrid class)

2.3. Adaptive with both signals (main-all-)

ada+pot, ada+csi, ada+pv, ada+kats Method 1: use integrated signals (AdaHybrid class with modified signal) Method 2: use concept drift to reinit ada (RegulatedAdaHybrid class)

Exp 3: Ablation: (ablation-)

(ablation-data-) different data (ablation-subs-) DATE/bATE on data-t