search

automl / NASLib

NASLib is a Neural Architecture Search (NAS) library for facilitating NAS research for the community by providing interfaces to several state-of-the-art NAS search spaces and optimizers.
Apache License 2.0
512 stars 117 forks source link

XGBoost experiments generalization #181

Closed AnaRisnoveanu closed 6 months ago

AnaRisnoveanu commented 7 months ago

This question is more about what the paper ("NAS-Bench-Suite-Zero: Accelerating Research on Zero Cost Proxies") describes in Table 10 from Appendix D. It is still unclear to me whether the XGBoost was trained on ZC scores and/or encodings on one single benchmark and then tested on the rest of the benchmarks or XGBoost was trained on a subset of benchmark models and later tested on models from the same benchmark.

Neonkraft commented 6 months ago

Hi @AnaRisnoveanu!

Thank you for your interest in our paper! The XGBoost was trained on a subset of models from one benchmark, and then tested on other models from the same benchmark. Since the encodings are searchspace-specific, we can't train on one and test on another.

I hope this helps!

Best, Arjun

AnaRisnoveanu commented 6 months ago

Thank you for the clarification!