search

Guang000 / Awesome-Dataset-Distillation

A curated list of awesome papers on dataset distillation and related applications.
https://guang000.github.io/Awesome-Dataset-Distillation/
MIT License
1.41k stars 132 forks source link

Two related papers #7

Closed yongchaoz closed 2 years ago

yongchaoz commented 2 years ago

Hello, Bo, Guang, and Tongzhou.

Thanks for this fantastic repository! The two papers below, in my opinion, should be included in this repository.

  1. Optimizing Millions of Hyperparameters by Implicit Differentiation: This paper proposes the use of Implicit Differentiation, a general gradient-based hyperparameter optimization method, for dataset distillation tasks (Section 5.3).
  2. On Implicit Bias in Overparameterized Bilevel Optimization: The dataset distillation is covered in this study from a general bi-level optimization perspective. It provides some valuable insights into the implicit bias in dataset distillation. We may want to create a "Theory" section for this paper.

Best, Yongchao

Guang000 commented 2 years ago

Hi, Yongchao.

Thank you for your valuable suggestion! I will discuss it with Bo and Tongzhou. We will reply as soon as possible.

Best, Guang

Guang000 commented 2 years ago

After discussion, we decided to add them to the Optimization section. Thanks for your contribution.