Approach the problem from an information theory angle

Tensor-Reloaded / Convergence

Artificial Neural Network training convergence boosting by smart data ordering

Academic Free License v3.0

0 stars 0 forks source link

Approach the problem from an information theory angle #10

Open simi2525 opened 3 years ago

simi2525 commented 3 years ago

Attempt an OED approach to sampling/sorting the batches. https://docs.pyro.ai/en/dev/contrib.oed.html#module-pyro.contrib.oed.eig

Either by calculating the estimated information gains, information gains directly or the mutual information between samples to form batches: https://github.com/gtegner/mine-pytorch