search

varungohil / Generalizing-Lottery-Tickets

This repository contains code to replicate the experiments given in NeurIPS 2019 paper "One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers"
https://rescience.github.io/bibliography/Gohil_2020.html
MIT License
51 stars 9 forks source link

Understanding the pipeline to generate the Winning Tickets #6

Closed whxndxll closed 4 years ago

whxndxll commented 4 years ago

Hi, congrats for your work!

I took a look athe the scripts and became confused about the entire pipeline to generate winning tickets from scratch.

  1. Where the final weights of the pruned model are being used? The final weights are being used to define the weights to be pruned?
  2. When the model is loaded on iterative_pruning.py, it is loaded with random_weights?
  3. Why only pass the path to the init_weights on iterative_pruning.py?

Waiting for the answers.

sdeepaknarayanan commented 4 years ago

Hi @whxndxll, Thank you for your interest in our work.

We hope to get back to you by the end of the week. Hope you are safe during these uncertain times.

whxndxll commented 4 years ago

Reading the code again i found the answer. In pruning iteration 0 the model is trained again without pruning and the epoch 160 is saved.

And from pruning iteration 1 onward, the masks are generated considering the model saved at the immediately earlier iteration. Because of this, the final weights generated using train.py are useless, since new final weights will be generated in iteration 0.

whxndxll commented 4 years ago

Hi @whxndxll, Thank you for your interest in our work.

We hope to get back to you by the end of the week. Hope you are safe during these uncertain times.

Hope that all of you are safe too.