search

eeishaan / pretrain-gnns

1 stars 0 forks source link

Task 4: Implement PackNet and generate numbers for different thresholds #4

Open eeishaan opened 4 years ago

eeishaan commented 4 years ago
  1. Implement PackNet that whitens some weights that are less than a threshold.
  2. Freeze the remaining weights and train the 2nd stage that only trains the whitened weights.
  3. Fine-tune all weights for the downstream tasks and report performance for downstream tasks.
  4. Repeat the experiment with different threshold values.
eeishaan commented 4 years ago

Result for packnet: ~~Prune ratio: 0.25, final result: 0.666 +- 0.12. 67.37866578544747 +- 0.0 Prune ratio: 0.5, final result: 65.51061201715957 +- 0.0 Prun ration: 0.75 final result: 66.97552411391922 +- 0.0~~ finetuned_masking_0.75 71.70223739331004 +- 0.0

finetuned_masking_0.5 68.45754855903431 +- 0.0

finetuned_masking_0.25 66.35125719916452 +- 0.0

eeishaan commented 4 years ago

finetuned_masking_0.95 72.37336584548996 +- 0.0

eeishaan commented 4 years ago

finetuned_masking_0.99 70.4483470734903 +- 0.0

eeishaan commented 4 years ago

finetuned_contextpred_0.25 70.74098827319723 +- 0.0 finetuned_contextpred_0.5 64.7980447318762 +- 0.0 finetuned_contextpred_0.75 68.49372770357188 +- 0.0 finetuned_contextpred_0.95 70.65757145397369 +- 0.0

eeishaan commented 4 years ago

finetuned_contextpred_0.99 74.16079900638317 +- 0.0