Add percent pruned before after winnery

dorimedini / robustness_properties

GLOBIGLOBIGLOBI

0 stars 0 forks source link

Add percent pruned before after winnery #102

Closed dorimedini closed 5 years ago

dorimedini commented 5 years ago

Results! ...are weird, I'm assuming initial weight sampling is not uniform in some interval? Different layers are given different-magnitude random initial weights? Otherwise we should see a fixed percentage of each layer (the threshold percent) @noamloya @galshachaf thoughts?

Addresses #97

galshachaf commented 5 years ago

@dorimedini are you sure that you are returning the percentage? please not that the layers are different in size: the first layer is biggest (FC from 32*32 to h_dim), two middle ones are h_dim to h_dim, and the last one is smallest (h_dim to 10). Maybe you are not calculating the total number of edges properly?

dorimedini commented 5 years ago

@dorimedini are you sure that you are returning the percentage? please not that the layers are different in size: the first layer is biggest (FC from 32*32 to h_dim), two middle ones are h_dim to h_dim, and the last one is smallest (h_dim to 10). Maybe you are not calculating the total number of edges properly?

This explains why the results make sense... I returned the negation of the correct value in total_edges_in_layers and then computed the negation of what I meant when I wrote pruned_percents.append(total_nonzero / total) :D fixing...

dorimedini commented 5 years ago

@dorimedini are you sure that you are returning the percentage? please not that the layers are different in size: the first layer is biggest (FC from 32*32 to h_dim), two middle ones are h_dim to h_dim, and the last one is smallest (h_dim to 10). Maybe you are not calculating the total number of edges properly?

Fixed

galshachaf commented 5 years ago

@dorimedini how are the new graphs looking?

dorimedini commented 5 years ago

@dorimedini how are the new graphs looking?

@galshachaf with 30% prune threshold, these are the results. I'll open a task to show the accuracy of the trained+unpruned model vs. the trained+pruned model to make sure I'm not pruning too much / not enough