About Partial Zip

[ycjing]

Hi!

Thank you for the great work! May I ask how to run the experiments for partial zip? Thanks so much!

Cheers,

[gstoica27]

Hi,

I'm so sorry for the lateness of response. I kept missing to get back to you. I really apologize.

Absolutely. Partial zipping involves specifying a node to "stop" zipping at, labeled "stop_at" in our repository. For instance, you can run a partial zipping experiment by replacing "21" with another number on line 100 of "/non_imnet_evaluation_scripts/zipit_concept_merging.py":

        {'stop_node': 21, 'params':{'a': .0001, 'b': .075}},

Here, the "stop_node" will set 21 to be the value of the 'stop_at" argument in line 70 of the same script.

For our ResNet models, you can run a partial zipping experiment by stopping at node 21 (i.e. zipping until layer 7/20), 42 (layer 13/20), 63 (layer 19/20), or None (full merge).

More generally however, you can run a partial zip operation by specifying the stop_at to be any node labeled with "PREFIX" or "POSTFIX" in the model graph, such that all computational flow up to that node goes through it (i.e., it does not lie on a residual branch). For ResNet these are exactly the nodes 21, 42, and 63 (can be seen by looking at the graph obtained by running the "graphs/resnet_graph.py" script).

Hope this helps - so sorry again for such a late response!

Please let me know if you have any more questions, and feel free to email me as well! I will respond significantly more promptly.

EDIT: Added the resnet graph visualization to illustrate the characteristics of possible partial merging locations:

[ycjing]

Hi @gstoica27

Thank you for your incredibly comprehensive and helpful responses! I deeply appreciate it. Also, thanks again for the fantastic work!

Cheers, Yongcheng