Open m-schier opened 1 year ago
Hi Maximilian,
Yes, there are some mistakes that happen when I clean the code for GitHub Release. Thank you so much for pointing it out! (e.g., the l3 and l2 accidentally swapped since we swapped it in the encoder when the code is not for release)
The l4 is based on l1_norm is one of our experiment tricks, where we directly decode the representation from layer 4 to layer 1, in order to have faster training and coverage, which has also been proven effective and theoretically make sense in our case. Since we are not reconstructing the neighborhoods layer by layer but decoding the last layer node encoding to its neighborhood in different layers (as shown in our paper Pg. 5). And that says, self.layer1_generator is not necessary too, and that's also why the current code still has the State-of-the-arts results since decode to which layer (e.g., 4->1, or 4->3->2->1) doesn't make a huge difference according to our main idea and experiments. The previous one required longer training time and more epochs. But to be consistent with our paper's main procedure, I cleaned the code for the final release. I'll change them accordingly very soon.
Thank you so much, and hope I answered some of your questions. Let me know if you have further questions.
Best, Ming
Hello,
I have some questions regarding your model implementation from
src/model.py
. Starting off with the forward encoder:This code seems to implement the forward encoding with fixed k=4. However I do not understand why
l4
is not based onl3_norm
but onl1_norm
. Why doesl3_norm
not callself.norm
, but just ReLU? Why isl3
calculated usingl2
, but notl2_norm
?Next, the function returns the tuple
l4, l3_norm, l2_norm, l1_norm, h0
, but it is assigned inforward()
as:Are
l3
andl2
accidentally swapped?Finally,
self.layer1_generator
is initialized but appears to be never used. Isn't the model lacking the neighborhood decoder for the first convolution then?Thank you for considering my questions, Kind regards, Maximilian Schier