Open voetberg opened 11 months ago
Progress update:
Yayyyy it works
Meeting questions:
I was missing the backwards pass of the message passing - here it is.
Curious as to why the results changed so dramatically. I did have some shape problems I didn't have last week, I should verify the load method is picking up the network properly.
Okay, this one makes much more sense.
Something to note, the linear probes are untrained, so the loss doesn't converge to 0. I'm going to reread the paper this concept is based off of and verify that this makes sense.
Corrected probe shapes, it all makes sense now. Training the probes itself is a little more debatable.
Question to investigate:
While the wc cluster is down, I'm testing this with a Single data point. Loss here is binary cross entropy loss for the probe output for a point in the network. The three different planes are averaged. (Would it make more sense to split them up? Maybe).
After discussion in the workshop - it would be advantageous to be able to monitor the message passing after each iteration