num_iter/recompute_mean in DataAugmentationLayer

[sampepose]

I am running the FlowNetC deploy model.

What is the purpose of num_iter (aka blobs_[0]) in the DataAugmentationLayer?

From what I can tell, num_iter is initialized to 0 and is incremented with each forward pass of the layer. If num_iter <= recompute_mean, then the mean of the current batch is computed. Otherwise, we don't recalculate the mean, and just use an old mean? Why?

Also, I've found that the code to recompute the mean is never actually run. I placed a LOG(INFO) after if (num_iter <= aug.recompute_mean()) { in data_augmentation_layer.cu and it is never logged. Why is this happening?

"recovered iteration count 1.9491e+06" is logged from data_augmentation_layer.cpp. I don't understand how that number can be so high, we're only doing 1 forward GPU passes per image through the DataAugmentationLayer.

Thanks!

[sampepose]

Just ran this in training mode, and num_iter increments from 0 as expected, so the image mean is calculated correctly.

I've found in data_augmentation_layer.cpp, when adjust_blobs is called in a forward pass, the value coped to blob0 is incorrect.

LOG(INFO) << "before: " << (this->blobs_[0]->mutable_cpu_data())[0];
LOG(INFO) << "assignment: " << (blobs[0]->mutable_cpu_data())[0];
this->blobs_[0]->CopyFrom(*blobs[0],false,true); // Line 172
LOG(INFO) << "after: " << (this->blobs_[0]->mutable_cpu_data())[0];

outputs:

before: 0
assignment: 1.9491e+06
after: 1.9491e+06

[nikolausmayer]

Hi Sam,

• num_iter simply counts the iterations that the layer has seen.

• The "recovered iteration count" is high because it's part of the data augmentation layer's data saved in the weights file. You see a high number during testing because the number "remembers" the entire training.

• The recompute_mean is just an empirical mean for the dataset where we want to avoid updating the mean every iteration throughout training, but we also want to avoid precomputing the mean over the entire dataset.

• Hm, it looks like the assignment is correct? Maybe I am misunderstanding something?

Best, Nikolaus

[eddy-ilg]

The mean is "learned" that is, during the first 1000 iterations (parameter recompute_mean) the network computes a running average from the data (this is the learning phase). After that the mean is fixed and never changed again. When loading an existing snapshot the saved mean is used.

[sampepose]

Thanks! Makes sense!