Open UltraCoderRU opened 4 years ago
I seem to experience the exact same error, Is there a way to solve this problem?
I have the same warning too
It seems like something related with the value of "kBatchSize". According to the warning, the output expect [64,1,1,64], the last number 64 should stand for the batch size. I changed the value of kBatchSize from 64 to 1, and the warning disappeared. But I am not familiar with ATen, could someone tell me the root cause?
Setting the batch size to 1 will work, however it will significantly slow down GPU training. Reshaping network outputs (which get a bogus second dimension of size one) just so allows training with batches larger than one:
for (torch::data::Example<>& batch : *data_loader) {
const auto batch_size = batch.data.size(0);
// Train discriminator with real images.
discriminator->zero_grad();
torch::Tensor real_images = batch.data.to(device);
torch::Tensor real_labels =
torch::empty(batch_size, device).uniform_(0.8, 1.0);
torch::Tensor real_output =
discriminator->forward(real_images).reshape({batch_size});
torch::Tensor d_loss_real =
torch::binary_cross_entropy(real_output, real_labels);
d_loss_real.backward();
// Train discriminator with fake images.
torch::Tensor noise =
torch::randn({batch.data.size(0), kNoiseSize, 1, 1}, device);
torch::Tensor fake_images = generator->forward(noise);
torch::Tensor fake_labels = torch::zeros(batch.data.size(0), device);
torch::Tensor fake_output =
discriminator->forward(fake_images.detach()).reshape({batch_size});
torch::Tensor d_loss_fake =
torch::binary_cross_entropy(fake_output, fake_labels);
d_loss_fake.backward();
torch::Tensor d_loss = d_loss_real + d_loss_fake;
discriminator_optimizer.step();
// Train generator.
generator->zero_grad();
fake_labels.fill_(1);
fake_output = discriminator->forward(fake_images).reshape({batch_size});
torch::Tensor g_loss =
torch::binary_cross_entropy(fake_output, fake_labels);
g_loss.backward();
generator_optimizer.step();
batch_index++;
if (batch_index % kLogInterval == 0) {
std::printf("\r[%2ld/%2ld][%3ld/%3ld] D_loss: %.4f | G_loss: %.4f\n",
epoch, kNumberOfEpochs, batch_index, batches_per_epoch,
d_loss.item<float>(), g_loss.item<float>());
}
Call .reshape({batch_size})
on results of the forward
calls.
@a-pushkin Reshape works, thank you.
reshape(fake_labels.sizes()) would be better in case that batch.data.size(0) != batch_size
@a-pushkin would you like to make a PR with your fix?
After PyTorch update I get following message while running sample DCGAN C++ application:
[W Resize.cpp:19] Warning: An output with one or more elements was resized since it had shape [64, 1, 1, 1], which does not match the required output shape [64, 1, 1, 64].This behavior is deprecated, and in a future PyTorch release outputs will not be resized unless they have zero elements. You can explicitly reuse an out tensor t by resizing it, inplace, to zero elements with t.resize_(0). (function resize_output)
Message goes from calls to
Tensor::backward()
. It seems like this warning is a result of changes in 2f840b166. I don't know if it's a PyTorch or sample application issue.