Should the position of this code be adjusted?

[wpumain]

https://github.com/Nanne/pytorch-NetVlad/blob/8f7c37ba7a79a499dd0430ce3d3d5df40ea80581/main.py#L89

  with h5py.File(train_set.cache, mode='w') as h5: 
            pool_size = encoder_dim
            if opt.pooling.lower() == 'netvlad': pool_size *= opt.num_clusters
            h5feat = h5.create_dataset("features", 
                    [len(whole_train_set), pool_size], 
                    dtype=np.float32)
            with torch.no_grad():
                for iteration, (input, indices) in enumerate(whole_training_data_loader, 1):
                    input = input.to(device)
                    image_encoding = model.encoder(input)
                    vlad_encoding = model.pool(image_encoding) 
                    h5feat[indices.detach().numpy(), :] = vlad_encoding.detach().cpu().numpy()
                    del input, image_encoding, vlad_encoding

In the main.py file, the above code is used to encode the images in the training set into a vector to represent, should it be placed above the for loop statement below instead of inside the for loop below? https://github.com/Nanne/pytorch-NetVlad/blob/8f7c37ba7a79a499dd0430ce3d3d5df40ea80581/main.py#L85 for subIter in range(subsetN):

[Nanne]

I'm not sure I understand what the question is. At the start of each subIteration the trainset is first encoded intov ectors and saved, and then the actual training happens. As such all of these things should happen inside the loop - hence it is like that in the code.

Why do you think this should be different?

[wpumain]

If the above code is not changed, this statement 【pytorch-NetVlad/main.py】 will be executed under each for loop 【pytorch-NetVlad/main.py】, but the results of executing this statement are the same each time. Why not place this statement 【pytorch-NetVlad/main.py】 above the for loop 【pytorch-NetVlad/main.py】, so that each iteration can share the computed result of this statement and avoid unnecessary duplicate computations?

[Nanne]

The result isn't the same each time, because the model gets updated. We repeat this action to refresh the cache.

To be able to do tasks such as hard negative mining we need to cache the training set, which is what is constructed there. However, if we construct the cache once - whilst contuining to update the model then the cache will get less useful as the feature representations drift apart. Instead, we every now and then update the cache by repeating the loop.

[wpumain]

Thank you very much for your help. Based on your guidance, I think I understand it now. Do you see if this is the meaning? The training is done in batches, and each time the model parameters are updated through backpropagation, the "descriptors" cache, 'centroids' cache, and "features" cache will all change. The "descriptors" cache and 'centroids' cache use the model.encoder, while the "features" cache uses both the model.encoder and the NetVlad layer. However, only the "features" cache needs to be updated, and not the "descriptors" cache and 'centroids' cache. The 'centroids' cache is calculated from the "descriptors" cache and serves as the initial value for the NetVlad model parameters. The NetVlad model parameters will be updated during training, but the purpose of the "descriptors" cache and 'centroids' cache is only to serve as the initial values, so they only need to be calculated at the beginning and not updated during training. The "features" cache is directly calculated by the model.encoder and NetVlad network, and it determines the selection of positive and negative samples. Every batch of training updates the model parameters, causing the "features" value to change, which in turn affects the selection of positive and negative samples in the next batch of training. Therefore, the "features" cache needs to be updated in every batch of the 8 batches of training. Note that in order to reduce computational cost, it is not necessary to update it in every batch of the training_data_loader, but only needs to be updated before generating each batch of the training_data_loader within the 8 batches of loop.

[wpumain]

Thank you for your help. I understand now. Thank you again for your help

---

发件人: Nanne @.> 发送时间: 2023年2月27日 0:39 收件人: Nanne/pytorch-NetVlad @.> 抄送: wpumain @.>; Author @.> 主题: Re: [Nanne/pytorch-NetVlad] Should the position of this code be adjusted? (Issue #82)

The result isn't the same each time, because the model gets updated. We repeat this action to refresh the cache.

To be able to do tasks such as hard negative mining we need to cache the training set, which is what is constructed there. However, if we construct the cache once - whilst contuining to update the model then the cache will get less useful as the feature representations drift apart. Instead, we every now and then update the cache by repeating the loop.

― Reply to this email directly, view it on GitHubhttps://github.com/Nanne/pytorch-NetVlad/issues/82#issuecomment-1445404876, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A5FNGT3GVDI5TCP25AEOEV3WZOBN5ANCNFSM6AAAAAAVIKHS5I. You are receiving this because you authored the thread.Message ID: @.***>