search

visionml / pytracking

Visual tracking library based on PyTorch.
GNU General Public License v3.0
3.21k stars 605 forks source link

The code for :Know Your Surroundings #150

Closed xxAna closed 4 years ago

xxAna commented 4 years ago

Thank you for your share. It is a good job! And I wonder weather the code for "Know Your Surroundings: Exploiting Scene Information for Object Tracking" will alse be released in this project? As I notice that it is mentioned in the paper you will release the code upon publication and it was accepted by CVPR2020 now. Waiting for your reply and thank you again. Best wishes!

martin-danelljan commented 4 years ago

Hi. Yes, the code will be released here. But please have patience :) We hope to finish it within a month or so. /Martin

xxAna commented 4 years ago

Get it. Thank you for your reply. Not rush. Looking forward to your work. That's excellent!

xxAna commented 4 years ago

@martin-danelljan Hi~ When I read the paper "Know Your Surroundings", I got a question. Did the states actually act as a special spatial attention and have the same size of Appearance model score and Target confidence score? Please correct me if I am wrong. Waiting for your reply. Best wishes.

goutamgmb commented 4 years ago

The states have the same spatial size as the feature map used for target classification (i.e. same size as the appearance model score and target confidence score), but have 8 channels. They do not act as spatial attention. Rather they encode information for each cell in the feature map, e.g. whether that cell corresponds to the target or background. They are concatenated with the appearance model score and passed through the predictor module to obtain the fused target confidence scores.

xxAna commented 4 years ago

ok. I get it. That's clear now. Thank you so much! @goutamgmb

xxAna commented 4 years ago

Hello. I get another question about 'Know your surroundings' so I reopen the issue. I wonder how you process your training data. As we konw, in common appearance models, they will crop the image to 255*255 or other fixed size to train the model. While if we want the model to learn some massage passing information, should we preprocess the data? Will you share how you process the data to train such a model? Please correct me if I am wrong. Waiting for your reply. Best wishes.

goutamgmb commented 4 years ago

Hi,

Yes, we preprocess the data in a similar way as in DiMP or ATOM. We will release the complete training and inference code, including the preprocessing.

xxAna commented 4 years ago

OK. I get it. Thank you for your patience! Would you mind I keep this issue open for questions about the paper "Know your surroundings" until you release its code? I think it was an excellent job and maybe someone else will be interested in it too. ^-^

xxAna commented 4 years ago

Hi. Yes, the code will be released here. But please have patience :) We hope to finish it within a month or so.

Hi~July is coming. Looking forward to your codes. It do an excellent job.^^

xxAna commented 4 years ago

Hello.So sorry to bother you.I wonder what the state propagation module in the "know your surroundings" consists of ? Does it a CNN or LSTM? Waiting for your reply. Best wishes.

goutamgmb commented 4 years ago

Hi,

The state propagation module is a feedforward neural network which estimates dense correspondence between consecutive frames and warps the state vectors using the estimated correspondence. The state update module on the other hand consists of convolutional gate recurrent unit (ConvGRU).

xxAna commented 4 years ago

Get it. Thank you for your reply.

goutamgmb commented 4 years ago

Hi,

The code for KYS has been released now.