search

Sara-Ahmed / SiT

Self-supervised vIsion Transformer (SiT)
324 stars 49 forks source link

How to see the image reconstruction task results #16

Closed E-Loba closed 3 years ago

E-Loba commented 3 years ago

The usage example shows how to finetune the classifier head of the model in the command line, but I'm not sure how to get the reconstructed image from this output. Can you please provide a code sample for image reconstruction? Which part of the model output can be used to visually represent inference results like in the diagram?

Sara-Ahmed commented 3 years ago

Dear E-loba,

the forward function in the vision transformer class returns x_rec as well, which is the reconstructed image.

E-Loba commented 3 years ago

I see! Thank you very much. Should any additional transformation be applied to the output, just in case?

Sara-Ahmed commented 3 years ago

You are welcome. No transformation is required, x_rec is a tensor that suppose to be directly similar to x which is the input Image

Please refer to evaluate_SSL() function. https://github.com/Sara-Ahmed/SiT/blob/1aacd6adcd39b71efc903d16b4e9095b97dda76f/engine.py#L272

E-Loba commented 3 years ago

I'll check it out. Appreciate the swift reply and thank you for sharing the code!