Transformers in remote sensing | Blog of Robin Cole

[utterances-bot]

Transformers in remote sensing | Blog of Robin Cole

Introduction to Transformers and why you should care

https://robmarkcole.com/markdown/2022/08/15/transformers.html

[wgcban]

Nice article & thanks. Here is the another recent work on transformers for remote sensing change detection: https://github.com/wgcban/ChangeFormer. We simplify the attention mechanism using sequence reduction process, so that it is much faster to train. Further, by design, it utilizes convolutions for positional embedding, which allows images with different resolution during testing.

[robmarkcole]

Thanks for sharing and providing a nice summary. You touch on an interesting technical point, the use of positional embeddings typically limits the range of resolutions that can be used with a transformer?